Skip to content
This repository was archived by the owner on Jan 7, 2023. It is now read-only.

Commit bebf06a

Browse files
astojiljastojilj
committed
Typing in the air tutorial code added with initial documentation.
1 parent abdd324 commit bebf06a

16 files changed

+1070
-99
lines changed

‎README.md

+21-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,24 @@
11
# Depth camera capture in HTML5
22

3-
![Alt text](/how_the_demo_looks.gif?raw=true "Depth camera capture demo")
3+
<table cellspacing="0" cellpadding="0" style="border-collapse: collapse; border: none;">
4+
<tr>
5+
<td align="center" valign="center">
6+
<img src="how_the_demo_looks.gif" alt="how_the_demo_looks.gif is not yet loaded." style="height:400px;width:452px;"/>
7+
<br />
8+
<p>HTML5 Depth Capture tutorial shows how to access depth stream, check the <a href="https://01.org/chromium/blogs/astojilj/2017/depth-camera-capture-html5">tutorial text</a> or <a href="depthdemo.html">run the live demo here.</a></p>
9+
</td>
10+
<td align="center" valign="center">
11+
<img src="typing_in_the_air/typing_in_the_air.gif" alt="typing_in_the_air.gif is not yet loaded." style="height:400px;width:702px;"/>
12+
<br />
13+
<p>Typing in the air tutorial shows how to use depth stream and WebGL transform feedback to do simple gesture recognition. Check the <a href="typing_in_the_air/doc/tutorial.html">tutorial text</a> and <a href="typing_in_the_air/front_capture_typing.html">run the live demo here.</a></p>
14+
</td>
15+
<td align="center" valign="center">
16+
<img src="https://github.com/01org/depthcamera-pointcloud-web-demo/raw/master/recording.gif" alt="https://github.com/01org/depthcamera-pointcloud-web-demo/raw/master/recording.gif is not yet loaded." style="height:400px;width:422px;"/>
17+
<br />
18+
<p>3D point cloud rendering demo shows how to render and synchronize depth and color video on GPU. <a href="https://01org.github.io/depthcamera-pointcloud-web-demo/">Run the live demo here.</a></p>
19+
</td>
20+
</tr>
21+
</table>
422

523
To capture and manipulate depth camera stream in HTML5, you'll need:
624
* Chrome browser version 58 or later (no need for additional extensions),
@@ -15,7 +33,9 @@ These are the constraints of current implementation. The plan is to support othe
1533
An explanation on how to use the depth camera is in the article
1634
[Depth Camera Capture in HTML5](https://01.org/chromium/blogs/astojilj/2017/depth-camera-capture-html5).
1735

36+
1837
The example code here covers:
38+
1939
* Displaying depth video stream in <video> element
2040
* uploading depth frame data to WebGL texture,
2141
* enumerating depth frame data pixel values in loop.

‎depth-camera.js

+88-95
Original file line numberDiff line numberDiff line change
@@ -14,110 +14,98 @@
1414
// See the License for the specific language governing permissions and
1515
// limitations under the License.
1616

17-
DepthCamera.getDepthStream = function () {
18-
if (!navigator.mediaDevices || !navigator.mediaDevices.enumerateDevices ||
19-
!navigator.mediaDevices.getUserMedia)
20-
return Promise.reject("Your browser doesn't support the mediaDevices API.");
17+
DepthCamera.getDepthStream = async function () {
18+
if (!navigator.mediaDevices || !navigator.mediaDevices.enumerateDevices ||
19+
!navigator.mediaDevices.getUserMedia)
20+
throw new Error("Your browser doesn't support the mediaDevices API.");
2121

22-
return new Promise(async function(resolve, reject) {
23-
try {
24-
const constraints = {
25-
audio: false,
26-
video:{
27-
// We don't use videoKind as it is still under development.
28-
// videoKind: {exact:"depth"},
29-
// R200 related hack: prefer depth (width = 628) to IR (width = 641)
30-
// stream.
31-
width: {ideal:628, max:640},
22+
let constraints = {
23+
audio: false,
24+
video: {
25+
// We don't use videoKind as it is still under development.
26+
// videoKind: {exact:"depth"}, R200 related hack: prefer
27+
// depth (width = 628) to IR (width = 641) stream.
28+
width: { ideal: 628, max: 640 },
3229

33-
// SR300 depth camera enables capture at 110 frames per second.
34-
frameRate: {ideal:110},
30+
// SR300 depth camera enables capture at 110 frames per
31+
// second.
32+
frameRate: { ideal:110 },
3533
}
36-
}
37-
const stream = await navigator.mediaDevices.getUserMedia(constraints);
38-
const track = stream.getVideoTracks()[0];
39-
if (track.label.indexOf("RealSense") == -1) {
34+
}
35+
let stream = await navigator.mediaDevices.getUserMedia(constraints);
36+
let track = stream.getVideoTracks()[0];
37+
if (track.label.indexOf("RealSense") == -1) {
38+
throw new Error(chromeVersion() < 58 ?
39+
"Your browser version is too old. Get Chrome version 58 or later." :
40+
"No RealSense camera connected.");
41+
}
4042

41-
function getChromeVersion () {
42-
var raw = navigator.userAgent.match(/Chrom(e|ium)\/([0-9]+)\./);
43-
return raw ? parseInt(raw[2], 10) : false;
43+
if (track.getSettings && track.getSettings().frameRate > 60) {
44+
// After Chrome 59, returned track is scaled to 628 and frameCount 110.
45+
// We got the deviceId, so we the deviceId to select the stream with
46+
// default resolution and frameRate.
47+
track.stop();
48+
constraints = {
49+
audio: false,
50+
video: {
51+
deviceId: {exact: track.getSettings().deviceId},
52+
}
4453
}
45-
46-
return reject(getChromeVersion() < 58 ?
47-
"Your browser version is too old. Get Chrome version 58 or later." :
48-
"No RealSense camera connected.");
49-
}
50-
return resolve(stream);
51-
} catch(e) {
52-
return reject("Error getting the depth camera stream:" + e);
54+
stream = await navigator.mediaDevices.getUserMedia(constraints);
55+
track = stream.getVideoTracks()[0];
5356
}
54-
});
57+
return stream;
5558
}
5659

5760
// Call the method after getting depth_stream using getDepthStream.
58-
DepthCamera.getColorStreamForDepthStream = function(depth_stream,
59-
ideal_width = undefined) {
60-
// To get color stream from the same physical device providing the depth
61-
// stream, we will use groupId, once it is implemented:
62-
// See https://crbug.com/627793
63-
// For now, enumerate devices based on label.
64-
// Note: depth_stream is not used, for now, but deliberatelly added as a
65-
// parameter to mandate the need for previous call to getDepthStream.
66-
return new Promise(function(resolve, reject) {
67-
navigator.mediaDevices.enumerateDevices()
68-
.then(function(all_devices) {
69-
70-
let depth_device_id = null;
71-
const depth = depth_stream.getVideoTracks()[0];
72-
// Chrome, starting with version 59, implements getSettings() API.
73-
if (depth.getSettings) {
61+
DepthCamera.getColorStreamForDepthStream = async function(depth_stream,
62+
ideal_width = undefined) {
63+
// To get color stream from the same physical device providing the depth
64+
// stream, we will use groupId, once it is implemented:
65+
// See https://crbug.com/627793
66+
// For now, enumerate devices based on label.
67+
// Note: depth_stream is not used, for now, but deliberatelly added as a
68+
// parameter to mandate the need for previous call to getDepthStream.
69+
var all_devices = await navigator.mediaDevices.enumerateDevices();
70+
let depth_device_id = null;
71+
const depth = depth_stream.getVideoTracks()[0];
72+
// Chrome, starting with version 59, implements getSettings() API.
73+
if (depth.getSettings) {
7474
depth_device_id = depth.getSettings().deviceId;
75-
} else if (ideal_width) {
76-
console.warn("Not able to set ideal width for color video as \
77-
MediaStreamTrack getSettings() API is not available. Try with \
78-
Chromium version > 59.");
79-
}
80-
81-
82-
const devices = all_devices
75+
} else if (ideal_width) {
76+
console.warn(`Not able to set ideal width for color video as
77+
MediaStreamTrack getSettings() API is not available. Try
78+
with Chromium version > 59.`);
79+
}
80+
const devices = all_devices
8381
.filter((device) => (device.kind == "videoinput" &&
84-
device.label.indexOf("RealSense") !== -1 &&
85-
device.deviceId != depth_device_id));
86-
if (devices.length < 1) {
87-
return reject("No RealSense camera connected.");
88-
}
82+
device.label.indexOf("RealSense") !== -1 &&
83+
device.deviceId != depth_device_id));
84+
if (devices.length < 1) {
85+
throw new Error("No RealSense camera connected.");
86+
}
87+
// Select streams from these ids, so that some other camera doesn't get
88+
// selected (e.g. if the user has another rgb camera).
89+
const ids = devices.map((device) => device.deviceId);
8990

90-
// Select streams from these ids, so that some other camera doesn't get
91-
// selected (e.g. if the user has another rgb camera).
92-
const ids = devices.map((device) => device.deviceId);
93-
94-
// Select color stream.
95-
// Color stream tracks have larger resolution than depth stream tracks.
96-
// If we cannot use deviceId to select, for now, we need to misuse width.
97-
ideal_width = ids.length == 1 ? ideal_width : 1280;
98-
const constraints = ideal_width ?
99-
{
100-
video: {
101-
width: {ideal:ideal_width},
102-
deviceId: {exact: ids},
103-
},
104-
} : {
105-
video: {
106-
deviceId: {exact: ids},
107-
},
108-
}
109-
navigator.mediaDevices.getUserMedia(constraints)
110-
.then(function(stream) {
111-
return resolve(stream);
112-
})
113-
.catch(function(error) {
114-
return reject(error);
115-
});
116-
})
117-
.catch(function(error) {
118-
return reject(error);
119-
})
120-
});
91+
// Select color stream.
92+
// Color stream tracks have larger resolution than depth stream
93+
// tracks. If we cannot use deviceId to select, for now, we need
94+
// to misuse width.
95+
ideal_width = ids.length == 1 ? ideal_width : 1280;
96+
const constraints = ideal_width ?
97+
{
98+
video: {
99+
width: {ideal:ideal_width},
100+
deviceId: {exact: ids},
101+
},
102+
} : {
103+
video: {
104+
deviceId: {exact: ids},
105+
},
106+
};
107+
var stream = await navigator.mediaDevices.getUserMedia(constraints);
108+
return stream;
121109
}
122110

123111
// Figure out the camera intristics and extrinsics based on the depth stream
@@ -127,10 +115,10 @@ DepthCamera.getColorStreamForDepthStream = function(depth_stream,
127115
// hardcode the values based on camera model, but query it from the API.
128116
//
129117
// See the documentation at
130-
// https://w3c.github.io/mediacapture-depth/#synchronizing-depth-and-color-video-rendering
118+
// https://w3c.github.io/mediacapture-depth/#synchronizing-depth-and-color-video-rendering
131119
DepthCamera.getCameraCalibration = function(depth_stream) {
132120
const label = depth_stream.getVideoTracks()[0].label;
133-
const cameraName = label.includes("R200") ? "R200"
121+
const cameraName = label.includes("R200") ? "R200"
134122
: (label.includes("Camera S") || label.includes("SR300")) ? "SR300"
135123
: label;
136124

@@ -217,4 +205,9 @@ DepthCamera.getCameraCalibration = function(depth_stream) {
217205
}
218206

219207
function DepthCamera() {
220-
}
208+
}
209+
210+
function chromeVersion () {
211+
var raw = navigator.userAgent.match(/Chrom(e|ium)\/([0-9]+)\./);
212+
return raw ? parseInt(raw[2], 10) : false;
213+
}

‎index.html

+13-3
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,20 @@
11
<h1>HTML5 Depth camera capture demos</h1>
2-
<table width="500" cellspacing="0" cellpadding="0" style="border-collapse: collapse; border: none;">
2+
<table width="550" cellspacing="0" cellpadding="0" style="border-collapse: collapse; border: none;">
33
<tr>
44
<td align="center" valign="center">
5-
<img src="how_the_demo_looks.gif" alt="how_the_demo_looks.gif is not yet loaded." style="max-width:400px;"/>
5+
<img src="how_the_demo_looks.gif" alt="how_the_demo_looks.gif is not yet loaded." style="height:400px;width:452px;"/>
66
<br />
7-
<p>HTML5 Depth Capture Demo. <a href="depthdemo.html">Run live demo here.</a></p>
7+
<p>HTML5 Depth Capture tutorial shows how to access depth stream, check the <a href="https://01.org/chromium/blogs/astojilj/2017/depth-camera-capture-html5">tutorial text</a> or <a href="depthdemo.html">run the live demo here.</a></p>
8+
</td>
9+
<td align="center" valign="center">
10+
<img src="https://github.com/01org/depthcamera-pointcloud-web-demo/raw/master/recording.gif" alt="https://github.com/01org/depthcamera-pointcloud-web-demo/raw/master/recording.gif is not yet loaded." style="height:400px;width:422px;"/>
11+
<br />
12+
<p>3D point cloud rendering demo shows how to render and synchronize depth and color video on GPU. <a href="https://01org.github.io/depthcamera-pointcloud-web-demo/">Run the live demo here.</a></p>
13+
</td>
14+
<td align="center" valign="center">
15+
<img src="typing_in_the_air/typing_in_the_air.gif" alt="typing_in_the_air.gif is not yet loaded." style="height:400px;width:702px;"/>
16+
<br />
17+
<p>Typing in the air tutorial shows how to use depth stream and WebGL transform feedback to do simple gesture recognition. Check the <a href="typing_in_the_air/doc/tutorial.html">tutorial text</a> and <a href="typing_in_the_air/front_capture_typing.html">run the live demo here.</a></p>
818
</td>
919
</tr>
1020
</table>

0 commit comments

Comments
 (0)