Depth Camera

Glossary Item Box

DSS Interop Samples: Kinect Sensor

Robotics Common: Webcam Sensor

Microsoft Robotics Developer Studio

Send feedback on this topic

Depth Camera

The DepthCam Sensor service provides information from a depth camera. At present the only supported depth camera is the Kinect, which means that the DepthCam Sensor service is not really generic.

The DepthCam Sensor service provides the following operations. The Replace operation is used for notifications.

Operation	Description
Get	Gets the State of the DepthCamSensor service.
HttpGet	Gets the State via a HTTP request, usually from a web browser.
HttpQuery	Used to request images from a web browser.
Replace	Reserved. Do not send this operation request.
Subscribe	Subscribe to notifications. Only Replace messages are sent as notifications.

You can Subscribe to the DepthCam Sensor and it will send frames as notifications (Replace messages). Alternatively, you can poll the service using Get.

The state of a Webcam Sensor is represented the following properties.

Name	Type	Description
DepthImage	short[]	This is the depth data arranged in scanline order from top to bottom and left to right as a single-dimensional array. Values are in millimeters.
FurtherThanMaxDepthValue	short	The value of a pixel when it is further than the maximum range.
MaximumRange	double	Maximum depth value in meters.
MinimumRange	double	Minimum depth value in meters.
NearerThanMinDepthValue	short	The value of a pixel when it is nearer than the minimum range.
Pose	Pose	Defines the position and orientation of the camera. This must be set in an initial state (config) file.
Timestamp	DateTime	Time when this image was captured.
VisibleImage	byte[]	The raw visible image data in BGR24 format. Note that the order is not RGB.
DepthImageSize	Size	The dimensions of the depth image.
FieldOfView	double	Horizontal Field of View of the camera in Radians.
ProjectionMatrix	Matrix	A matrix that defines the conversion between real world view space (3D) and camera space (pixel coordinates plus depth).
InverseProjectionMatrix	Matrix	Inverse of the Projection Matrix, i.e. it converts in the opposite direction.
ImageMode	DepthCam- SensorImageMode	Reserved for future use.

Note that the state includes both the depth data (as an array of shorts) and the RGB data.

The depth data is 11 bits and is in millimeters measured from the camera as shown in the diagram below:

The depth data includes a player index in the low-order three bits. The depth data is therefore shifted to the left by 3 bits. You must take this into account when reading the depth data.

Depth Image space consists of X and Y coordinates of pixels in the image (normalized from 0 to 1) as well as a depth (Z) value in millimeters. The pixel values do not represent anything in physical space. Note that depth pixels do not align perfectly with RGB pixels for several reasons. The Kinect service provides a way to map depth pixels to color (RGB) pixels.

Skeletons are represented as a set of joints in 3D Skeleton Space. See the Kinect Sensor for more details.

For more information, see the Kinect Services for RDS document in the Documentation folder of your RDS installation.

 

See Also

DSS Interop Samples: Kinect Sensor

Robotics Common: Webcam Sensor

 

 

© 2012 Microsoft Corporation. All Rights Reserved.