Near Mode: What it is (and isn’t)

There has been a lot of speculation on what near mode is since we announced it. As I mentioned in the original post, the Kinect for Windows device has new firmware which enables the depth camera to see objects as close as 50 centimeters in front of the device without losing accuracy or precision, with graceful degradation down to 40 centimeters.

The lenses on the Kinect for Windows sensor are the same as the Kinect for Xbox 360 sensor, so near mode does not change the field of view as some people have been speculating. As some have observed, the Kinect for Xbox 360 sensor was already technically capable of seeing down 50 centimeters – but with the caveat “as long as the light is right”.

That caveat turned out to be a pretty big caveat. The Kinect for Windows team spent many months developing a way to overcome this so the sensor would properly detect close up objects in more general lighting conditions. This resulted not only in the need for new firmware, but changes to the way the devices are tested on the manufacturing line. In addition to allowing the sensor to see objects as close as 40 centimeters, these changes make the sensor less sensitive to more distant objects: when the sensor is in near mode, it has full accuracy and precision for objects 2 meters away, with graceful degradation out to 3 meters. Here is a handy chart one of our engineers made that shows the types of depth values returned by the runtime:

In Beta 2, for an object 800-4000 millimeters from the sensor the runtime would return the depth value, and the runtime returned a 0 regardless if the detected depth was unknown, too near or too far. Our version 1.0 runtime will return depth values if an object is in the above cyan zone. If the object is in the purple, brown or white zones, the runtime will return a distinct value indicating the appropriate zone.

Additionally, in version 1.0 of the runtime, near mode will have some skeletal support, although not full 20-joint skeletal tracking (ST). The below table outlines the differences between the default mode and near mode:

We believe that near mode, with its operational envelope of 40 centimeters to 3 meters, will enable many new classes of applications. While full 20-joint ST will not be supported in near mode with version 1.0 of the runtime, we will be working hard to support ST in near mode in the future!

Craig Eisler
General Manager, Kinect for Windows

Comments

• Anonymous
  January 20, 2012
  This is great news. I believe, if possible, that hand/wrist/elbow tracking would be very powerful, if it could be pulled off.  This may allow for sign language type apps, or hand gestures.  I don't know if that would be a stretch feature or not, but I can think of quite a few uses for it :)

• Anonymous
  January 21, 2012
  Will we able to upgrade our current kinect for 360 to this new firmware?

• Anonymous
  January 22, 2012
  Hi Craig.  Does that mean the commercial Kinect for Windows will have both Default and Near Modes?  Is Near Mode something we can flip on and off.  I actually prefer the Default Mode; don't have much use for Near Mode in my current apps.  Thanks, Craig.

• Anonymous
  January 22, 2012
  Kinect Fan, yes, you will be able to switch between near mode and default mode on the Kinect for Windows hardware, using the Kinect for Windows SDK 1.0.

• Anonymous
  January 22, 2012
  Woohoo!  That's awesome!  Thanks, Josh.  By the way, I attended your Kinect SDK seminar at Carnegie Mellon last October.  Great stuff.  :)

• Anonymous
  January 22, 2012
  @Andrea: No, this firware is not designed for the XBox Kinect.

• Anonymous
  January 27, 2012
  Will you be considering supporting multiple monitors or will the application have to run on the screen where the Kinect is running? It would be an interesting experiment to see if Kinect could calibrate between the placement of monitors.

• Anonymous
  January 27, 2012
  @Matthew: Not sure I understand your question. The Kinect sensor can be placed wherever is appropriate for the specific application. It doesn't need to be associated with a particular monitor, or with any monitor at all, for that matter.

• Anonymous
  March 02, 2012
  Is it possible for kinect windows device to capture gestures of a person when sitting down behind a desk (i.e. Kinect working in Near mode)? I understood that the currently available kinect device is not capable of partial skeleton detection. Is that true? Thanks, Vijay

• Anonymous
  March 25, 2012
  Hi! May I know whether the sdk has support for eyebrow and lip detection stated by the rumors?

• Anonymous
  May 24, 2012
  The comment has been removed

• Anonymous
  June 05, 2012
  Please confirm whether the hardware (not just firmware) is different between Kinect XBOX and PC (except the USB cable of course, but you can buy a cheap adapter for that). Reading between the lines, the XBOX Kinect is still the same hardware but logistical problems (worldwide manufacturing, testing and distribution) made it just too big a change to risk right now. If the hardware is the same, could the firmware upgrade come some time in the future. Or are all improvements just being withheld to help sell a Kinect 2, released with perhaps even built into an XBOX 720?

• Anonymous
  June 07, 2012
  Found the answer myself here... www.winsupersite.com/.../kinect-windows-142223

• Anonymous
  February 24, 2014
  comment trouver le profondeur d'une troux dans la rue