Helping doctors focus on what matters
We’ve grown accustomed to seeing the Kinect sensor put to amazing uses in healthcare—from stroke rehabilitation, to fall prevention, to seizure monitoring—but even we were surprised to learn that Kinect for Windows is now being used to watch the doctor! That’s right: researchers at the University of California, San Diego (UCSD) are using the Kinect for Windows sensor as part of an innovative tool to monitor how physicians interact with patients during consultations.
The project, called Lab-in-a-Box, is the brainchild of UCSD researcher Nadir Weibel and his colleagues at the San Diego Veterans Affairs (VA) Medical Center. Designed to fit unobtrusively in a doctor’s office, the apparatus detects when physicians are so focused on their computer screen that they fail to establish eye contact and one-to-one rapport with the patient. The Kinect sensor plays a key role in the process, as its depth camera accurately records the movements of the physician’s head and body. An independent eye-tracker device detects the doctor’s gaze, while a microphone picks up the doctor-patient conversation. The Lab-in-a-Box software merges all of these data streams and synchronizes them with data on the doctor’s computer usage, combining everything to give a detailed picture of the physician’s interaction with the patient.
The Kinect sensor captures a video and depth image; the latter is then overlaid with joint and
gaze estimation based on yaw, roll, and pitch.
Weibel, who is a research assistant professor in the Department of Computer Science and Engineering, devised the Lab-in-a-Box in part to help physicians deal with the ever-expanding barrage of digital medical records. With so much digital information about the patient, doctors can find themselves staring at their computer or tablet screen, instead of focusing attention on the patient.
The Lab-in-a-Box, which will be used only with the consent of both the doctor and the patient, is currently being tested at the UCSD Medical Center and the San Diego VA Medical Center as part of a study funded by the Agency for Healthcare Research and Quality and directed by physician and researcher Zia Agha. Its developers hope that it will help medical personnel run their practices more efficiently and with greater doctor-patient rapport.
The Kinect for Windows Team
Key links