Kinect helps VR film capture the terrors of war
Even in the rarefied atmosphere of the Sundance Film Festival, where innovative and engaging storytelling abound, the New Frontier exhibition stands out for its cutting-edge amalgam of film, art, multimedia, and technology. It was in this select narrative realm that Giant, a virtual reality (VR) short, left its mark last week. And, we’re pleased to note, Kinect for Windows played a critical supporting role.
Giant tells the story of a family trying to survive in an active war zone. It draws from the personal experiences of its director and co-creator, Milica Zec, who grew up in war-torn Serbia. But rather than the ravaged Balkans, Giant is set in the United States, where an American family—father, mother, and six-year-old daughter—hide in their basement during an attack. Explosions, coming ever closer, reverberate from the world outside. Seeking to shield their daughter from the knowledge of imminent catastrophe, the parents tell her that the noises are the thundering footsteps of a giant who wants to play with her. The parents' fairy tale intensifies as the bomb blasts draw closer and closer.
Giant was presented at Sundance Film Festival as an immersive, virtual reality installation. Wearing virtual reality headsets (the Oculus Rift CV1) and seated in specially-designed chairs, viewers experienced a five-minute narrative that mixed live-action video with a 3D video-game-engine environment. The chair contained a subwoofer synchronized to activate when explosions occur or the building shakes in the video, immersing the viewer in the action on screen. The screening room was designed to match the set and lighting in the video itself, to better acclimate the audience to the virtual surroundings before the video even began.
The basement in “Giant,” where the family seeks refuge from the bombardment outside.
“Through virtual reality, the viewer can see, hear, and feel what it’s like to be at the epicenter of a conflict zone,” said Zec in a statement. “They can also witness the fear and suffering that millions of innocent families around the globe are going through, at this very moment.”
Viewers were extremely enthusiastic about the experience, including Mad Men actor Jon Hamm, who, having heard the buzz about Giant at the festival, was one of the first to immerse himself in the film’s VR experience. Attendees found themselves part of the action—feeling like participants rather than passive viewers—which demonstrates the emotional potential of VR films.
The creators used Kinect for Windows v2 along with DepthKit to record depth data in conjunction with their image camera (RED Dragon, recording 5k). The resulting depth information was aligned temporally and spatially with the image, using DepthKit. Next, the team created a high-resolution, hue-encoded depth image, which was spatially and temporally noise reduced. This combination of technologies and techniques enabled use of live action performance as volumetric video assets inside a computer-generated environment rendered live by the Unreal game engine.
“Kinect was an immensely affordable and beneficial solution for us,” said Winslow Turner Porter III, Giant’s producer and co-creator. “We were able to leverage the low-cost device with sophisticated processing to enable a one-of-kind immersive experience.”
What’s next for Giant? Continued sponsorship with Microsoft for one thing, including potential screenings at the Game Developers Conference (GDC), the GPU Technology Conference, and the Eyeo Festival. And more VR experiences for the creative team of Zec and Porter, as Giant is just the first of a planned trilogy.
The innovative VR filmmaking that went into Giant demonstrates the power of Kinect for Windows as well as Microsoft’s commitment to “More Personal Computing,” a vision that integrates the capabilities of Windows, Bing search technologies, and Microsoft hardware to provide seamless creativity and productivity for users.
The Kinect for Windows Team
Key links