The Reality of Virtual Reality

NEW YORK— Virtual reality may not quite be a shipping product, but it has some very real companies championing its merits and promoting the technology in the marketplace. Last year, social media behemoth Facebook purchased VR startup Oculus VR for $2 billion, while other players continue to push the bounds of what it means as a developing medium.

VR is now clearly more than vaporware, as noted by 21st Century Fox’s use of the technology at CES to demonstrate how this immersion technology could change the storytelling process. For the Fox Searchlight film “Wild,” based on Cheryl Strayed’s book about her solitary hike on the Pacific Crest Trail, the film studio created a three-minute companion VR piece that practically took viewers out to the wilderness.

Attendees at CES (and later at the Sundance Film Festival) were able to “view” “Wild—The Experience,” on Gear VR headsets, which were developed by Oculus VR and Samsung. Instead of watching the movie in the traditional manner, with eyes facing a large screen, viewers have a 360-degree virtual reality encounter with the film’s Cheryl Strayed, played by Reese Witherspoon, and her giant backpack.

“There is no fourth wall in VR,” said Ted Schilowitz, VR consultant for 21st Century Fox. “You look everywhere and you see everything in the forest and experience a sense of place.”

“The whole idea isn’t just to demonstrate the technology but rather what it can do,” he said. “It is to make it interesting to the audience, and I think this is what we accomplished with the ‘Wild’ project. This showed what a movie studio can do when it approaches a new medium. It creates a new experience.”

While VR is getting real, it won’t likely make the current movie experience go away. Nor is anyone predicting that it will even become a big part of the mainstream moviemaking process—at least not the in the way that 3D was hyped to do back in 2009 and 2010. VR may be the next big thing, but it will have its own place as a new medium.

“It isn’t going to replace film, and it isn’t going to replace TV, but it is a huge opportunity for entertainment,” said Scott Broock, vice president of content at VR development studio Jaunt. “This product is radically different from what you see on a 16:9 box.”

“VR is supplemental, not a replacement, and not necessarily even just a complementary medium,” adds Dale Carman, executive creative director and founder of Reel FX. “HD on an iPad didn’t threaten the big screen in the house, and in that way VR is supplemental. You can watch a movie, but with VR, you can also be in the story and experience it in that way. VR allows me to be in the Millennium Falcon in a way that I can’t be by just watching Star Wars.”

Because VR immerses the viewer in an experience in a way that is fundamentally different from watching a movie or TV show on a flat screen, it presents new challenges both in the production of content and even in the way a story is told.

All objects on the set—everything from the lights and cameras to the crew, which normally can be positioned just outside the view of the camera lens—need to be handled differently given that the viewer can pan in any direction. For this reason, there is no such place as “off camera” and no traditional “fourth wall” to break down.

“There are endless technical hurdles and creative hurdles that we need to overcome when filming a VR experience,” Carman said. “It can begin with needing to have multiple cameras to capture 360 degrees of action.”

Carman adds that VR production is still in its infancy. The rules are still being written, and then updated with each new VR production.

“There are a number of techniques we’ve used that range from wide angle lenses to using several GoPro cameras,” he notes. “We’ve tried them all and will likely continue to try to new things.”

“The shooting logistics and process are different, and that is true for the talent, for the technical crew, for the director of photography, for the sound men, for everyone involved,” said Felix Lajeunesse, co-founder of Felix & Paul Studios, the Montreal-based developer of proprietary 360-degree stereoscopic camera recording technology. “It’s a challenge of adaptation, but I wouldn’t say it’s ultimately a more difficult process than a traditional film shoot.

“A 3D, 360-degree VR shoot simplifies certain aspects of the process,” he said. “For instance, we definitely do not shoot as many camera angles, but it makes certain other aspects trickier. Lighting for VR can be quite a puzzle.”

Much of the work of creating a virtual reality experience takes place in postproduction, where the perspectives shot by different cameras are unwrapped and stitched back together so that viewers can ultimately look wherever they like and see a seamless world.

Carman shares a recent discovery, that too much motion—where the camera and the viewer’s perspective move independent of each other—can create a type of “cyber sickness.” This is just one hurdle that VR production is overcoming.

“This is vastly more complicated than traditional filmmaking,” Carman said.

There are specific guidelines that need to be observed in the production stage as well, such as hiding the crew and equipment. For Wild—The Experience, all the members of the large crew had to duck down below the line of sight during filming.

“You need to hide everything or remove it in postproduction,” Schilowitz said. “But either way, this is where the movie magic comes in to create the illusion of the world, because there are things that you simply can’t hide when you’re making a film.”

table border="0" height="334" cellpadding="2" cellspacing="2" width="432"> Jaunt captures Paul McCartney’s Out There tour in 2014 for a VR presentation. “You have to think a lot beforehand in the production of VR,” adds Broock. “Where is the lighting coming from? You can’t have umbrella stands and traditional cans, as these would be seen.”

CGI can offer an alternative to hiding the crew and equipment, and visual effects work can provide full control over the environment, but the downside, according to Carman, is that it’s expensive.

There are two trains of thought about how to handle computer-generated content during the presentation of a VR experience, Carman adds. “If it is generated in real time, it gives total control as the viewer moves, but this requires a more powerful system to do the rendering.

“The other method is to pre-render,” he continues. “This has almost no limits on the complexity of the world, but the downside is that you don’t have total freedom, so you need to predetermine the world when it is possible,” before the presentation begins.

As well as presenting a new set of challenges for the film crew, the means of telling a story via a VR interface is radically different as well. “The biggest challenge for a filmmaker is to reconfigure your brain for the medium of virtual reality, to fundamentally embrace it as a distinct art form, and to overcome the evident formal and conceptual challenges of storytelling in VR by boldly facing the unknown and taking creative leaps and risks, instead of trying to retrofit the traditional cinematic language with a hope that eventually audiences will get used to it,” Lajeunesse said. “VR is not cinema.”

From the director’s standpoint, the new medium may require coaxing the viewer to look somewhere to advance the story; at the same time, VR brings the ability for viewers can see things in a way that they hadn’t before. “This is where being very creative really comes into play,” Schilowitz said. “You need to make it feel natural, but provide different cues that are moving from scene to scene or into the depth of field.”

“It has to become navigational,” Broock said. “But at the same time, you have complete control to look around as the viewer. That’s why this is such an exciting technology. I feel this is really that once-in-a-lifetime chance to be at the start of a totally new medium.”