SMPTE 2017: Keynote Explores the Potential of Immersive Experiences

LOS ANGELES—Although Andrew Shulkind's SMPTE keynote was titled “The Immersive Future: Broaden Your Horizons,” it was clear that Shulkind was really considering the immersive present (or imminentness).

Andrew Shulkind delivering his SMPTE keynote, "The Immersive Future: Broaden Your Horizons"

A co-founder of HeadcaseVR and an expert on virtual reality, augmented reality and mixed reality (VR, AR and MR) content capture and creation, Shulkind emphasized that he loves visual storytelling and "spends a lot of time thinking about the nuts and bolts of how images communicate emotion."

Acknowledging the power of cinema, Shulkind marveled at how “the writer, director, visual effects supervisor, cinematographer, actor all align on this single vector to teleport us to another place... another reality.” He cited a number of cinematic moments, including the first five minutes of Pixar's “Up,” “Tom Hanks shipwrecked on an island and confiding in a volleyball [in ‘Castaway’], Denzel Washington's stoic resistance in ‘Glory’... Denzel Washington embodying ‘Malcolm X’... Denzel Washington in anything.”

But, Shulkind warned, “What traditional films did well doesn't work necessarily any more,” identifying business changes within the film industry, primarily that with the “bottom-line model... the blockbuster system is broken."

The Age of Accelerations

"We live in an age of acceleration, as Tom Friedman likes to say," Shulkind explained. "And we may be reaching the asymptote as it steeply heads to infinity. In his new book, ‘Thank You for Being Late,’ Friedman identifies 2007 as the moment in time when the charted course of technological advancement changes. Here's a reminder he gives us. In 2007, IBM launched Watson; GitHub launched as an archive of open source software; Google launched Android; Google bought YouTube; Amazon launched the Kindle; internet usage crossed the billion users mark; cloud computing took off; Airbnb was created.

"2007 was also the year that ‘300,’ ‘Transformers’ and ‘Beowulf’ were released, three benchmarks that set a new bar for visual effects in film. Canon released the 5D Mark II, the camera that combined full HD in a DSLR for a low cost. RED delivered the RED ONE, which brought 4K to the masses.

"Netflix started streaming. 3ality shot U2 3D, a concert film, one of the best examples of live action stereo 3D for non-exclusive theatrical use. Dropbox was founded. Paramount Vantage closed its doors. The WGA went on strike from November 2007 for five months because of an impasse about new media residuals from streaming and online sales. Oh yeah, and Apple released the iPhone.

"You can see a profile of what's afoot. Friedman calls 2007 'the single most influential inflection point since Gutenberg.'" His story goes on to say that we were distracted because 2008 happened next. But back to 2007. That year is when technology in our world began changing the course of entertainment, which has, in turn, fundamentally changed how we photograph the stories that we tell.

"The pace of acceleration in every sector continues, from computer vision photography, volumetric capture, high dynamic range, wide color gamut imaging, light field imaging, six-stop deliverables, depth maps to replace greenscreen, virtual viewpoints. These advances in capture are consistently at the core of the future of this changing industry."

“Making movies as we know them,” he said, “has become kind of an impossible proposition." He recognized, however, that this “has set the stage for the outstanding diversity of [projects from broadcast] television and streaming sites: the small screen has become the big screen... with so many options at our disposal, the modern viewer doesn't really have to go to the theater at all.”

Citing the capabilities of web-enabled devices and the growing consumption of materials on multiple screens, Shulkind said, “The future of content is decidedly mobile first.”

“The intersection of traditional content, mobile devices and gaming,” Shulkind continued, “is where immersive content is found.” Noting that while new technologies are making immersive content possible, “the technology is timed to a growing appetite for the experiential... Audiences and artists are looking for active ways of consuming content in a more engaged way than simply watching a flat screen.”

With that in mind, he explained, this new generation of immersive content is “a natural successor... 360 video has removed a frame from our content experiences.” Shulkind then defined each type of emerging content: “360 video is a passive experience; VR is an active one. Using the same 360-degree canvas, virtual reality adds a layer of interactivity, choice and engagement. Augmented reality brings those elements from the virtual world into ours. Mixed reality combines those layers by augmented VR experiences with layered elements and allowing users to transition between environments.”

Shulkind sees that “the latent opportunity in the near future of entertainment is unprecedented in scale, between the hunger for content, diversity of tastes, buyers for ideas, products to innovate, a global mobile audience and the ability to laser in on what works best for whom.

“We may have what could be a once-in-a-lifetime chance to reset the bar and deliberately fashion the future of content—a finely tuned balance of traditional media, immersive, gaming, e-sports and live streaming—into a new composite experience, like cinema itself.

The power of cinema, Shulkind explained, can be seen in the first five minutes of Pixar's Up.

“Between immersive media, AI and the blockchain, a tidal wave of technology is coming down the pike that stands to upend the business,” Shulkind noted. “And that's the point. But in this era, we have to be more than technicians; we have to be generalists, multi-hyphenates. The business isn't refined enough yet to have cogs in wheels.

“Let's take ownership of this new medium and rewrite the rules of the diversity of stories and the diversity of who tells them, holding this new business to the high standards we have always demanded for ourselves.”

“So this is a call to action: Don't let technology lead the story,” Shulkind concluded. “We'll be selling ourselves short and underselling what could be. We have the potential and the render power to create that kind of empathy and connections to reconnect our world and shape the future.”