Any lingering resistance to virtual production involving next-generation elements like LED walls, in-camera visual effects (ICVFX) and mixed reality (MR) is rapidly disappearing across the media and entertainment industry as an increasingly viable hybrid approach to using the technology takes hold.

From broadcast programming to motion pictures and advertising, producers and directors are no longer faced with making an either-or choice between VP or traditional production modes. Instead, thanks to a spate of groundbreaking innovations that have made hybrid use-case flexibility a reality, virtual production (VP) in all of its permutations is now seen as an essential tool that can contribute time- and cost-saving efficiency to just about any project.

“Virtual production is an ever-evolving solution,”
Jaime Raffone, senior manager of cinematic production at Sony Electronics, says. “The move to immersive and hybrid technologies has been extremely helpful for us.”

Even before the hybrid VP paradigm took hold, researchers predicted big things for VP. In one outlook typical of many findings, Research and Markets projects the global VP market will grow at a 19.98% compound annual growth rate (CAGR) from $1.99 billion in 2022 to $7.13 billion by the end of 2029. According to Grand View Research, the VP spend in the U.S. over that span will increase at a 15.9% CAGR, reaching $1.09 billion in 2029.

Pros Are Embracing VP
By all accounts, resistance within professional production ranks to VP is rapidly giving way to enthusiastic embrace as benefits accrue from the more versatile hybrid approaches to exploiting the technology. Indeed, says Bob Caniglia, director of sales operations for the Americas at Blackmagic Design, the shift to VP is more creative-driven than a top-down, front-office phenomenon like the industry’s shift to video streaming.

“As effects guys and directors are pulled in, they start to realize they can really improve the entire workflow,” Caniglia says. While some directors are resistant to making the changes in production planning and execution required by reliance on LED volumes and ICFVX, many more want to benefit from things like providing “actors on set with live environments” and doing “more in shorter periods of time.”

The rise in VP adoption is having a big impact on Blackmagic’s output of cameras, switchers, editing tools and other products, as it meets demand for things like the locational flexibility enabled by cloud-based workflows and the LED wall-display quality stemming from the use of 12K and, soon with a forthcoming release, 16K cameras. The trendline in VP is “definitely moving upwards,” Caniglia says.

Sony’s Raffone, too, says the shift in perspectives on VP has been a big force behind new product development at her company. Notably, Sony responded to customer demand for LED solutions that aren’t fixed components of big studios by introducing the Verona line of modular LED displays in 2023. These employ the ground-breaking Crystal LED technology Sony created to achieve life-like realism through independent illumination of each pixel.

Over the past year, Verona displays have been deployed in multiple broadcast, cinema and VP educational projects, Raffone notes. New VP engagements in the broadcast industry include hybrid applications of Verona displays in live sports programming at WWE facilities in Stamford, Connecticut, and at the studios of another major sports broadcaster she declined to name.

Sony Bolsters Hybrid Use
Sony has also strengthened its position in the production space with a Virtual Production toolset that supports ICVFX (in-camera visual effects) within the Epic Games Unreal Engine multidisplay rendering platform to meld the real and virtual domains and cut production time through virtualized renderings of camera settings ahead of actual production. Integration partnerships with Brompton Technology and Megapixel VR ensure out-of-the box compatibility between Verona LED displays and leading production LED controllers, Raffone also notes.

The upshot is a hybrid-optimized VP portfolio that’s gaining traction worldwide across the company’s traditional market base as well as in new areas of business, education and government. “We continue to evolve with the needs of the market,” she says.

Ross Video is another major player benefitting from the hybrid VP wave. “Control and integration across all elements is probably our forte and strong suit,” Mike Paquin, senior product manager for virtual solutions at Ross Video, says.

Ross, which in December announced its intention to sell the G3 LED wall-supply business it acquired in 2021, is seeing widespread use of LED walls in VP, but primarily in hybrid situations where broadcast productions can take place on stages without a VP component or be augmented with LED and/or MR elements to whatever degree is warranted in a given situation.

A big contributor to this versatility is Ross’s support for rendering graphic elements in whatever dimensions fit a given scenario, Paquin says. The company’s XPression Tessera platform employs a distributed workflow system that allows users to configure graphic elements for pixel-accurate rendering across multiple display environments with a dynamism that enables real-time changes in content in tandem with live events.

Such flexibility has created a multiuse studio environment that’s saving broadcasters a ton of money. “Building three or four studios for a TV station doesn’t work anymore,” he says, noting that many Ross customers, including smaller stations that “see what their network parents are doing,” are in the process of refreshing their studio environments. “Virtual is a part of almost every one of them,” he says.

Toward that end, Ross at NAB Show in April will demonstrate support for a flexible workspace where a hard set and broadcast desk with display wall can become a weather reporting studio with the kind of green-wall functionality weathercasters are accustomed to, and then be turned instantly into a news or sports reporting venue with LED wall support for rendering the setting and attendant graphics. “You can rotate people through that space for different segments of the same live show,” Paquin says.

ICVFX Without the Walls
ENCO Systems, a pioneer in automated broadcast production workflows, has taken its own approach to VP by leveraging decades-old chroma key video monitoring technology with the compositing tool supplied by Miami-based Qimera. ENCO, the exclusive North American sales outlet for Qimera, markets the high-powered computing appliance Qimera Chroma as a way to weave VP capabilities typically supported by LED volumes into existing live broadcast production workflows without actually using LED walls to provide a visual context for onscreen personalities to work in.

Relying on Unreal Engine to pull together all the elements of virtual sets and augmented reality (AR) graphics created by the Qimera compositing tool, ENCO allows newscasters to view on their monitors what’s happening with their placement in the virtualized output their audiences are seeing on their home screens. “Just like VP with LED volumes, using chroma key with Qimera, you can change the sets and images with the click of a mouse,” says Bill Bennet, media solutions and account manager for ENCO.

This requires processing delivered by Nvidia’s GeForce RTX GPU and a 13th-generation Intel core along with other high-end hardware components in the Qimera appliance. Bennet says that’s a lot cheaper than investing in LED walls and all the processing they require. “The costs of LED volumes often start at $100,000 and go up from there,” he notes. “We built Qimera Chroma to avoid that. I love LED volumes, but we don’t always need them.”

Hybrid VP on Wheels
One of the more dramatic manifestations of the new hybrid flexibility in VP was recently on display in the middle of nowhere at Disney’s Golden Oak Ranch filming lot, a sprawling property north of Los Angeles. That’s where Brian Nowac, founder and CEO of Magicbox, was on hand for production of a series of Bush’s Baked Beans commercials. While the setting was appropriate for shooting the outdoor scenes, the rest of the filming was taking place in a tractor trailer outfitted by Magicbox as a complete self-contained LED studio.

Speaking with TV Tech, Nowac describes the rapidly expanding role the uniquely equipped Magicbox trailers are playing in hybrid M&E productions. “We’re building a fleet of Superstudio products we’ll be deploying all over the country,” he says.

The Magicbox trailers, measuring 52 feet long, 32 feet wide and 13½ feet high, can be used with a 10-foot high LED volume. A motorized turntable floor makes it makes it possible to rotate the car when inside the truck.

A recent Dodge Hornet commercial with a child version of an adult passenger at the wheel was shot entirely inside the Magicbox, Nowac notes, which would have been impossible with a child driving on city streets. Another example of what can only be done using virtual production is a commercial for Subaru that features in-car scenes involving a driver talking with her dog.

Along with the convenience of bringing LED studios to virtually any location, Nowac notes that Magicbox is making it possible for producers to save a lot of money by having another production space at hand when set-changing and other main studio downtimes would leave high-paid actors and staff standing idly by.

“Actors can walk right outside a sound stage into the Magicbox studio and do alternate takes or more of their key lines using the exact same lighting and other aspects as they’re using in the big studio,” Nowac says.

Global shipments of virtual reality and mixed reality headsets are expected to reach approximately 9.6 million units in 2024, representing a year-over-year increase of 8.8% according to a new report from TrendForce.

Three key industry trends are shaping the global VR and MR ecosystem, according to the researcher: The dominance of low-cost devices, the shift from entertainment accessories to productivity tools and OLEDs on Silicon (OLEDoS) emerging as the preferred display technology for high-end near-eye devices. These three trends are expected to continue influencing the global VR and MR market over the next several years, TrendForce said.

Meta retains its position as the global leader in VR and MR device shipments with a 73% market share in 2024. The primary growth driver is the affordable Quest 3S—priced at just $299—contributing to an 11% year-over-year increase in shipments.

TrendForce said the limited availability of new applications for VR and MR has prompted Meta to adopt a strategy that prioritizes affordable pricing to capture consumer interest. To this end, Meta has opted to release the Quest 3S early while also suspending plans for the high-end Quest Pro 2, signaling a shift in focus toward budget-friendly devices, the researcher said.

Sony’s PS VR2 secured the second-largest market share at 9% for 2024. “Despite Sony’s efforts to integrate the PS VR2 with PC platforms using adapters to broaden the content ecosystem, limited functionality and application support have hindered its performance,” TrendForce said. “Consequently, annual shipments declined by 25% YoY.”

The Apple Vision Pro, which debuted in 2024, quickly captured a 5% share, making it the third-largest player in the VR/MR market. Sales figures fell short of previous Apple product launches, though, largely due to its high price and limited application resources, according to the researcher.

“Vision Pro’s entry, along with Meta’s decision to accelerate the launch of its budget-friendly model and suspend its high-end model, demonstrates that affordable pricing remains the key factor driving consumer adoption,” Trendforce said. “Price sensitivity among consumers has solidified LCD technology as the dominant display technology for VR/MR devices, with a market share exceeding 80%.”

The launch of the Apple Vision Pro has shifted VR and MR devices beyond their traditional focus on consumer-oriented entertainment toward a broader role as multifunctional productivity tools, TrendForce said.

“Vision Pro is redefining the capabilities and use cases of VR/MR devices—from document editing and virtual meetings to advanced applications in healthcare and education,” it said. “This paradigm shift is expected to encourage other brands to reevaluate the functional attributes of their own VR/MR devices, expanding their role in both consumer and enterprise markets.”

Despite the Vision Pro’s underwhelming sales performance, it has succeeded in establishing a new standard for user expectations of VR/MR. It is also the first device in the industry to feature OLEDoS display technology, setting a new benchmark for product specifications and visual quality in the sector, TrendForce said.

“Apple’s extensive experience in hardware design and its existing expertise in smartphones, tablets, and laptops positions the company as a key driver of the adoption and diversification of VR and MR applications,” it said. “Looking ahead, Apple is expected to launch its next generation of VR/MR devices as early as 2026, with a strategy to introduce two distinct models targeting both high-end and mainstream segments.”

ATLANTA—CNN is giving its viewers the opportunity to interact with the latest news in a new way. The network has launched an app on Magic Leap One, which provides an interactive, mixed reality experience.

The app allows for the viewing of live TV, CNN shows, CNN originals and CNN films, with interacting news and information available while they watch. There will also be exclusive reporting, data and live video feeds from around the world.

To access the CNN app, users will launch it through “Screens” within Magic Leap One and sign in with their TV service provider. The app can be navigated through gesture control.

The launch of the CNN app on Magic Leap adds to CNN’s previous efforts with Oculus Rift.

HOLLYWOOD--The 2018 SMPTE conference opened on Monday with a overview of the opportunities M&E companies will one day pursue as private transportation transitions from cars driven by individuals to a future where self-driving vehicles turn drivers into passengers looking to be entertained.

Richard Merchant, Managing Director in Monitor Deloitte’s Strategy practice, and leader of Deloitte’s Media & Entertainment efforts on the Future of Mobility, painted a picture of where this transportation transformation that the M&E industry cannot ignore.

TV Technology Contributing Editor Phil Kurz sat down with Merchant after his address to discuss these opportunities in depth.

TV Technology:You seem very bullish on the future of autonomous vehicles. What gives you that confidence?

Greg Merchant: Deloitte has been doing a great deal of research studying the future of mobility and really delving deep into each of the industries it’s going to impact.

So even if you are wrong on the timing, it’s hard to argue against many of these impacts coming into society.

When you really look at the likelihood of everything transforming and the accelerating delivery of technologies that are making all of this possible, it’s hard to argue that it’s not going to change the way transportation works and the way mobility works sooner and permanently.

TVT:Earlier today in his keynote, Doug Davis of Intel quoted a Strategy Analytics study the company commissioned. It valued the autonomous vehicle market at $7 trillion by 2050. What do your figures show? And what slice of that will be related to media?

GM: The estimate we’ve done predicts about $2 trillion of economic activity every year. By 2030, the ecosystem should actually be humming along. In that, media specifically was pegged at $16 billion of economic impact every year.

But as I said in today’s speech, I would consider that more of a floor than a ceiling because there are more than just consumption opportunities we are talking about with the in-transit experience.

There are also going to be entirely new forms of experiences that are going to be developed across video, social, music, advertising, live entertainment, education and productivity. So, when you think about how all of those are going to be constructed both technically and experientially in the car, $16 billion will be on the low end.

TVT:Many of the speakers here at SMPTE today have made the point that the younger generation is receptive to ride sharing –a sign that Americans’ love affair with their cars may be coming to an end. But that generation was growing up in a time when the economy was flat. Given today’s strong economy, improvements in their financial means and growing ability to own their own cars, do you think their willingness to ride share will wane?

GM: I think we may see slight wavers, but I think the trend is headed in a direction where it’s not going to reverse. Put it this way, do you take more Ubers now than you did two years ago?

Ride sharing is actually expanding into different groups more than it is contracting. The conversation you have more often than not with leading and trailing millennials is about the reasons why they should ever buy a car.

Perhaps as they move out to the distant suburbs they may find an incentive to buy their own cars. But as long as they live around urban centers, ride sharing is only going to continue to grow.

TVT:Tell me more about the revenue possibilities this change in mobility will create for M&E.

GM: I think advertising is going to be revolutionized. You’ll be able to see different kinds of campaigns structured and targeted and sustained over the course of your daily commute, or even over the course of weeks.

You are going to be able to use location-based technologies to make them increasingly sophisticated and effective.

Ad insertion technologies for the inside of an automobile that are AR-enabled will absolutely change the way you interact with and view the world around you.

TVT:You mentioned in your presentation that billboards might evolve dramatically.

GM: Right now, the top of the line is a billboard that switches. It may, at most, have 30 different advertisements on it.

In the future, you could simply have a blank billboard that projects a customized ad for every single person that drives past it because it will know you are coming, and it will be able to do that seamlessly through an AR environment through your windshield.

TVT:Why would you even need that if you have all of the displays in your autonomous car?

GM: That’s part of the discussion. Do you still want to keep billboards in some way, shape or form? Remember, people still walk, they’ll see it outside the car.

But some billboards next to a high-volume highway, you probably don’t even need. If you can project an ad onto the billboard, you can most likely project a billboard.

That’s one area. The other that gets opened up, particularly with 5G, is that you are going to start to see ad-supported transportation –so the route you take home may change if someone pays a little extra to run you past their grocery store.

The cost of your ride will be determined by how much advertising you are willing to absorb on the trip.

I think you will actually see Uber partner with someone else to manage the cost of the ride.

TVT:What are some of your thoughts about how TV broadcasters and others might capitalize on this through their production businesses?

GM: Because of the development phase of the technology, most of what we are going to be limited by is our imagination.

Said another way, not everything is where it is going to be, but there is such a tailwind behind this technology that it will get there.

When you think about the different form factors you can put into a car –with ATSC 3.0 for instance—you’ll be able to have this integrated advertising. You’ll be able to have interactive like you’ve never dreamed of.

You’ll be able to play a game with an acquaintance halfway around the world while you are coming home from work and they are going to work.

You’ll be able to serve so much data into these delivery platforms at any one time.

TVT:What about the demand for production stemming from this anticipated mobility play?

GM: I think that is going to be a huge challenge. There is all of this technology on one side. On the other is the need to tell creative stories. There’s a need to tell a compelling version of what happened with completely engaging content.

I think there will be growing pains. It probably won’t be the most elegant experience as we get started.

But there is going to be an evolution as there has been with pretty much every media we come out with as people learn how to communicate through that particular medium.

You know, we are talking about something that could be an immersive 360-degree environment. So, to tell a story in that while you’re moving is going to be a little more complicated than just a two-dimensional story being told through your mobile phone.

I think it makes sense to start working on that now, particularly the technology and how to structure stories that get told in those environments as you go from your kitchen to the car, to the train, to a city bike –the ability to keep a story going and tell it across all those different media is going to be important. And there is a big production component to that.

But the real reason I think it makes a lot of sense is historically you’ve tried to simulate entertainment experiences and bring them into the car as best we can. Now we are talking about something where you have a contained environment where you could develop something that is the new standard of what people are going to want in their very own living room.

This isn’t something where the destination is only going to be the car. The destination could be the living room, but we are starting in the car.

The 2018 SMPTE conference continues through Thursday, Oct. 25. For more information, visit www.smpte2018.org