SAN FRANCISCO—I was fortunate enough to spend the better part of the pandemic working with some of the most talented and forward-thinking creatives in live television production on season 15 of “America’s Got Talent” on NBC. My company, All of it Now (AOIN) was hired by Fremantle to work with the show’s Screens Producer Scott Chmielewski, DMDS7UDIOS, to create and build live XR (Extended Reality) scenes for some of the season’s top contenders.

There were a lot of vendors involved in bringing this very, very ambitious show to life. DMDS7UDIOS ran everything live on set, my team at AOIN together with creative teams at Gravity in Germany and XITELabs in Los Angeles produced XR content for the season’s signature performances. This was no small task, as each of the season’s seven episodes featured two XR performances, and in many of these performances we managed the initial integration and technical handoff of assets to the DMD team who drove the onsite programming, calibrations, integrations and real-time rendering out to a state-of-the-art 60-foot wide curved LED stage.  

What made the creation of this much digital content manageable was having multiple design teams on board working in parallel in Unreal Engine. With source control, we were able to create branches for each designer to work in independently, and then merge their respective changes with the larger team whether the assets or scenes were coming from Gravity, XITE or AOIN. Source control provided a clear path to produce this quantity and quality of content in the timeframe we needed to hit repeated weekly deadlines.

Each week we would receive creative direction for each XR performance in the form of mood boards and concepts that came in from creative producer Brian Burke and Harriett Cuddeford from the talented Syco Entertainment team. We then had three days to submit our first iterations, and the remainder of the week to refine our scenes for Friday ingest and onto the live performances, just one week later.   

These were ambitious visuals. Rather than a few months for each scene, we had five days and we were constantly figuring out how to reverse engineer the initial creative concepts to maximize assets we could pull from the Unreal Marketplace and Quixel Megascans to synthesize into environments that would look great on television. The Megascans in particular played beautifully on screen—the forest and foliage tools gave us a level of realism in performance visuals that would have otherwise been impossible. One of my favorite performances of the season, Noah Epps in week four, used the Decagon arcades from the Marketplace assets. 

The hardest part about this show is that it’s live and performance-based, and there are constant on-the-fly changes. In some cases we had to adapt some scenes meant to be shot in XR, or performers would be swapped for use in a different CG scene. The teams were able to make both minor and major adjustments to scenes in a short amount of time, and sometimes even repurpose scenes from XR to rendered videos for non-XR performances. The tools in Unreal Engine gave us the flexibility to render out scenes dynamically and have the option of outputting real-time or rendered content.

The sheer volume of content required from this production made it practical to have dispersed creative talent. Our team in San Francisco could be working on butterflies, and XITE’s team on modeling, then handing off to Gravity’s team in Germany to work on trees and foliage, all in the same world for a given performance. The collaboration was extended with the show’s creative directors who sat in on daily review sessions where we would share Unreal Engine scenes on Zoom calls, and could iterate in real time without having any of the creative direction get lost in translation. 

Onset, the DMDS7UDIOS team worked with director Russ Norman to capture performances using four XR cameras and Stype Red Spy Fiber systems for camera tracking. Florian built virtual cameras and other helpful assets into an Unreal template so that he could give us an accurate camera plot that allowed us to block everything in advance, and know how to place assets in scenes in a more thoughtful way. This tool was very useful and avoided a lot of potential issues by the time it came to live rehearsals. The XR performances were shot on a 60-foot wide curved LED stage, which along with a highly reflective floor, posed certain creative challenges. We had to take the real-world reflections and the virtual world reflections into consideration on shoot day, and Scott was able to use live DMX data to control hundreds of virtual DMX fixtures and effectively manipulate lighting in virtual environments. 

The sheer scale of each of these XR performances, and the ability to create these fantastic worlds, makes this season stand out. The “AGT” team wanted to try this out a few years ago, but the technology wasn’t quite there yet to make this a practical option. Two years later we came back, and amidst the social distancing requirements due to the pandemic, performances were spread across multiple timelines, which made the production timeline more conducive to working in XR. It was a thrill and an honor to get to work under the brave creative supervision of Brian Burke and Harriet Cuddeford who came up with incredible concepts to build into XR performances.

This was our first full season episodic series done entirely in Unreal Engine for the XR performances, and we would have struggled to complete the XR portions of this show in any other way. We were heavily dependent on the ability to support multiple designers and automate their creative flow with source control, along with having access to premium pre-built assets in the Marketplace. We also needed the ability to roll back when a producer said “we liked what you did yesterday, can you revert to that version?” or if a performer would want to go in a different direction creatively on the day of live rehearsals. 

Working in Unreal gave us the unique flexibility to avoid being locked into too many creative variables while delivering scenes that transport the viewer in a whole new way. Producing live XR content in a real-time game engine feels like what surely will be the future of performance-based television programming.  

Danny Firpo is the co-founder of All of It Now Productions.

LOS ANGELES—“America’s Got Talent,” the acclaimed nationwide talent show that has become a mainstay on NBC, is set to premier its 12th season this year. Cineo Lighting has been with us since the beginning, helping ensure judges, hosts and performers always look their best for the camera.

Featuring unique performers from across the country, the series is a true celebration of creativity and talent, offering a colorful array of singers, dancers, comedians, contortionists, impressionists, jugglers, magicians, ventriloquists and other hopefuls. These acts rely not only on their abilities, but also on the aesthetics of their acts. For this, I need a lighting solution that I know will give me the color temperature I want right out of the box. I can’t have any degradation of field or uneven color output, otherwise I’ll lose precious time adjusting each light during setup. Also, if the lighting isn’t properly adjusted, the post-production team will have to spend time touching up the brightness and evenness of the shot for the viewers at home.

PUTTING THE LIGHT’S TALENT TO USE
To achieve this, we feature 25 Cineo HS and several HSX fixtures, supplemented by 10 Maverick fixtures with battery attachments, and Cineo Matchstix kits and Matchboxes, which light all the ancillary action not on the stage, including the holding room, interview positions and the Judges Gold Room.

Darran Webb uses Cineo lights for all areas of production for “America’s Got Talent.” We’ve used the Cineo HS since it debuted back in 2012, during what I call the “wild west” of LED lights. The 3200–5600K HS has the even field I expect from Cineo. Cineo’s Remote Phosphor Technology (RPT), eliminates the color-accuracy limitations that can be an issue in LED fixtures over time. From the holding room, to the talent on stage, to the judges, the HS lights have a hand in lighting several aspects of “America’s Got Talent.”

I use Matchstix as a low fill fixture to enhance the lighting effect of the judge’s interviews. The Mavericks with Matchboxes are used for smaller areas due to the irs small dimensions.

TIME-SAVER
Compared to other lighting options, I’d pick Cineo every time. The products are designed for what we want to accomplish with our lighting. Not only can Cineo lights handle the rigors of our intense travel schedule, but the features make my job easier when time is of the essence.

For example, the HSX features an output control that automatically increases lighting output one full stop (from 20 percent, to 40 percent, to 60 percent, etc.), with a userfriendly interface that allows our team to read the exact light measurement. This helps keep the camera and light totally coordinated and saves us a lot of time not only on the set, but in post-production as well.

The amount of time we have saved in setup and post-production due to the color-accuracy and dependability of Cineo products is second-to-none. They are an integral part of how “America’s Got Talent” is lit and filmed.

Darran Webb is a realty television lighting designer. He has been working with “America’s Got Talent” in 2006. He can be contacted atdarranwebb@mac.com.

For more information, please visitwww.cineolighting.comor call 310-425-3425.