About 20 years ago while attending a trade show in Las Vegas, I crossed a walkway between buildings along The Strip. I stepped out into the dazzling light of one of the new outdoor digital billboards that had just started to appear. In glancing away from the considerable glare, I saw that everyone on the walkway was bathed in an ever-changing light from the garish display.
For me, this accidental lighting was a lot more interesting than whatever was being hawked on the billboard itself. The play of light was animated like fireworks on the upturned faces in the crowd. It was a thing of beauty in a sea of over-the-top excess.
Although I didn’t realize it at the time, that was my introduction to image-based lighting—years before anyone had named it or figured out how to use it.
Virtual Production
In its most basic form, LED panel lights consist of individual pixels. Advanced versions can be mapped out to form a kind of low-resolution display screen. At the point where you can “drive” these lights with images played across the light’s face, you have an image-based light (IBL). Need a larger area? Just add another panel and map it accordingly.
If that sounds similar to a video wall, it is.
IBL fixtures and video walls actually have a lot in common with each other. Dot-pitch, brightness and color quality may be different, but they’re both fundamentally screens. One is designed to be viewed directly, while the other is indirectly viewed by way of its incidental light and reflections.
When used together in the realm of virtual production, the results are the latest in movie magic—with the benefit of capturing realistic virtual environments and the actors’ performance directly in the same camera shot.
IBL fixtures are now integral to special effects production.
Special effects (SFX) used to rely heavily on “green screen” and required extensive post production to piece effects together. Virtual production studios can now accomplish that in one pass, permitting virtual backgrounds to be simultaneously combined with live foreground action, representing a transformational change in the industry.
Shows such as “The Mandalorian,” and current “Star Trek” shows depend heavily on virtual production within LED volumes.
One byproduct of shooting within these immersive LED volumes is that ambient light from the screens, along with their reflections, illuminate the foreground—much as they would in nature.
Deficiencies
There are, however, still a few problems.
Most video screens are made of RGB (red, green and blue) LEDs. Although these screens look fine when viewed directly, the RGB emitters give off light that’s deficient. It lacks the spectral fidelity needed for accurate color reproduction. This is particularly noticeable on flesh tones.
In an attempt to address these deficiencies, some video wall manufacturers are beginning to take a page from the lighting fixture industry, broadening their color spectrum by adding phosphor-based “white” LEDs to the RGB mix. Although the color fidelity of these RGBW displays don’t quite equal the accuracy of purpose-built lighting fixtures, they’re a big improvement.
In practice, using these white LEDs becomes a balancing act between how it looks to the camera and how it lights the actors. One company addressing this challenge is Emmy Award-winning Brompton Technologies, which developed “TrueLight” to better control the light balance. Breakthrough advances like this are part of the ongoing evolution in lighting today.
It’s now possible to open up a virtual area of white light within an LED volume (but outside of the camera shot) to help illuminate a scene. At the moment, however, the best LED volume walls are still somewhat deficient, only scoring a color accuracy of around CRI-80 out of a possible 100. Those displays will eventually improve, although change will require considerable investment.
IBL fixtures are inherently better at lighting the foreground by all measures. And their ability to work in conjunction with the dynamically changing images can now permit a seamless match with background images.
Accurate Color Reproduction
Inspired by the industry’s need for better lighting tools, several manufacturers have stepped up. Among them, Kino Flo and Quasar Science have created lights that convincingly match virtual backgrounds, as well as providing for accurate color reproduction.
For example, envision a car chase past a row of street lights. While a conventional light gag could be used, the IBL fixtures can more accurately mimic the movement of the street lights past the car by inputting the moving street lights directly. Combined with the matching images in the Volume, these tools work together to build a more convincing illusion.
To accomplish this, IBL fixtures have taken a page from video displays to become a sort of low-resolution video panel, but with more output and higher color fidelity. Likewise, LED volumes have borrowed some tricks from lighting. Micro LEDs and flip-chips create tighter pixel structures that allow direct recording without moiré. This tight clustering has also enabled the addition of “white” LEDs to improve the color fidelity of the light the volume emits.
Image processing that was originally developed for the video game industry adds to the realism. 3D creation tools, such as Unreal Engine, have made it possible to add real-time background perspective changes that appear in-camera.
The amount of control required for IBL lighting fixtures has exposed the limitations of the DMX-512 lighting protocol. Fortunately, there are already practical workarounds. IBL lights require input control that more closely resembles a video feed. Designing better control solutions will be an ongoing process.
These are the latest tools for the literal expression of virtual light. They’re already playing an important role in media and entertainment production. We should also remember that the “poetic” interpretation of light can be of equal importance in storytelling.
Conventional lighting fixtures will remain important tools in production, but the unique capabilities of IBL fixtures open new possibilities for lighting effects beyond virtual production studios. The full impact of these new technologies is a tale still being written.
