I have written about supercomputers in the pages of TV Tech for more than six years. I even had an ABC director tell me my musings on computer dominance in broadcasting were pure folly and that I was “full of it.” However, my earliest contemplations did not take into account learning models which, to me, now further legitimize the possibilities of AI in broadcasting.
Let’s use my friend at ABC as an example. Bob has less than a dozen cuts in his directing repertoire, with only a handful of permutations between them—very predictable. Now, if you take it to the next level, Bob-Bot can remember these camera sequences and their outcomes, plus how effective their entertainment value was. You could even teach Bob-Bot the patterns of different directors and producers and mix up a pretty good show.
FIFA World Cup soccer and Host Broadcast Services have led the way in advancing sound quality and also in developing computer-assisted sound.
Not only was the concept of mixing matches in the broadcast center’s master control developed at the World Cup, but Lawo also created a mixing algorithm that compares sound intensities from different microphone positions, selects the best ones and creates the most desirable sports sound. I wrote in the pages of TV Tech that I thought the 2022 World Cup was the best-sounding World Cup ever and better yet, the best-sounding sporting event ever!
From Assistant to Controller
The difference between the early AI systems and current (and future) systems is that AI brings the ability for the computer to learn and remember. For example, a machine model can combine audio sources and create a mix from the existing data. A learning model will be able to elevate the entertainment value with more and better permutations each learning cycle.
Computer-assisted is quickly moving to computer-controlled. It is easy to see how the entire production chain can be computer-controlled. Cameras do not need operators anymore, with autofocus and simple commands like “follow the ball” or “follow car No. 6.” I was resistant to “audio-follow-video” for many years because I thought the switch did not sound very good and subjective control was the job of the mixer. Basically, my prejudices were ego-related and not engineering-related.
The Lawo “mixing assist” compares sound levels from different microphones and adjusts mix levels for the most desirable combination of sounds. So, what about computer-generated sound fields and sporting sound effects, like what Ben Shirley and Rob Oldfield of Salsa Sound have been developing? The Salsa program not only can mix, but can change the timbre, texture and composition of sounds. I imagine a ball kick with a little thunder or dynamite explosion added to it. That is entertaining!
Entertainment vs. Reality
When gaming came along, I realized my role as a sound mixer was to entertain the audience and not necessarily document the sound image. Some sounds were boring or not easily captured, which is why, in the early ’80s, I started to use a sound sampler—a primitive analog computer to play back and enhance the sound field—everything from pit sounds and tire squeals to canoe rows and crowds. Engaging sound, camera work and graphics drive broadcast retention, but also cost money, unless perhaps there is a heavy dose of generative computer influence and output.
The difference between the early AI systems and current (and future) systems is that AI brings the ability for the computer to learn and remember.”
Every sport can develop an audience via the internet, and there is clearly an abundance of content that is captured and streamed. Not only are supercomputers good at automating tasks and improving efficiency, but they are capable of creating realistic content. How about nude beach volleyball? How will content production keep up? Through AI.
Supercomputers with learning and adapting capabilities are known as “generative,” where computers create new content using similar or existing data. Think about the commentators. You rarely see them, and they tell the viewer what they just saw and then read something about the person, place or thing that was just shown. It is called play-by-play and color, and I think these folks are on the way out.
I have read postings on LinkedIn for audio practitioners to train AI algorithms. That makes sense, since the power of artificial intelligence lies with its ability to ultimately learn and think for itself. It also seems that the best models have a deep reservoir of knowledge to build on.
A Better Model Me
But where is this knowledge coming from? Well, one significant avenue has been through theft of intellectual property.
America has long had a beef with China over intellectual property theft, but it is happening right here in the United States—with impunity. I don’t know who is to blame—the publisher or the AI company.
I heard on the news recently that Anthropic, a leading AI company that I thought had scruples, has settled a class-action suit for copyright theft, which included two of my broadcast publications with Focal Press. If you are going to train a computer to do my job, then learning the fundamentals of signal processing as well as the aesthetics of signal balancing will make a better Model Me.
All sound disciplines are changing and the evolution probably will not stop. No doubt, analytical tools for quality control are soon to arrive, if they are not already here. Things like distortion, phase and loudness are easily quantified and corrected. While working on this article, one of my sources said they have developed a profanity eliminator that works basically in real time and even in different languages.
I used to think the safest jobs in television were held by those who set up and maintained the equipment. I don’t see that as the case anymore when I see stories of Chinese robots playing soccer. Television broadcasts are not hard to set up, and any deficiencies can be compensated or covered up by the computer.
Are you scared yet?
