I like to watch the beginning of big sporting events and was looking forward to hearing and seeing the band The War and Treaty sing the Star-Spangled Banner on “Monday Night Football” back in November.

The sound started out bad and, surprisingly, never got better during the entire performance. Clue one: Basic troubleshooting. Two singers were on wireless microphones, accompanied by an acoustic guitar—direct—three faders. Each channel was distorted and the balance between the sound elements never changed.

Before submitting this article and blaming the “MNF” crew, I decided to check out YouTube, where The Sports Video Channel was credited, not “Monday Night Football.” I am glad I did, because I was shocked—this certainly was not the same mix I heard live on my ABC affiliate.

What happened and who is really listening?

I can only ask, what was the mixer listening to? What was master control listening to? It reminds me of a favorite saying from the late, great television producer Fred Rheinstein: “Are we doing the same show?” What are you listening to?

Staying in Phase
In the early days of stereo sound over analog copper wire, it was not uncommon for the left and right channels to get out of phase. Often, this would occur in transmission, clearly beyond the event mixer’s control. It became common to send a “split track” of the announcers on the left channel and the other sounds on the right channel, with the two-channel (stereo) mix taking place back at master control. That worked, but I never thought it sounded very good, especially when you used an Orban stereo synthesizer to create stereo—“phase-y” stereo at best.

Surround sound was a nightmare with the widespread use of Dolby Pro Logic, an analog synthesis of surround sound delivered over two analog audio channels. If you had the decoder, then you could decode the surround, but if you didn’t, you had what was dubbed “super stereo”—once again, phase-y-sounding stereo. Dolby Surround was problematic until digital transmission and the first set of ATSC standards.

As a very green sports location sound mixer, I quickly learned to listen to and monitor the output of the OB van and as many other places as possible. There could be several processing or signal-splitting stages before the sound leaves the truck, and a problem could easily happen at a spot where you may not be listening.

After the audio program leaves the audio production space, further processing may happen somewhere in the audio signal flow. Isn’t someone listening to the sound? I remember the story about a master control technician who told the field mixer, “the meters looked fine there.” Who is listening to the sound?

But the question still lingers on how can the audio mixer produce sound for the masses when the consumer listens on earbuds, TV speakers, and sound bars? Not to mention the sound may be listened to in stereo, surround and even immersive by a few. How about language? Language intelligibility has plagued the broadcast sound mixer since advancing from mono to stereo. In mono, there is no gimmick like a “phantom center” that could disappear when the left and right channels are out of phase.

Immersive Challenges
Surround sound was difficult for the sound mixer because there were four channels of event sound and music and only one channel for dialogue. Compounding the mixing challenge is speaker alignment and placement, and often the surround speakers are too close to the mixer. Significantly, if the center speaker is too close it can give the sonic impression that the voices are too loud, resulting in the mixer turning the voices down and making them hard to hear.

With immersive sound the problems are further complicated by the fact that you have just added four overhead speakers. Now you have between eight and 10 effect and music speakers and still just one voice channel.

Part of the problem is how we define channels and how we mix them. I do not see any reason to not put voice in the left and right channels or in the front immersive channel, in addition to the center channel. You might argue that true reproduction in the home may be off, but then consider how sound bars project the sound. Who are you mixing for?

Mixing audio beyond stereo is arduous because of speakers and speaker placement, but significantly because of the acoustic mix space. World Cup soccer is hosted in multiple locations with different equipment and mix spaces making a cohesive consistent sound challenging.

Beginning in 2010, Felix Kruckles and Christian Gobbel of HBS (Host Broadcast Services) devised a signal flow and schedule where all the World Cup matches were produced in stereo in the venue and stems were sent to the International Broadcast Center. Then, a surround or immersive sound overlay was blended and mixed in a proper mixing room. All matches were mixed in the same audio mix studio—that is consistency! Felix and Christian were listening.

Who is listening? The misuse of compression is at an all-time high, and I would bet that it did not sound like that in the audio room. When there were only a couple of channels of compression, compression was tricky. Now virtually every audio channel and signal path has compression available, and maybe that is the problem—the compression is cumulative over the signal flow. I hear overcompression every time I turn on the TV. I bet it has to do with meeting the loudness numbers required by law!

Who is really listening? Maybe quality control is the best use of AI. You could program a gazillion qualitative and quantitative factors into a “QC bot” and it could “steer” the mix with some DSP into perfection so we can all listen in high fidelity.

The real question is, what did I really hear? I know I am not crazy! Yet.

On an NFL game day, the mix isn’t just sound, it’s the heartbeat of the broadcast. The goal is simple: Make viewers feel like they’re standing on the field without overwhelming the action. Neither Daniel Haggard nor Jeremy Katz originally imagined themselves shaping the audio for primetime NFL games, but once they found their way into broadcast audio, both discovered the rare mix of pressure, precision and excitement that keeps them coming back.

Haggard started in the early 2000s in Syracuse, back when the workflow was still mostly analog and troubleshooting was half the job. His big break came in 2015 when veteran mixer Wendel Stevens invited him to join NBC’s “Sunday Night Football,” a leap that meant six months on the road with a brand-new crew.

“It was terrifying and thrilling,” he said. “Only 6 million people watching.”

Seven years as an A2 gave him hands-on experience with RF, comms and field audio. That training made stepping into submixing a natural next move.

Katz’s route looks different. He cut his teeth working with bands, eventually landed at regional sports networks before joining Prime Video’s “Thursday Night Football” four seasons ago. His philosophy is straightforward: “My job is to capture everything happening on the field and hand the A1 something solid and clean. I try to be as aggressive as I can, and the A1 can decide what’s safe for air.”

Katz said the key is chasing clarity, not noise for the sake of noise.

Showing and Telling
Both mixers talk about audio in practical, picture-first terms. If something is happening on screen, the viewer should hear it—simple as that. From the quarterback’s cadence to a clean catch or the thump of a tackle, the goal is to match the energy and detail the cameras are showing. Anything that’s unnecessary, distracting or muddy doesn’t belong in the mix.

One tool they fully agree on is the mic mounted on the SkyCam. Hovering just above the action, it’s positioned closer to the players than almost any other microphone. Katz calls it “the best mic on the field,” and Haggard uses it as a bed of natural sound.

From there, player mics provided by the league give him extra detail on the line. He brings them up after the huddle breaks, rides them through the snap, then pulls them down quickly to avoid smearing the rest of the soundscape.

Parabs are another staple. Both mixers prefer committing to a single parab per play rather than blending multiple at once.

“Quality over quantity,” Katz said. “You get a clearer picture when you’re not stacking too much.” Haggard agreed that picking the right mic and trusting it is part of the craft. Sometimes you guess wrong, but when you guess right, it feels perfect.”

Nowhere is that more true than with the most coveted sound in NFL audio: the “doink.”

Across crews, mixers keep a friendly, season-long competition over who can capture the best goalpost hit. Katz joked that the entire “TNF” truck starts cheering the moment a kick drifts toward the upright.

Haggard remembered one playoff game where everything lined up: the contact mics on the post, the 416s under the crossbar, the tension of the moment and Mike Tirico’s call. “If that mic was down, it would’ve ruined the whole moment,” he said. “Instead, it was perfect. I rode the adrenaline all the way back to the hotel.”

An Art and a Science
Mixing the NFL isn’t easy—you’re dealing with massive crowd surges, stadium PA music, sideline chaos, sweaty linemen, unpredictable ref mics and dozens of camera perspectives that shift direction every few seconds. You’re anticipating where a play is going before it happens. You’re making choices that millions of viewers will react to instantly.

Both mixers emphasize that it’s a creative craft, not just a technical job. You learn by watching others, borrowing ideas, shaping your own style, and adjusting to each game’s personality. “There’s no single right way to mix,” Haggard said. “You pick up techniques, try new things, and keep refining. That’s what makes it fun.”

For viewers at home, the result is seamless: Every snap, tackle and catch lands with clarity and intensity.

Behind the screens, it’s hundreds of subtle decisions: mic choice, fader ride and EQ tweaks that make the difference between a flat, lifeless broadcast and one that truly feels alive.

I often hear there is a shortage of top-tier sound mixers. I think I have heard that since I did my first show with ESPN back in 1983, and I would say there was possibly a shortage of television technicians at that time, particularly in audio. ESPN found me in a recording studio and was patient enough or, as Donna Shackett once said, “desperate enough to train you.”

I was told that in the early days of ESPN, the Big Three television networks, ABC, CBS and NBC, even discouraged their union technicians from working freelance for the fledgling all-sports cable network.

Is the shortage of qualified people because the education, recruitment and training process is flawed, or what?

I quickly learned there was a process for grooming audio folks. You started in the field and worked (earned) your way up the ladder into the OB truck or studio. This was the opportunity to find a mentor, because training was on the job.

There was limited technical or college curricula and it was a time when good troubleshooting skills and some electronics went a long way. Now there are schools with audio OB vans. While this may be good for the school’s recruitment, the graduates are probably a ways off from mixing an event and are now probably in debt.

More Automation, More Stress
When I hear that there is a shortage of top-tier qualified mixers, I tend to think that there is a shortage of certain qualifications. As early as May 2021, I wrote in TV Tech about computer-driven mix automation before the term AI was even used. The new-era audio supervisor/producer role is not about beautiful-sounding productions; it’s about getting on and off the air cleanly, not up-cutting announcers or playback, and making sure the director’s and producer’s headsets are working flawlessly.

The audio supervisor must not only plan for multiformat audio output for a potentially wide range of playout options—plus plan for archives—but also program the traffic over intercoms. The job can be intimidating.

When I talk to some audio assistants, I have often heard there is not enough money to move up for the stress that comes with the “hot seat.”

What Went Wrong?
The change from analog to digital technology accounted for increased productivity, but also demanded an increase in technology skills.

It did not help that the development and learning curve of almost all broadcast equipment was happening at the same time. The analog-to-digital migration seemed to develop first with intercoms. It wasn’t enough to have exponentially more channels; there were now significantly more options and possibilities—and often more headaches. The largest shows often required an intercoms specialist, but just getting a couple of channels of intercom on a basketball game with an inexperienced audio crew and a grumpy engineer in charge sometimes seemed daunting.

Digital technology is well-entrenched, and there are far fewer new devices. Operationally, the equipment has more similarities than differences, which relieves some of the audio mixer’s anxiety.

There is more demand for content than ever, which translates to more productions, producers, technicians and engineers—often doing multiple tasks.

To head off labor issues, in 2014 ESPN hired hundreds of audio technicians in a move that the network said was to “insure continued technical support” with its meteoric growth in sports production not only for ESPN, but also for ABC.

How and Where to Start
So how do you get an entry-level job in broadcasting? The press on AI would lead you to believe that it will replace entry-level positions. Just like with most television audio professionals, you start in the field or on the stage and work your way up. With more productions than ever, there is a lot of on-site setup that has to be accomplished. Besides the basics of needing to set up microphones, there are still a lot of intercoms.

For the new-era audio supervisor, managing all the audio at an event will become the norm. They will be mapping the signals, mixes and outputs, plus all communications, as well as supervising the setup crew—and maybe from a distant location.

AI has proven itself as a mixing assistant for effects and even enhanced supplementation. When I wrote about AI, “audio intelligence,” I was told by some television directors and producers that I was full of it. Maybe, maybe not. Computers (AI) are good at logical replication, such as camera rotation over a baseball diamond, but I have never seen a computer set up a microphone or headset. Come to think about it, I have never seen a producer or director pull cable or program an intercom.

If you want a job in television, learn how to properly set up microphones and program a mixing console or intercom. Those entry-level jobs are not going away or getting replaced by high-powered computers, like some production and directing jobs.

I once told a university class that if they ever saw a television truck, follow it and volunteer to unpack, pull cable or whatever you can do to make yourself useful.

There’s a moment in fly fishing when you spot a new bend in the river for the first time. The current moves differently here. It’s deeper, faster, with little eddies and pockets where something might be waiting. Standing there, you realize that finding success is more than just showing up. You have to read the water, pick the right fly and cast just right.

Like a river, a live broadcast is in constant motion. The pace changes from slow stretches to sudden bursts and the flow can turn without warning. Cameras have to be in position before the play unfolds. The replay team has to grab the perfect angles in seconds. Graphics have to update in an instant. The audio mix needs to capture the roar of the stadium without drowning out the announcers. A single missed cue can ripple through the entire show. As a technical manager, I’m the one watching the whole current, making sure every part of the system is moving in harmony.

I’ve seen just about everything that can happen on a live production: power failures; cables going bad; graphics systems locking up moments before going to air. The challenge isn’t just fixing the issue; it’s doing it without breaking the rhythm of the show. In fly fishing, if a tangle forms in your line mid-drift, you don’t freeze or throw your hands up. You work it out quickly and get back to fishing before you miss your window. Broadcasting is the same way.

Knowing When It’s Flowing
When a crew is in sync, the production takes on a rhythm you can feel. Everyone knows their role, but more importantly, they know how their actions affect the people around them. A camera operator frames a shot with the replay team in mind. An A1 rides the crowd noise just right so the announcer’s words cut through. The coordination is almost invisible to the audience, but to those who live and breathe production, it’s the difference between simply getting the show on air and creating something that pulls people in.

Technology has reshaped the craft in ways that were hard to imagine when I started. We’ve gone from analog boards and fixed-truck setups to IP-based workflows, 4K resolution and fully remote productions. Some of those changes have made life easier. What used to take hours of patching can now be done with a few clicks. But new tools bring new kinds of problems. The day you assume everything will work perfectly is the day you get blindsided.

Being There
Nowhere has that evolution been more dramatic than in audio. Today, it’s about creating an immersive, almost cinematic soundscape. Viewers expect to feel like they’re in the building, on the field, even inside the huddle. That means capturing more than just the obvious. It’s not enough to hear the crack of the bat. You need to hear the chatter from the dugout, the sharp intake of breath right before the pitch, and the eruption after contact.

We’ve seen a huge shift toward spatial audio and object-based mixing, where sounds aren’t just panned left or right, but placed in a three-dimensional space. For sports especially, this opens up new ways to tell the story: The mix captures the full story from the quarterback’s calls, the sideline voices and the crowd’s energy, all working together to bring the game to life. It’s not just a mix, it’s an experience.

Microphone technology has also advanced in ways that make a difference on every show. From wireless lavaliers that give athletes mobility without losing clarity to mics that can survive being buried in ice without compromise, the tools we have now let us grab sound that would have been impossible decades ago. Wireless reliability has improved dramatically, but with that comes the challenge of managing dozens of channels in a congested RF environment. That’s a part of the job the audience never sees, but it’s as critical as any camera angle.

Flexibility Is Expected
As productions get more complex, so do the expectations for flexibility. IP audio networking has given us the ability to route and reroute sources anywhere in the world in real time. That means an A1 mixing from a facility hundreds of miles away can have the same control as they would in a truck on site. It’s efficient, but it also demands a higher level of planning. If there’s a hiccup in the network, it’s not just an inconvenience—it can take your entire show down.

The future of broadcast audio is going to be shaped by two competing forces: the drive for ever-greater immersion and the demand for leaner, more remote-friendly workflows. On one side, you’ll have technology pushing us toward hyperrealistic sound environments that rival live attendance. On the other, you’ll have budget and logistical realities pushing productions to do more with less resulting in fewer people on site, smaller crews and more automation. The winners will be the teams that can balance both without losing the soul of the show.

For me, the audio side has always been about authenticity. You can have the sharpest 4K picture in the world, but if the sound doesn’t match the moment, the emotional impact is gone. The excitement of a game-winning play comes alive when the sound matches the energy of the moment. Like reading a river, capturing that moment is a mix of skill, preparation and instinct. You can’t always predict when the big fish will strike or when the perfect crowd swell will happen, but you can be ready for it.

Broadcast sound is crafted from a variety of microphones so that the final/composite sound field replicates what your ears really hear and the information your brain processes as the sound environment. It is amazing that your brain, along with some directional head-pointing, creates and mixes the soundtrack of life, everything around you, with all of its glorious sonic details. Plus, the brain remembers what you think it all sounds like and what you like.

A sound field is a composite of all the acoustic environment’s sounds with all the different intensities, moshed with direct and reflected sound waves that all blend and interfere with each other to create an “acoustic gumbo.” This is the simple reason why it is impossible to capture all the details of a soundscape with a single microphone.

Basic microphone physics begins with a single transducer/diaphragm capturing the entire sound field. In theory, a single omnidirectional microphone should be able to capture the sound field—like your ear—however, there are significant diminishing returns the further you move away from the microphone diaphragm. An analogy—a camera lens can focus in on an object for a “close-up” while a microphone cannot.

Separating the Noise
Typical sound environments are full of noise, which attenuates the clarity and fidelity of a sound source. Go into a venue and just listen—there is a base level of sound from HVAC systems and ice coolers, but mostly from the diffusion and decay of many complex sound waves. The problem is that your audio capture must be above this base layer of noise.

Early sports sound depended on just a few microphones that often blended a pleasant holistic representation of the sound field. Shotgun microphones inherently overreach their targeted sound and detect background clutter, like spectators. Often, this resulted in a good balance of sport and spectator sound from a single microphone. It was not until surround and immersive sound that the audio mix needed more defined spectator zones, which required the spectator atmosphere and ambiance to be captured separately from the event-specific sounds.

Mono transducers inherently have “full omnidirectional pickup” and are crafted for separation by physical characteristics like housing, porting and mounting. For example, a directional microphone is designed to reject directions and frequencies either physically by construction and design or electronically. Construction and porting creates cancellation of certain frequencies in the sound field that may not be desired, while electronics are also used to shape aspects of a sound field through cancellation, equalization and electronic manipulation.

Further, with fast computers, there has been success with multicapsule microphones that are capable of specific pattern isolation known as “beam forming.” The beams are electronically mapped and able to track the focused microphone beam pattern on a moving object such as an athlete or a ball.

Additionally, combinations of microphone capsules are used to create the illusion of stereo, surround and even immersive sound.

There is no doubt that the materials and technology of modern microphones are capable of a close-to-perfect representation of a sound field; however, capture still seems to be a mystery. A mediocre microphone can sound decent if it is properly mounted. Unfortunately, I am surprised that I still see shotgun microphones flat on camera lenses—check out baseball.

How to Aim a Shotgun
A shotgun microphone is designed and constructed to have focused-forward reach where the microphone is pointed; however, this requires full 360-degree free-field sound, not reflective sound from the camera lens. When the shotgun microphone is placed flat on the camera lens there is a “boundary” boost, but there is also frequency alteration.

Microphones benefit from accurate and proper axis and direction orientation. When stereo shotgun microphones were introduced, there was some carelessness with the left/right and even the up/down orientation of the microphone, so the sound didn’t match the picture. In a quiet setup, this may be difficult to detect, but it is obvious when the microphone is mounted on a handheld camera.

Television sound is about presenting an appropriate enhancement to the picture subject to interpretation, taste and the technology of the day. Most sound mixers work hard to get those singular microsonics that are unmistakably the event, but the great sound mixers have clean capture and a knack for that proper sonic blend.

Even though the entire broadcasting signal chain is digital, the beginning and the end are still analog. We live in an analog world and we ingest analog sound and visuals and convert frequencies to brain waves. So when the A1 sound mixer asks for some more microphone, humor them, because you know why.

I remember being glued to the television set every Sunday night to hear Ed Sullivan say, “We have a really big show.” Those were really big shows for audio. The stage changes for “The Ed Sullivan Show” were managed from stacks of analog radio and PA mixers feeding microphones and audio signals into each other. A musical act, standup, puppets, animals and with Ed in front of a live audience, it sounded pretty good. The crew moved microphones around, made chalk marks on the audio mixer, unplugged and plugged in equipment and prayed that there were no failures or miss-patches.

With “The Ed Sullivan Show” (aka “Toast of the Town”) there was only one musical act with basic instruments, but the future of big television shows has resoundingly come to mean big musical productions. Possibly 1985 was the year that stretched the analog resources almost to the breaking point with live broadcast music productions from Farm Aid and Live Aid, but help was on the way. By 1987, the audio world saw the commercial release of digital mixing consoles.

Instant Recall
Complex changes in signal flow and processing are easily possible with digital consoles because of the ability to instantly recall a flow or setting. It is hard to imagine any music awards show without the luxury of being able to retrieve an entire audio production from a rehearsal or sound check or even the drum sound from the studio.

Shows like Farm Aid and Live Aid had weeks of planning, but a recent music production accomplished a large-scale event with just a few days. The FireAid benefit concert, an event supporting the victims of the recent L.A. fires, had 30 different artists, each with a seven minute change-over goal between acts. There were over 300 audio channels available and each act had a sound check during the couple of days before the event. The “soundcheck” was stored and recalled across four identical DiGiCo mixing desks for the day of the show.

Large-scale commercial audio mixers with more than 48 channels were not available until the early ’70s and those extra audio channels required more electronics and physical space. Overweight equipment and miles of copper wire were not good for the developmental days of outside broadcast and it cannot be understated that digital technology has made possible the really big audio extravaganzas like the Super Bowl, music, awards and live performance talent shows.

Hundreds of audio channels are available in a manageable frame size because of the concept of shared space and the use of virtual layers under a single control surface. For example, a single channel strip with a volume fader and controls for equalization and dynamics could be designed/ programmed to control as many channels below the top surface as the processing power (DSP) would allow. This means a top layer of 36 channels doubles to 72 by adding another layer and jumps to 108 with a third layer all contained in the same frame. It is not uncommon to see four- and eight-layers deep for 288 channels. For reference, a 36-fader mixing desk would be about 4 to 5 feet wide.

Channel density was a game-changer, but to me, one of the most significant contributions from digital mixing consoles to high-fidelity sound was that you can insert dynamic range options (compressor/limiter) into any section of the gain stage. In the ‘80s, I worked on a CBS OB van that had three compressor/limiters: basically one for the announcers, one for effects and one for replays. This clearly presented a compromise in settings, particularly with three announcers. Current digital technology allows each audio source to have dynamic range controls making for a much-smoother-sounding show.

Have We Peaked?
Finally, the utility aspects of digitized audio can not be overstated. What used to take thousands of feet and hundreds of pounds of copper wire now uses a few strands of fiber. Even though the contribution of digital audio to the audio practitioner is nothing short of a miracle, digital technology significantly reduced the operating cost of a production by reducing the weight to transport and the time needed for pulling cable and engineering setup.

Consider this: Perhaps broadcasting technology has peaked with its complete transformation from analog to digital! Perhaps the future of the big broadcast experience is producing big sounding entertainment and content.

That’s not hard—the fastest, most consistent way to a big sound is through digital electronic manipulation of the audio. For example, a little reverb, delay and processing go a long way toward creating sonic space and interest. I am all for electronic up-processing of audio, which, by the way, gained acceptance by broadcasters in the early days of stereo and was used by NBC to enhance the entire stereo production of the 1988 Summer Olympics. Even recently, Fox Sports upconverted/processed its Super Bowl LIX production to 4K and Dolby Atmos.

Digital audio allows for instantaneous changing and sculpting of the sonic qualities of a sound mix, including dynamic spatialization. Digital technology offers the benefits of a creative audio pallet to create sonic Nirvana—however the generative process.

Spend a little time on the sonic qualities of your mix. A great-sounding show gets attention, someone is listening.

Another year has passed and it seems there has been little progress toward improving the sound of television, plus there are the same old problems with microphones, compression, synchronization, audio-follow-video and ATSC standards.

With COVID-19 came the proliferation of stay-at-home journalism, principally with so-called experts who seem to fill time and echo the thoughts of the studio commentators. The occasional off-microphone talking head was tolerable, but with over four years of this practice, I find this unacceptable and very unprofessional.

The computer is a wonderful all-in-one streaming device with automatic gain control for poor lighting and some level of bad sound, but someone decided that poor lighting and “raccoon eyes” were not a good look for television. Quickly, light rings and the occasional background lighting on the bookcases or guitars were adopted, so the viewer could see the subject’s faces and how cool they were by what they read or did. Most news stories could exist without a fuzzy portrait of a talking head but when speech intelligibility becomes a problem, what’s the point?

Upgrade the Home Studio
Every home journalist should have a kit that includes lights, a real microphone, earbuds or headphones and an audio interface for the computer. The lapel microphone is often the first choice because of its small size, but that size also makes the microphone more omnidirectional. Thus, it picks up more room clutter than a larger diaphragm microphone with more directivity that picks up the desired voice and less room tone. Obviously, the art director would rather not see the presence of a large microphone, while the sound person wants better and closer sound capture.

Earbuds or headphones may be a bit of a stretch for the home journalist, but sometimes you can hear the audio return from the network if the sound from the computer is too loud. Finally, an easy improvement for the serious home journalist would be some sound treatment to reduce the sound bouncing around a big room with pretty books and a guitar in the background.

Every day, I am shocked at the misuse of audio compression across virtually all networks. How often have you heard the audio level drop dramatically and then recover a few milliseconds later? Audio compressors have an attack time and a release time that govern the function of sound compression.

A quick attack time is often undetected until a slow release time seems to keep the audio levels too unnaturally low for too long. Conversely, a quick release can make the audio sound like it is pumping up and down, electronically controlling the sound levels. My preference is to limit the audio peaks to not distort the program and to use your fingers and ears to find the right balance. Before there was compression or limiting, there was mixing.

Too Fast, Too Furious
Live television production is fast and furious and it can be challenging to get the right microphone fader up at the right time and to the correct level when the director is bouncing all over the place with more cameras than they know what to do with. College football is the worst, because the handhelds are close to the crowd and the poor audio person often does not know when the director will cut to that handheld. Most directors will argue that that is not the case, but in my 40-plus years of live sports mixing, I have heard more than once, “Ready camera two, ready camera four—take camera seven.” What?

Since the beginning of outside broadcasting, the video switcher has been capable of switching camera tally lights and microphones on and off. As a young, inexperienced sound mixer, I remember that I don’t need audio-follow-video until I uncut a couple of cheerleaders who had a camera in their face and heard about it from the director. AFV—audio-follow-video, even in its most basic form—is better than “uncut” sound.

My final two pet peeves have proven more difficult to tame and are beyond the control of the audio practitioner: lip sync and standards. Every year, I write about lip sync, but there seems to be little progress toward implementing a solution—probably because there are so many places for the sound to get out of sync.

Staying in Sync
I know you have seen programs where the content is out of sync and the commercials are in sync, and I know you have seen live content where different elements within the show are out of sync. Golf is the worst!

Finally, the ATSC needs to adopt—and the FCC needs to mandate—immersive sound as the audio standard. We probably would not have immersive sound because we probably would not have had surround sound if the ATSC and FCC had not selected the latter as the sound standard in ATSC 1.0.

I have covered all these issues on several occasions in these pages, including things the ATSC could endorse—for a long time now.

There have been a series of significant milestones in the evolution of Olympic broadcast sound culminating in the greatest audio production of an Olympics ever. Clearly NBC has led and dominated the soundwaves for more than four decades and 2024 is the pinnacle of their persistence.

Host Broadcasters
Basically, live television sound was mono until the 1980s with the first Olympics stereo broadcast in 1988. Under the direction of sound designer Bob Dixon, NBC placed a shotgun microphone alongside the host broadcaster’s single shotgun microphone to capture XY stereo. This was no easy feat politically or technically since the Host Broadcaster Korean Broadcasting System (KBS) was only obligated to delivered mono sound to the rightsholders, including NBC.

The Host Broadcaster was traditionally the national broadcaster of the host country, but after 2008, Olympic Broadcast Services (OBS) became the permanent Host Broadcaster, fully under the direction of the International Olympic Committee.

The Host Broadcaster for the 1992 Games in Barcelona was TVE, Spain’s national broadcaster, where they produced the Opening and Closing Ceremonies in stereo, but sports were captured and produced in mono. The year 1996 was the first Olympics that the Host Broadcaster, Atlanta Olympic Broadcaster (AOB), captured and produced all events in stereo sound.

After 1996, Dixon encouraged Mike Edwards and Ken Reichel of Audio Technica to manufacture a stereo shotgun. In fact, Edwards initiated and supervised the development of three stereo microphones and two mono shotgun microphones, which were still heard on all Olympic sports and ceremonies in Paris.

Signal Management
In addition to the extensive development of suitable microphones, there was considerable work to be done with signal management and distribution. Broadcasters had to solve the problem of multichannel audio over a stereo infrastructure. Various schemes were developed to get more than two channels of sound to the home viewer/listener, but it was not until the sound was digitized that a credible surround sound was possible for distribution and transmission.

In 2006, after consulting with his audio director, Olympic Broadcast Services chief Manolo Romero determined that producing surround sound with unprocessed, discrete audio channels was the best way to satisfy the needs of all the rights holders at the Olympics—including NBC.

With 2008 came the implementation of surround sound at the Summer Olympics with the host broadcaster delivering six discrete channels of sound for the 5.1 sound format.

The sound of the 2008 Games was a significant challenge for NBC because so much of the viewing/listening audience was still listening in stereo since soundbars had only recently entered the marketplace in early 2000s.

NBC continued to develop surround sound with various Dolby analog schemes, but it was NBC’s adoption of ATSC 1.0 with Dolby AC3, along with a market full of affordable soundbars that multichannel sound would take off.

Immersive Sound
NBC polished its surround sound coverage but was persistent with the goal of true immersion. For the 2012 Games in Russia, NBC Sports Sound Designer and Olympic Supervisor, Karl Malone vigorously pursued immersive sound, beginning in Rio where NBC mixed Opening and Closing Ceremonies in immersive sound.

Pyeongchang and Tokyo followed, and immersive sound was heard in both Opening and Closing Ceremonies as well as big venues. For Beijing, all primetime coverage was produced in immersive sound. Finally in Paris, all primetime, USA Network and all Olympic sports received the immersive sonic enhancements.

By the Tokyo 2020 Summer Games, immersive sound was also captured and delivered by the permanent Host Broadcaster OBS. Nuno Duarte, the OBS sound designer and supervisor, was dealt a surprise—no audience! And the same for Beijing. The magic of sports and particularly the Olympics is seeing and hearing the arenas and stadiums full of spectators.

The games were delayed a year and everyone had time to prepare a design without spectators. The actual sound signal capture and production of immersive sound is not difficult particularly if you have a sonic layer of ambiance and atmosphere.

Malone had to make some difficult decisions about the sound design.

“Tokyo gave us the ability to focus on the details of the sports and the athletes,” he said. “The sound of the hands on the gymnastics apparatus and the creak of the wood on the parallel bars; the actual physical exertion of the athlete, the breath, the sighs, the joy all there to be captured without the masking by the crowd. We knew we were missing the passion of the crowd as much as the athletes were, but adding any fake crowd was unthinkable, even for a company like NBC, which is largely an entertainment one.”

A Milestone
The 2024 Summer Games in Paris were also a milestone as they were the first summer games where immersive sound was produced with the return of the audiences, the third dimension that was missing in Tokyo.

NBC seamlessly creates a production mix that includes “stems” of the host broadcaster’s 5.1.4 mix, plus any microphone splits, any additional crowd capture, any camera microphones, plus commentators, replays and music. Along with NBC’s branding and personal touch, it makes it look like NBC did the entire production.

“The absence of the crowd in an event of the magnitude of the greatest sporting contest in the world is almost unthinkable, and Paris is a return to what an event of this scope requires for the greatest athletes, the biggest crowds and the loudest cheers,” Malone said.

I anxiously tweaked up my Yamaha, Dolby Atmos-equipped soundbar and paid my subscription to Peacock over my Roku streamer and noticed a significant improvement with the sonic quality of the broadcasts.

The return of the spectator and the abundance of “open-air” stadiums has created a rich gumbo of sound succulence! I applaud Nuno Duarte and Karl Malone and thoroughly enjoyed listening.

The fidelity and quality of live programing has certainly improved over the last couple of decades; however, inconsistencies exist between events and broadcasters.

Through the efforts of many sound designers and mixers, sonically boring sports such as baseball and football have been transformed into consistent aural master pieces. Fox Sports audio mixer Fred Aldous changed the sound of football with the application of wireless microphones on the players, which is only growing in scope and popularity.

My proposal for wireless microphones in the bases at the 1996 Summer Olympics was shut down by the Cuban delegation fearing that the athletes could communicate to the outside world through my base microphone. Enough cannot be said about Joe Carpenter, audio engineer at Fox Sports, and his impact on baseball sound. He has put wireless microphones in all the bases plus on the homeplate umpire for years.

Soccer Coming Up Short
There’s no doubt that the power of persuasion from John Madden and the Fox production staff swayed the powers at the National Football League to recognize that this is what the public and viewers want. Clearly gaming sound, particularly on John Madden’s Football video game opened the ears of the consumer who began to ask, “Why can’t I hear that on TV?” Well, you can.

Wireless microphones on just about everybody now has piqued viewers’ interest and increased viewership for the struggling United Football League (UFL).

I commented in a previous article that I thought sound coverage of the FIFA World Cup had evolved to the best ever. This was basically accomplished with super-fast computing and some very clever algorithms assisting the mix. However, recently I watched/listened to the international production of UEFA Champions League football (soccer) finals between Dortmund and Real Madrid, and to politely state—it sounded very average and boring. I don’t understand­—virtually every sport has upped the game with pictures and sound except soccer.

Basic audio-follow-video was absent, when on multiple occasions I saw handheld cameras with no “close perspective” sound. Basic audio-follow-video is nothing new and was used as far back as 40 years ago where certain microphones, such as microphones on handheld cameras, were controlled by the switching of the camera. Audio-follow-video — basic sound for football/soccer has to include all corner kicks—no exception. Once again, the microphone is only a few feet from the corner and a basic camera switch would never miss a corner kick sound.

We are beyond the basics and once upon a time, sonically deficient audio coverage in sports was acceptable, but not anymore.

Some Immediate Improvements
I have expressed my opinion about AI and super computers completely taking over a live news or sports production. I still have faith in my prophecy but believe there are some immediate improvements that can be achieved with audio and the general production with AI supercomputing.

For example, Lawo’s HOME app platform with its use in the Rugby World Cup, has proven itself able to control the natural field effects. But one level further is available where you can automate the effects portion of the mix with subtle augmentation of field sounds. This powerful and disruptive technology has been developed by the English company SalsaSound and their MixAiR technology. AI can not only execute simple intelligent audio follow, but it can easily improve speech intelligibility through filtering and sampling.

With the 2024 Summer Olympics coming up, I would expect the sound to be significantly better than the Tokyo 2020 Summer Olympics just because the spectators are back. But with immersive sound it is easy to bury the sports effect with too much atmosphere and ambiance. My hope is that they are advancing their microphone coverage using some innovative sound technique like I have outlined in my book “Immersive Sound Production, a Practical Guide.”

High-quality audio production has trickled down from the production of large events like the Olympics, National Football League and Major League Baseball to where quality production of audio and video is common at all events, from bowling to darts. Take a listen to ESPN’s college softball coverage.

Cinematic sound is not hard—just hard work. Grow with the technology. There is an abundance of affordable sound enhancements that, with digital mixing consoles and audio plug-ins, can help dial in and create your sound. Learn. Last year I went to a Mix Magazine-sponsored event in Nashville about immersive sound and there were more than 100 people who were interested in immersive sound for music production as well as broadcast. The next generation of sound designers must diversify their skills, be persistent and be open minded about how to best entertain their audience.

Dennis Baxter has contributed to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering” and “Immersive Sound Production – A Practical Guide on Focal Press.”

He can be reached at dbaxter@dennisbaxtersound.com or at dennisbaxtersound.com

Television operations and production is a team effort where clear communication between all members is critical. Communicating a camera angle, inserting a graphic or counting into a video replay all require clear, uncluttered, undistorted communications between producers, directors, operators and technicians.

When I started working in television/outside broadcast, I used to think the most important job of audio was to get on-air and mix a good sounding show. I learned quickly that intercoms were probably the most important aspect of audio’s job. You can lose a camera, crash a recording machine and even up-cut a commentary microphone, but if you lose the producer’s or director’s microphone you are probably screwed—believe me I know.

Intercom Etiquette
Television communications evolved from readily available telephone technology and even into the 1980s broadcast communication systems were a basic telephone “party line” configuration. The director’s camera conference would be where all the camera operators communicate on a single channel. Even a communications channel was often called a “P.L.,” which is shorthand for private line or party line.

In those days, the CBS OB van had only three basic channels of party-line communications—the director’s channel, the producer’s channel and an engineering channel. Additionally, there was the ever-present director and producer “hot mic,” which was heard throughout the OB van over open speakers so that if you were not on the director’s headset channel everyone could hear the director or producer over their hot microphone.

The 1980s began a period of improvements as the needs of broadcasters became defined and refined. Initial six-channel modular intercom systems were a lot like the analog days of telephones with delays and echoes that inhibited clear communications. 

Equipment manufacturers were quick to adopt the latest telecommunications technology and next came various schemes of analog multiplexing where the systems evolved with increased channel counts along with programing flexibility. As with most things electronic, the final transition to digital solved most of the inherent deficiencies in an analog system.

I admit that there were some advantages to only having a few channels of communications, but the downside is that chaos can quickly ensue when someone gets “wound-up” and starts yelling over people on the same channel. Intercom etiquette not only includes polite interaction, but quick conversation. 

My good friend and fellow mixer/A1 Pete Addams preached the concept of “word economization” when talking on intercom systems. You don’t need a chapter’s worth of information when only a sentence or a word will do. The infamous producer Don Ohlymeyer only needed one channel of communications and had the quietest communications of any live network production I witnessed. It was simple—the general rule was that everybody listened to Don and spoke only when spoken to. I miss Don!

Current communication systems have almost unlimited channels permitting abundant programing possibilities without worrying about limitations on channel capacity. Digital communications systems can easily be programmed for “party lines” or a “point-to-point” individualized channel. In addition to channel flexibility, digital intercoms have become incredibly sophisticated with programming that makes possible such things as splitting the left and right ears between different channels and being able to blend selectable channels together in each ear. 

These communication tweaks are a tremendous benefit in a complex and hectic broadcast production world and personalized communications will endear you to most directors, producers and operators.

The Dark Side
As necessary as intercoms are, there is a dark side to them. There seems to be a shield of anonymity when some people put on a headset. I remember being told by a three-lettered network that I had “thin skin.” The shield/excuse often was called “in the heat of the moment, or it was nothing personal.” Well, some of it I took personal and so did some of my colleagues. What has been excused as “locker-room talk” is not excusable over the headset.

Remember, a headset is a microphone and a speaker and in the broadcast world there are many remote intercom stations that also have built-in speakers—and from my years of experience there was loose chatter that could be considered offensive, rude and down-right hateful. Before the HR and legal departments interceded, outside broadcasting had become wild and wooly, and the intercoms were the undoing of several on-camera talent and a few directors and producers.

Intercoms are often viewed as mundane and a necessary evil, but I contend that intercoms should be viewed as the backbone of a broadcast production and essential. A director will remember you if they had intercom problems before they would ever remember if you had a great mix! 

Sound production is the artistic and technical aspect of broadcasting that is completely subjective. The balance of the sound elements, for example the voices (commentary), as compared to the volume levels of the sound effects and music, is subjective to the point until you cannot hear and understand what is being said. 

As a sound practitioner, I express the tone of a production using equalization to adjust the bass and treble of an audio passage, however my experience guides where I ultimately make a subjective evaluation of the proper level and tone of any adjustments.

Even certain sound phrases and terminology indicates a level of subjectivity. I worked with a British producer at the BBC who said that the mix sounded “wooly.” What is my subjective understanding of wooly sound?

Who Get the Blame?
However, this subjectivity may be related to the reproduction technology or method or perhaps a medical condition such as tinnitus. 

Broadcast sound is subjective because, as the sound mixer, I control the outcomes. But what if I have tinnitus or frequency-specific hearing difficulties? For example if the mixer has lost sensitivity in the higher frequencies then there is a tendency to compensate by adding high frequencies to the mix resulting in a shrill, more brittle sound to the consumer. 

What if the listener has frequency-specific hearing loss? Then possibly the listener may have difficulty with speech intelligibility. Consider if the listener has cheap or improperly installed speakers or just rear-mounted speakers in the TV set? This is clearly beyond the control of the sound mixer, but who gets the blame?

As a newly minted sound mixer, I will never forget a phone call I received from an ESPN executive telling me the mix did not sound right. Perplexed by the call I asked, “What’s wrong?” After some hemming and hawing from the executive, I asked, “Is it the balance or tone or what?” After some silence, I hung up. I am sitting in a 5x8-foot audio control room, trying to mix a live show with the incessant roar of equipment cooling fans behind me and a communications network that is never silent. 

There may be a chance that something doesn’t “sound right.” Well, the executive called back, and I quipped that, “It sounds fine here,” and hung up again. Needless to say, I did not work for ESPN for awhile.

My subjective impression was that it did sound fine where I was, and I am sure that it did not sound right to the executive listening over TV speakers. I never got the opportunity to ask the person what they were listening on, but improper speaker placement and setup was the demise of some fellow audio folks who got burned for bad sound because someone improperly set up the speaker system for Dolby Surround in the network QC room. Who got the blame?

Balance is the most difficult aspect of an audio mix to master. Overly zealous announcers and an unscripted event such as football or baseball can result in the voice being either too loud or too soft. Audio compressors help to smooth out the audio, but to me overly compressed audio is just as hideous as “bouncy” audio or burying the voice in the mix.

Surround and Immersive Sound
Audio is subjective and specific to the listener’s environment, reproduction device and physiological condition. Now factor that by two—the producer of the sound and the consumer of the sound. Sound was pretty easy when it was mono and the television cabinet (housing) was the size of a refrigerator with front-facing speakers. 

Stereo was a little more complex with two speakers and the task of creating a “phantom center” for the voice. Surround sound created more problems with a dedicated center speaker for voice and five other speakers for effects and music. The other problem for surround was proper placement of the center speaker in a world where there really is no place for a center speaker. Technology helped here with the divergence of the voice into the side speakers and the introduction of soundbars.

Immersive sound ushered in a whole new set of issues with still only one center speaker plus an additional four upper speakers to generate more noise and further mask the announcers. What’s an audio person to do?

First, understand your own hearing abilities and do not always trust your ears. I have tinnitus and high-frequency hearing loss, but I do not artificially compensate for the tone of a mix to my liking. Generally, a “flat” mix will cover a lot of reproduction possibilities from poor room acoustics, improperly placed speakers and even cheap speakers.

Second, new mixing techniques can deliver better results. Consider this. The center speaker may not be just for voice. Try spreading out the sound (diverging the sound) into the left, right, left height and right height speaker; not to the point of distracting the listener, but to reinforce the sound like a phantom image. 

This technique works well with immersive sound and soundbars. (See my November column on sound mixing or my book “Immersive Sound Production: A Practical Guide.”)

Finally, at home, after the show, listen to your mix. Also ask for input from your peers and non-audio types. Fred Aldous from Fox Sports used to tell me he always listened to his mother when it came to his mixing. Mama is usually right. 

Immersive sound was almost a causality of the pandemic because in-person broadcast and sports productions were shuttered and virtually all engineering and financial resources were diverted to getting and keeping live content flowing—by remote control. Remote production practices and technology advanced quickly and subsequently benefitted less popular secondary sports and content for smaller targeted audiences with an easier engineering setup and overall cheaper production costs. 

Consequently, immersive sound was pushed to the back burner. However, I believe 2024 will see a revitalization of the dimensional sound format. Why? Because immersive sound production and reproduction technology is available and affordable, however the next task for practitioners is to create the “wow” factor and stoke the consumer attraction.

Consider this: Television is basically two dimensional with the picture parked directly in front of the viewer and significantly, television depends on the sound to give dimensionality to the experience. Since the picture is in front and the screen stretches vertically, why not fill the front sound space with sound from top to bottom? It may not seem intuitive to put more sound above the viewer and it may not be appropriate for all applications, but a sport like basketball certainly benefits from sound above the viewer.

Where to Put the Sound
I believe there still is a reluctance to push the sound envelope (experience), which may be a hangover from uninspiring 5.1 surround sound production and strict loudness compliance; but clearly more atmosphere behind and above the viewer does not add much to the interest of the soundfield. Many sound practitioners approach immersive sound as another horizontal layer over the existing surround, ear-level layer. No doubt this is a limited creative approach to immersive sound. 

Additionally horizontal layers of sound are hard for most soundbars to reproduce so by pushing the sound into the upper front space, the sound design also takes advantage of the mechanical sound reproduction of a soundbar using up-firing speakers. Immersive sound design for soundbars is more about creating a sound experience over reproduction of an accurate soundfield.

Basketball on television is perfect for advanced immersive sound applications because the visual coverage beyond a wide shot of the action on the court is close-up, head-to-toe of the athlete under the baskets. With the basket perceptibly above the athlete there is a clear motivation to put the basket sound above the viewer and create a front vertical soundfield of sports specific sounds. The front soundfield consists of the Left, Right, Left Height and Right Height sound zones acting as a coherent, controllable sound space where sound can be steered anywhere vertically in front of the listener.

Basketball sound is perfect for immersive soundbars and application of the principles of Front Vertical Soundfield Reinforcement (FVSR). What sounds go above the viewer? With film there is “room tone” ambiance as well as specific effects, and increasingly, film sound production uses electronic digital spatial enhancements.

Broadcasters and content streamers of dramatic and serial content produce their sound similarly to film, but with sports, the sonic palate is undeveloped beyond the absolute consumer-expected sounds like the crack of the bat. Imagine head-to-toe sound of the athlete or putting the viewer inside a bobsled at 80 mph.

What About Field Sports?
With basketball the benefits of advanced immersion applications such as FVSR is obvious, but look at the majority of televised sports, particularly field sports such as baseball as well as football and soccer. These sports are heavy ambiance with a center focus on the sport’s specific sounds. 

With advanced immersive sound practices such as trajectory mapping it would be possible to recreate the dimensional movement of the ball in the soundfield—think a moving asteroid! Think AI-rendered soundfields. Or simply by elevating mono and stereo sounds, elements between the ear-level soundfield and the above soundfield can create a more integrated soundfield with clear correlation between the front and front height speakers.

Sound design for immersive sound is about reproducing a dimensional experience, which to me is all about creating and mixing for soundbars. Since the production sound mixer will probably not mix over a soundbar and the vast selection of consumer soundbars are so different in design it is hard to expect a sound practitioner to mix for soundbars. 

However, the sound practitioner should be considering the reproduction side of sound. The sound mixer needs to be familiar with the physics of soundbars and have a willingness to experiment. After a little practice the sonic outcomes for the sound practitioner will quickly improve. Additionally, I have published a practical guide to immersive sound production with case studies and examples to share my experience and ideas and help jump start the immersive sound initiative.

The creative possibility for immersive sound is an open proposition although a challenge because the ATSC 3.0 did not mandate advanced audio standards for immersive sound. Some in our industry don’t really see the need for immersive sound, but its time has arrived, and it is up to the audio practitioners and the entertainment industry to stretch the boundaries and promote this truly innovative entertainment experience. 

The state of audio and video synchronicity in the broadcast world is definitely improving, but deficiencies are still apparent on some broadcasts, particularly sports. 

Audio/video synchronization problems are aggravating and probably inevitable. Frequently many problems experienced in the home are solved by software updates, replacing cables, adjusting the hardware settings in your TV and on your soundbar or by simply unplugging the unit for 30 seconds. 

A Head Scratcher
However, many persistent and seemingly random sync problems that plague broadcasters are a head scratcher. For example, how can various elements within a production such as wireless microphones and cameras be out of sync while other production elements are in sync? I used to think my problems were a Roku (streaming device) problem, but then the network goes to a commercial break or local news and that content seems to mysteriously be in sync. How can an entire program be out of sync while the national and local commercials are in sync?

Although picture and sound in DTV are transmitted together they are distinct streams of data and processed by separate hardware or software paths at the station and in TV receivers. Here lies most of the problems: At-home audio and video enhancements such as video upscaling and soundbar decoding through ancillary speakers require processing time. 

Many issues can be resolved with a power reset, software download and even by putting a wire between your router and TV, eliminating the wireless aspect of Wi-Fi, but there are problems that the viewer cannot resolve, particularly in sports.

Out-of-sync audio and video sync can begin in the OB van or broadcast studio where the audio and video are captured, produced and processed independently of each other. 

With any piece of digital equipment—for instance the mixing console—the microphones have to be digitized and attention must be paid to the input/output stage and with the monitoring. 

Why? Because latency occurs with every digital conversion. High-end digital mixing consoles—such as those from Lawo, Calrec, Wheatstone and others—have resolved any noticeable delay, but with the pressure to reduce engineering costs, small and medium-sized operations have opted for mixing consoles from the PA industry that rely on I/Os that connect through CAT 5/6 along with console plug-ins for compressors and other processing equipment that certainly inject timing issues.

Mismatched conversion or sampling rates will certainly cause latency between digital equipment and can be easily overlooked during installation. At the end of the day, a “valid test” should be conducted to ensure that the audio and video are in sync before leaving the studio or OB van.

A Summer of Frustration
Every time I watch TV I observe A/V sync issues, but I decided to give broadcasters another look last month and was very surprised at what I saw and heard. Nothing demonstrates a sync issue better than the crack of the bat hitting a fastball or the whack of the golfer on a tee shot.

Earlier this season, ESPN’s Sunday Night Baseball caught my attention after I noticed that the first several batters were out of sync. This does not surprise me because the audio team might have to do some tweaking to get the sync dialed in. But after several innings the sync did not improve. 

Months later, ABC/ESPN appeared to have their sync issues under control and their coverage of softball championships with RF microphones on the coaches and umpire seemed to match the picture. ESPN’s WNBA coverage was impressive because the net microphones and coaches’ microphone were spot-on with the commercial breaks. Later that day I observed that ESPN’s SportsCenter was in sync, but once again Sunday Night Baseball was off. How does that happen?

Baseball is a tedious sport to sync audio because of the various graphic sources that all inject some processing latency. I asked Glenn Stilwell, who produces and mixes baseball for Apple TV, what the numerous audio sync pinch points in baseball are.

“Foremost the pitchcast or strikebox,” he responded. “Second is the advertising boards [green screen] behind home plate. Third can be any low depth of field RF cameras that shoot presenters. And then there are wireless mics placed on the home plate umpire. Getting all that to sync with the video can be a challenge.” 

Stilwell further said that “Camera 4 [center field] is the most common video signal that will need a sync adjustment, as it is the one typically used with the pitchcast and advertising signs. Camera 2 [high home] also has a different sync given that it is used to show certain graphics.”

In addition to baseball, golf is particularly difficult to sync because of the abundance of wireless microphones and cameras used. During my listening test I observed that both CBS and CW’s LIV golf were unacceptably out of sync. 

I waited for CBS to show the booth announcers and noticed that they were slightly out of sync, but when they showed the roving announcers, the sync only got worse and the tee shots on both CBS and LIV golf were hideous. I curiously waited for CBS to go to “60 Minutes” and found the sync on “60 Minutes” to be acceptable.

Finally I observed that NBC NASCAR had various degrees of sync issues between the many RF microphones and even the commercials, but when the network switched to local news, the sync issues cleared up. During the day and evening I noticed that one of the four local networks had sync issues on all programming and that the talking heads on CNN, MSNBC and PBS were in sync.

Sync issues are pervasive and elusive and can occur in production, transmission, distribution, and finally, in the home. With production, anything wireless comes with timing problems. Additionally, when moving audio and video through the network, frame synchronizers and encoders need processing time to accomplish their tasks. Bit rate reduction, better known as “compression,” is always prone to timing issues.

I know the technology exists to initiate proper sync, and there are various “time stamping” schemes that can be used to insure and maintain sync. I know that sync issues have been addressed in these pages; however what I do not know is why sync is still a pervasive problem, particularly with sports. I would appreciate hearing from my engineering and video friends who can explain to this non-techie audio guy what’s going on.

Audio is a unique discipline (practice) in the M&E world because sound operators and technicians are required across all aspects of production and distribution. Sound practitioners and technicians manage and mix PA systems, TV shows, work on sound stages and labor with the soundtracks of the recording studios, plus more. 

Even though all sound applications have their unique workplace requirements, the basics of capture and reproduce have more in common than not. I was reminded of this when I recently attended Mix Nashville: Immersive Music Production, hosted by my friends at TV Tech sister brand Mix magazine and Future.

Learning the Ropes
I came into television in 1983 when ESPN was still in diapers and the three commercial networks—ABC, CBS and NBC—were entirely unionized. Nonetheless there seemed to be opportunities in television for a fledgling musician and studio owner in Atlanta. I found out quickly that the “fly by the seat of your pants,” no turning back, no “fix it in post” attitude that was pervasive in live broadcasting suited me better than a recording studio. As a bonus I found that the live broadcast world was full of people like me who had worked in music production or were drawn to sound from a love of music.

My introduction to sound for outside broadcast came with ESPN and their first sound supervisor Dennis Finney. Finney was gruff and seemed unapproachable until you got him talking about being on the road with the R&B bands on the “Chitlin' Circuit.” He drove the truck, set up the PA and mixed the show. 

He did the same for ESPN—drove the truck, set up the broadcast audio, mixed the show and closed the bar. Finney also designed the audio system at ESPN and put into service the magnificent Neve 5000 Series mixing consoles in every ESPN outside broadcast truck. He knew about Neve mixing consoles from his music days.

Early on, ESPN's Donna Shackett, who was responsible for crew management, gave me a big break, not because I knew much about TV, not because I was particularly skilled, but because ESPN needed warm bodies and was reasonably willing to train you. Thank you.

No doubt I was in over my head but was a quick study and got along with the crew. After some humbling experiences with ESPN I was introduced to television entrepreneur Frederic (Fred) Reinstein who ventured into motor sports with his Atlanta-based company World Sports Enterprises. He realized quickly that I was green, but wanted some consistency with his crew and was willing to train me. Reinstein introduced me to Klaus Landsberg who was mixing sitcoms and sports but cut his sound chops doing music recordings with Wally Heider and his remote music recording truck. 

Landsberg taught me a lot about listening and how to strike an air of confidence with directors and producers, preparing me for Reinstein and a long series of difficult directors and producers. I quickly became Reinstein’s sound mixer and personal intercom technician and when he called I would put everything down, including a live mix and tend to his needs. 

Reinstein also gave me a direct connection to CBS and their sound supervisor Bobby Seiderman. Once again to my delight Seiderman and a few of his close technicians were also doing music recordings around New York and welcomed me into the fold.

Parallel Comparisons
There certainly are many parallel comparisons between live TV, live PA systems and recording studios. Live is live and there is no turning back or retakes and your mistakes can be as obvious as a dead skunk in the middle of the road. You must learn to think, react and recover quickly. Sports and live music are similar in that there are no two venues and no two bands that are the same. 

Consider this: The PA is feeding back and you have 50 open microphones. Which microphone is howling and how do you tame the beast? Or what if the mixing console goes down during the middle of a show? What is your backup plan?

The recording studio is less about immediacy and more about listening skills, but the studio experience exposes a sound practitioner to a different toolbox. I brought keyboard samplers into broadcasting shortly after a studio musician introduced me to samplers and keyboards in 1986. Klaus Landsberg showed me how to use gates and compressors and about getting a good tone with beat-up microphones.

Now is the best time to get into broadcast audio because there are more opportunities and it is easier to master the technology and work across multiple media platforms. Plus, I believe that the entire entertainment industry benefits from the cross pollination of different personalities, skills and ideas.

The broadcast engineers and technicians do not expect you to know everything at an entry level but do expect the crew to get along and work as a team. 

Success Secret: Find someone who is willing to train you. The skill set and technology is vastly different and easier now than when I started. I don’t want to say that outside broadcast technology is “plug and play,” but clearly “digital everything” made installation, maintenance and troubleshooting far less complicated.

The technology to create high-quality versatile sound for any media is immediately available, but the basics of setup, programing and delivery is still a requirement that will necessitate audio practitioners and technicians. 

Listening is what a sound person does—from listening to the sound sources to ensure clarity and listening, to the sound mix for balance and interest, to listening to the director and producer who control the flow and operation of the team. Listen up! 

This year we have heard the terms "artificial intelligence," "machine learning" and "deep fakes" tossed around frequently, particularly since Wall Street forecasters have discovered AI as a financial evaluation metric for a tech company’s future performance. Companies that are known to support broadcasting and media applications such as Microsoft (ChatGPT), Adobe, IBM (Watson), NVIDIA and Soundhound AI, all promote possibilities with the continued development of this technology.

After some research, I am still trying to understand what seems to be an evolving definition of AI, but I have no doubt that smart/fast computers will significantly impact broadcasters and media companies.

AI seems to encompass a bundle of operations like machine learning and computer-created deep fakes, dependent upon analyzing vast amounts of data to predict, create and deliver a desirable outcome. Content companies such as Netflix have benefitted from AI with computer-generated programing recommendations based on accumulated information of a person’s search and viewing habits.

Music, Hollywood Scripts
Computer jocks have used AI to develop complex algorithms to create art, music, even dialog for movie and TV scripts, (music, for instance, is pretty repetitive—you can only imagine how many hit songs are based on three chords/notes and a rhyming dictionary). I also think that there is no reason not to believe that ChatGPT could write a Hollywood script. Remember, the last Hollywood writers strike gave us “unscripted” reality TV—often a waste of electronic transmission time and electricity.

Since machine learning is data-driven, it is dependent on the accumulation of more and more data samples to improve the results. Constant sampling only improves the outcome and has been particularly effective with the AI sub-field of “deep fakes.” We think of deep fakes as replicating a person or object’s face or voice onto another, but deep fakes are as old as TV itself—background sound effects such as laugh and clap tracks, even coconuts used for horse hooves were intended to imitate reality and fool the gullible listener with the magic of radio and television.

Clearly machine learning has applications in broadcasting. 

There is no question that HBS—Host Broadcast Services, the production company for World Cup Football—used fast computers to make possible accurate microphone selection and mixing possible. Lawo worked with HBS to develop a mixing system that takes data of the ball position and translates that into an algorithm that captures the best possible sound from the best microphone or combination of microphones, plus determines the level to mix and blend the microphones together. Tracking the ball is done optically and in a sport like football, the focus of the game is the ball; basically, you tell the computer to follow the ball. 

Is this artificial intelligence? I would say more “deep learning.” Another sophisticated automated mixing algorithm, “Spatial Automated Live Sports Audio” or SALSA, was developed by my friends Rob Oldfield and Ben Shirley.

“In our case we are primarily using deep learning to automatically recognize sound events in broadcast microphones so we can automate and enhance the on-field mixing,” Oldfield said. “We are increasing what we are doing with AI beyond this though, and are now looking at crowd and commentary sentiment analysis so we can generate metadata for other parts of the broadcast chain, for example, automatic highlight generation. 

“It occurs to me that there is a lot of data from the microphones at an event (not just sound capture),” he added. “I think this has been overlooked in the past, but with the increased power of deep learning and AI we are in a better place to fully use the microphones as ‘data gatherers’ and audio can add value to all parts of the broadcast and fan experience.”

AI-Assisted Audio
Now let’s follow the flow for typical sport coverage and look at the advanced possibilities of AI—machine learning for sports coverage. Camera robotics has been around for a while and there is no reason the cameras and audio cannot follow the electronic commands of a computer that is following play action.

AI comes into play when a computer analyzes the switching patterns and compares the director’s commands to the position of the ball within the field of view of the broadcast cameras. The computer archives the director’s selection and patterns for future progressive learning and within a short period of time, repetitions will be detected, examined and programmed into event cycles to take over the direction of the cameras and audio. 

AI-assisted audio coverage/production could include speech interpretation and synthesis such as AI-driven subtitles, but could AI mimic a play-by-play or color commentator? 

Consider this: The computer can learn styles from “real life” commentators and further learn how to filter information from the cameras to match the visual action and build a reference library of players or actors. The “speech” computer can ingest all the data and artificially create the commentary track and even mimic certain styles and accents.

Speech synthesis has been around awhile and with the addition of faster computers and machine learning it becomes conceivable that you can create droid commentators that interpret/present the play-by-play action and side stories to complete the entire experience.

Sound analysis has been common practice, however artificial intelligence would be good at evaluating patterns and picking the best choice(s) for a replay from a set of indicators. For example, a very loud, sudden burst of crowd (sharp attack) with a long sustain is probably a good indication of a goal. 

The vocal inflections of the crowd—sustained screaming as opposed to a sigh of dismay that dies out quickly is another valuable and identifiable metric. From these simple learning indicators, the computer within a dozen repetitions will be able to accurately predict a good highlight moment. 

Clearly media production will benefit from computer augmentation and machine learning simply because of the amount of content that needs to be generated. Media entertainment cuts across a variety of viewing and listening options including small or large screens, even goggles with anything from immersive sound to earbuds and an adaptive learning algorithm such as AI can only contribute to the elevated experience.

It all impacts TV technology, particularly since we work in a deeply computerized world. Where do you fit in this brave new world.

Immersive sound for broadcasting is all about spatial orientation, sonic separation and hoping that the listener can reproduce some dimensional sound through speakers or headphones. Clearly the sound mixer can’t control the sound ecosystem or how the consumer listens to content, but it seems that immersive sound reproduction is a moving target and certainly creating a compelling sound mix that may well be heard over speakers, soundbars and headphones is no easy task.

Horizontal Layers
The problems often begin with overly conservative sound design, but as sound practitioners we must start somewhere and build forward. Typically, sound design looks at audio as horizontal layers—for example, immersive sound proposes an ear-level horizontal layer of sound and a stratus of sound above the head of the listener. NHK, the national broadcaster in Japan, has even recommended that there should be sound projected from below ear level.

When immersive sound was initially contemplated and tested it was reasoned that these above sounds should be captured and placed into the upper sound zones and reproduced by the height speakers. Even basic sound design principles would suggest that the sound mix should be what you would hear as if you were there. Horizontal layers of sound are fine for basic immersive sound production and academic consideration, but if the “real” sounds above the spectator’s head are diffused atmosphere and excessive PA noise, then not much of these sounds are particularly constructive to an interesting immersive soundfield/mix.

Further I would argue that too much atmosphere in front of the listener is boring, fatiguing and impacts speech intelligibility and is one reason why the immersive sound mixer should consider alternative mixing designs. For example, most soundbars project the sound from in front of the listener and I believe that this should be a major consideration for the sound designer/mixer. 

Since most immersive soundbars project frontward, upward and sideways then the question is—are up-firing soundbars projecting more atmosphere in front of the listener, thus diluting the mix? This could be the case if the front left and right height channel are used primarily for ambiance and atmosphere. 

Basketball, For Example
Consider that if the front height channels are projecting sounds from the front and upwards, then it makes sense to use the front height channels to reinforce the screen action, dialogue and ancillary audio. Front Soundfield Reinforcement (FSR) is a concept that I have promoted since a presentation I made at Germany’s Tonmeister conference in 2016. 

FSR basically supports using the complete front vertical soundfield—left and right channels along with the left and right height channels to mix a front “soundframe” around the visual presentation. For example, basketball has a clear top frame with the net and hoop and bottom frame with the floor. By placing some sounds in the vertical perspective this design helps to elevate a 2D image into a 3D soundspace.

Testing of basic immersive sound production began in 2014 with NHK in Japan and later in 2016 with NBC testing various microphone schemes at the Olympics and American football. In 2016, Seoul Broadcasting System adopted ATSC 3.0 and used the MPEG-H encoder to transmit the 2018 World Cup into the homes of Korea.

 The 2018 World Cup was probably the first major sporting event that made available an immersive sound mix with the addition of an overhead atmosphere layer constructed from Schoeps microphones. Felix Krückels was the sound designer and worked with Helmut Wittek on an immersive array, the ORTF 3D, which was described as a “double-layered ORTF using eight microphone capsules.”

Capturing immersive sound with ambisonic or 3D array-type microphones creates a stable dimensional foundation, but is not necessary to create an immersive experience, particularly with sports sound. 

For example, according to NBC Sports’ Karl Malone, director, sound design, NBC has created immersive sound at Notre Dame football games for several seasons and uses a variety of mono and stereo microphones to create immersive sound. Simple immersive sound production does not require 3D microphones or 3D mixing busses, however virtually all mixing consoles used by the broadcaster—Lawo, Calrec and SSL—have 3D panners, which easily allow for spotting/placing the sound elements around the intended listener. 

What 3D sound does need is more adventurous sound designs and a good understanding of encoding and how decoding/rendering affects the sound design. In my basketball example I suggest a pair of lavaliere microphones to capture the backboard and net swish and to route these sounds as audio elements/objects to be rendered into the front left and right height channels. 

An experienced sound practitioner might ask, “What happens to these above sounds in a stereo or surround mix?” Interoperability between formats has been a problem, but digital sound, metadata and rendering solved some of the conversion issues between formats. Dolby Atmos, Auro 3D, DTS:X and Fraunhofer MPEG-H are examples of encoders for distribution of immersive sound; and when the audio reproduction device detects a particular codex it will renderer the proper balance and spatialization appropriate for the reproduction device and the number of transducers. For example, with a Dolby Atmos enabled up-firing soundbar it should project the net sounds above the listener.

The wide range of reproduction devices does not guarantee similar sound experience, but I would argue that just about anything is better than rear-mounted television speakers.

Hope is not a strategy; successful immersive sound is in the mix. 

I began experimenting, testing and writing about immersive audio in early 2012 and remember hearing the term “Next Generation Audio” (NGA) sometime thereafter. Since then, a variety of groups have advocated for a variety of features and formats for the next-generation content producers and equipment manufacturers to build on. 

Currently NGA describes such features as immersion, interaction and intelligent interface between playback devices and reproduction devices, but in the United States there was no government mandate to keep forward momentum with the implementation of those features. To add to the confusion, not only was there no mandate, but there were also competing and incompatible formats with Dolby Atmos and MPEG-H.

Neither Dolby nor Fraunhofer invented immersive sound or NGA, which is actually a direct result of technology advances and clever compression schemes that achieved greater capacity and quality for producing advanced audio. Immersive sound is possible because broadcasters can get the minimum of 10 discrete channels of sound—5.1.4 to the consumer within a narrowly defined data bandwidth. The audio bandwidth can be allocation for audio beds, channels or objects to enhance the immersive experience or for interactive features.

Is it Worth It?
Immersive sound may not be the ultimate entertainment experience for every consumer—research shows a steady rise in stereo and 5.1 surround soundbars but with only a very modest increase in 7.1 and the “other category,” which would include some variation of immersive sound. This leads me to believe that perhaps immersive sound production, particularly for sports, is just not compelling enough to spend extra money on higher-quality speakers.

Immersive sound has been the most fostered focus of NGA, perhaps because it was the most developed feature with the ATSC 3.0 rollout. But the illusion of cinematic sound seems delusional with the proliferation of “faux immersive” DSP processing and soundbars that make all kinds of claims of immersive sound reproduction. I ask again: How can you get an immersive experience from an up and side firing soundbar? Maybe the consumer is savvier than first thought.

The broadcast world has understood the craving of user-controlled sound since consumers have been able to shape their home sound with the inclusion of the bass and treble controls. Dolby was quick to introduce dialog controls, but nothing as interactive as alternative dialog channels that have been suggested with NGA.

Dolby ushered in personalization with the introduction of Dialog Normalization, which was obviously a good idea for different listening environments and content, but Fraunhofer’s MPEG-H offers the possibilities of true user interactivity. For example, with over-the-air broadcasting the coaches and player language can be problematic, but an isolated and controllable coaches’ channel eliminates the prescreening and sanitizing that take place in a live broadcast. Significantly for the CFO’s, a coaches’ channel could be monetized. Note, Dolby claims the same level of interactivity through using object channels.

All this interactivity can be ultimately limited by the producers and rights holders with options for improved dialog, alternate narration and even select objects such as radios and wireless microphones. The ability to control any one of multiple players’ or coaches’ microphones and listen in mono or stereo—that seems like a pretty immersive experience to me.

Rendering the Content
The final and probably most under-valued benefit of NGA is the ability to render the audio content to virtually any consumer device or format. For example, consider that the audio elements are embedded in the digital stream and can be combined—rendered to a proper recipe for a mono, stereo, surround or immersive sound mix. 

The early shortcomings of surround sound were the downmix and metadata. Also remember that using the set-top box for combining surround channels to derive a stereo mix has always been problematic for accurate and equivalent sound productions because many of the sound elements are baked into the mix. Rendering takes all the ingredients and makes a whole new cake. 

Rendering is the final process before reaching the consumer and in theory, rendering can take different transducer characteristics, configurations and containers and optimize a soundscape for any listening device. However, I am still mystified as to how you can get an accurate representation of an immersive landscape from side and up-firing soundbars—although it really does sound better. 

A lot has been said about the potential of Next Generation Audio. While the technology is evolving, content with compelling immersive sound and interactivity is lagging, except in gaming. Just as surround sound started conservatively, immersive sound production has started conservatively as well. 

One significant difference is that early soundbars were a big improvement for surround sound, but I am not convinced that the immersive sound experience has benefitted as much from soundbars as surround sound did.

The immersive experience is different for us all, and Next Generation Audio provides a framework, but no roadmap. I have recently published a book with Focal Press titled “Immersive Sound Production—A Practical Guide” that presents an advanced approach to live immersive sound design with a concentration on live sports and includes more than 60 different sports case studies. 

Trying to get beyond all the hype you read about immersive sound, I spent several months preparing to listen to the 2022 Winter Olympics in Beijing and Super Bowl LVI in immersive in my home in Decatur Ala. I confirmed that NBC Sports produced 5.1.4 Channel Immersive Sound, but it is still difficult to get immersive sound into the home.

In an ideal digital world, you would be able to hook up your digital antenna to an ATSC 3.0 tuner and receive a UHD video signal and immersive Dolby Atmos sound from your regional terrestrial television provider. I am about 35 miles from the antenna and have good terrestrial reception, but Huntsville Ala. is not an ATSC 3.0 market yet. There are nearly 50 markets in the United States that transmit ATSC 3.0 with Dolby Atmos as the immersive audio standard but no luck with my trusty digital antenna.

My Roku Ultra streaming device said it is capable of Dolby Atmos so I searched out satellite, cable and streaming services and continued my quest for immersive sound. NBC delivered 4K UHD with Dolby Atmos sound for the 2022 Winter Olympics coverage to customers through distribution partners including Comcast, Verizon, fuboTV, YouTubeTV and Prime. Comcast and Verizon did not offer service in my area and Direct was only interested in selling me their biggest package and no one could tell me if the Dish Hopper delivers immersive sound. With a $69 a month contract minimum, satellite or cable was out of the question for this experiment.

‘Sophisticated Handshaking’
While I searched for a service provider I spent half a year’s earnings from my TV Tech writing commission and bought a high-end Yamaha Soundbar with Dolby Atmos and 49 speakers firing in all directions. I hooked up my soundbar directly to my Roku Ultra and hooked the 4K output from the soundbar to my Sony 4K television. 

After 10 minutes of fiddling with menus and remotes I was able to get sound through the system. Although the sound was better than my Vizio surround sound soundbar, I was not convinced that I had immersive sound. Initially I was suspect of the HDMI interfaces and contacted Neal Roberts, senior manager, Broadcast Partnerships at Dolby Laboratories (Neal has since moved on to HBO Max).

Neal told me that regardless of the version of HDMI, consumers will always have audio and with later versions supporting more capabilities.

“The connection is controlled by the sophisticated handshaking between source and sync devices that HDMI supports,” Neal said. “This ranges from stereo at minimum, to 5.1 channels of decoded PCM audio, to an encoded DD+ bitstream that includes Dolby Atmos, to decoded audio and metadata [known as “MAT”]. For example, a new set-top box connected to a legacy soundbar/AVR might produce a stereo or a 5.1 channel surround sound experience, while that same HDMI signal connected to a newer soundbar or AVR would be capable of reproducing a Dolby Atmos experience.”

So it seems that it is possible to deliver immersive sound to the consumer, but I question the depth of immersive experience since it was difficult to hear much sound above me. 

The Right Sound for the Right Sports
Now is the time to consider the consumers and how-to best create an effective soundfield over soundbars. Up-firing soundbars need something to up-fire. Birds and aircraft belong in the height channel, but I question how much ambiance and atmosphere should be directed into the height channel. Clearly the most popular American sport—football—does not offer any sports sound that logically should be in the height channels except the PA and atmosphere.

However, there is a large reservoir of activities, such as many winter sports, that benefit from an aggressive vertical enhancement. For example, with ski jumping, simply by placing the last microphones on the jump in the height channels it gives a clear aural impression of lifting off. Elevating sounds is a significant aspect of immersive sound and sports struggles with what sounds should be elevated into the height channels. My upcoming book shares more than 50 case studies on immersive sound production for sports.

I think the 2022 Winter Olympics proved that the infrastructure for producing and delivering immersive sound is working itself out, but significantly I think audio producers must consider advanced immersive sound production practices and create an immersive sound experience beyond atmospheric embellishments.

In 2016, I presented the concept of “Front Soundfield Reinforcement” which coalesces the front vertical soundfield into a cohesive aural pallet. Contemporary immersive sound production is typically horizontal layers with a surround layer at ear level and another height layer with usually four channels of sound. If you contemplate vertical layers where the front left, right and the height channels collectively reinforce the front soundfield, then the concept of Front Soundfield Reinforcement seems like a logical solution for soundbar reproduction.

Basketball clearly demonstrates the benefits of Front Soundfield Reinforcement. Basketball uses a camera under each basket which visually has significant vertical dimension. By placing microphones by the net and on the floor and routing the net microphones into the height channels, the illusion of being under the basket is complete because the net sounds are heard above the listener.

When I reflect on the state of immersive sound production I accept that perhaps atmospheric embellishments are enough for the average sports consumer, but I think an enhanced and entertaining experience is necessary to engage an audience that seems to have lost its appetite for TV sports. Listen up. 

I will never forget the day I brought an Avid ProTools system into our studio and my partner remarked that “there was no way a computer could be faster than the old-fashioned razor blade edit.” There were two computers in the studio: one for accounting and the other was a crude device that controlled the capstan motor on the 24-track tape machine to synchronize it to a video machine and timecode.

This was the early 1980s; there were no computers in the OB vans and every single piece of equipment was analog. Videotape editing was machine-to-machine with an operator—the video going through a switcher and audio going through the mixing desk. Music was played off of the NAB Cart, a magnetic-tape sound-recording format. I guess you could call the first “computer” I remember in an OB van was the DigiCart instant playback system from hard drives.

After several decades of computerization and the implementation of IP throughout broadcast ecosystems, innovation has put us in a place where everything is computerized and we are already seeing the concept of computers controlling computers. Computers controlling computers is nothing new, but machine learning is, and to me this is a haunting reminder of Kubrick’s “2001: A Space Odyssey.”

SOUND AS AN INDICATOR

Artificial intelligence (AI) has been used in sports for awhile. At Wimbledon, for example, the computer listens and watches the tennis match and identifies exciting indicators by applying a variety of metrics. The metrics guide the computer in learning how to recognize significant points of interest and for what makes a good highlight or replay.

Interestingly, sound is a leading and reliable indicator. For example, pandemonium in the crowd after a long quiet pause is a good indicator of a memorable highlight moment. One of my logic metrics would also include the duration of the crowd burst as well as the amplitude, threshold, attack and sustain of the sounds during the interesting moment.

Additionally, the voice inflection of the crowd—sustained screaming as opposed to a sigh of dismay that dies out quickly is another valuable and identifiable metric. From these simple learning indicators the computer within a dozen repetitions or even 100 times will be able to accurately predict a good highlight moment.

I would argue that in 2018 we were close to something with AI. Lawo had developed a mixing system that takes data of the ball position (or any interesting follow target) and translates that into capturing the best possible sound from the optimal microphone or combination of microphones, plus determines what level to mix and blend them together. Tracking the ball is done optically and in a sport like football, the focus of the game is the ball—basically you tell the computer to follow the ball.

Undeniably World Cup 2018 was the best-sounding football event I have ever heard. Praise goes to HBS Christian Gobbel, Felix Kruckles and the Lawo team for implementing a true paradigm shift in the world of sound for broadcasting, but I think Philipp Lawo is on to something else.

THE SALSA ALGORITHM

An alternative and interesting method to advance automation is "Spatial Automated Live Sports Audio," which uses existing shotgun microphones around the pitch to detect the ball kicks. The system not only looks for overall level intensity, but also the envelope across a range of frequency bands for each sound event type that a sound mixer might want to capture. The SALSA algorithm is capable of detecting ball kicks that are virtually inaudible on the microphone feeds and is more reliable at recognizing sound events than our ears.

During live production, SALSA uses one of two approaches: It can automate a mixing console’s faders to capture each on-pitch sound event, or use the frequency/envelope information of the ball kick to trigger pre-produced samples. These sounds can be added to the on-pitch sounds or can replace the game sounds if you want it to sound like an EA Sports Game or a Saturday afternoon match on SKY. It is up to you as the sound designer and consumer.

Now, let’s take a look into another possibility of AI for sports coverage. Artificial intelligence comes into play when a computer analyzes the switching patterns of a sample of directing styles and compares the director’s commands to the position of the ball within the field of view of the broadcast cameras. The computer archives the director’s selection for future learning.

Within a short period of time, patterns will be detected, examined and programmed into event cycles to take over the direction of the cameras. A basic “follow-the-ball” pattern is learned, however it would seem possible that you can modify the production by blending and altering production styles. I once worked with a director that had a rhythm and repetition to his cutting style and literally repeated a dozen or so patterns over the course of a three-hour game.

I can clearly envision the day when bots and droid-computers capture, direct and produce live sporting events with little human intervention. Let’s follow the flow; camera robotics support systems have been around for awhile and there is no reason the cameras and audio cannot follow the electronic commands of a computer that is following play action.

Imagine this possible scenario: The computer is calculating that, after a goal kick, seven out of 10 directors would cut to a wide shot while optical position tracking is continually sending the directoid mapping data of the field of play. The “directoid” directs camera X, Y and Z to follow the ball while simultaneously directing camera A and B to track the coaches.

Additionally, cameras A and B are capturing the audio from the coaches and sending the information to the directoid, which is learning the patterns of the coaches and when to cut to the coach. The directoid has a library of possibilities for each ball position and makes comparisons.

Real-time action coverage could include speech interpretation played out from a computer that has ingested all the data and artificially created the commentary track. Speech synthesis has existed for awhile and once you have optical tracking it becomes conceivable that you can create droid commentators that interpret the play-by-play action and sound resynthesis to complete the entire experience—alternative reality.

My vision of the future paints a different picture of the science, art and practice of audio as I/we know it, but I believe my speculation could become reality.

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production. He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com.

I have often said that I believed the broadcast industry is usually fair and judges you by your performance on every single show. But in the course of this story, I had candid conversations with some tenured and respected folks in television and scratched some old wounds.

I am from Atlanta, and Ted Turner brought much more than Braves baseball to America. No doubt, baseball and all Atlanta sports provided a lot of content for a fledgling television operation, but Turner Sports needed television operators and technicians to run the equipment.

Turner Sports found, developed and nurtured an army of hometown and regional people that provided a first-class crew in a world where a decade earlier there were few jobs available for camera, audio and other technical operators. Before the AOL debacle, Turner Sports was an operation that rivaled anything the networks had.

DIVERSITY AT TURNER SPORTS

I think of that group of people at Turner and I only remember seeing a family of different races and gender. That does not mean they all sang in perfect harmony, but when you put a diverse group of people together for more than 250 events a year there usually is an exchange of ideas and experiences that changes people. I still try to understand why racial prejudice and gender bias keeps hanging on, but I would like to say I think broadcast folks are beyond it. I hope!

Turner Sports and ESPN changed the system and by the mid-1980s the change from staff television technicians and practitioners to freelance was well underway and by the early ’90s most sports practitioners and technicians were freelance—wow, what a paradigm shift.

I think in the early part of the transition some of the hostilities between staff, former staff and the new cadre of freelancers was the fact that many of the staffers were worried about their jobs and having to compete in a new freelance world.

However, I would like to suggest a few more reasons why I think broadcasting does not have a more diverse workforce. I think attitude, accountability, education and lack of role models explain many of the obstacles for entry-level difficulty and long-term broadcast careers.

When I joined the ranks of remote operation it was obvious that outside broadcasts and live sports could be intimidating. The OB domain was worlds away from the eyes of executive management in Los Angeles and New York, and attitudes and the lack of accountability was pervasive.

Remember, before ESPN and Turner there were only three commercial networks—ABC, CBS and NBC. ABC had a history of arrogance and lack of accountability that is depicted by several books, but it was making money, had great ratings and was a leader in sports programming for decades. So if you wanted to work in sports, you essentially had to suck it up. It was generally laughed off, always stated that it was “nothing personal” and that you had to have thick skin to work in outside broadcast, (see my column “It’s Open Mic Night” in the October 2020 issue).

Certainly every day use of electronic communications known in the industry as “THE PL,” something that holds the entire crew hostage for hours during a live show, contributes to the insensitivity. You are talked to, talked at and talked around by many people you don’t see. I think this tends to bring out the worst in some people who might think the electronics desensitizes some of the hurtful language.

At the end of the show there usually is a lot of backslapping and “see you down the road” kind of stuff. But consider the fact that the director, producer and other senior level technicians control your employment—particularly in the early emerging freelance world. Most of the time you just sucked it up and moved on. I do remember sensing some of the “what are you doing here” attitude, loosely implying that someone was not smart enough or maybe that women are not technically adept. Smart enough? Audio folks like to call themselves “audio engineers.” Well I have only met a few real audio engineers in 40 years, but I know a lot of talented audio practitioners.

Additionally, I do not believe that an engineering degree is necessary for most of the jobs in outside broadcasting. If you are working in audio you need to know some basic electronics, have the stamina for setting up equipment and have the aptitude for logical thinking and troubleshooting. Audio practitioners now are not required to “fix it.” There are real engineers like my friend Pat Dixon, a Turner veteran and woman who has earned her stripes and respect to handle that.

ON-THE-JOB TRAINING

I think the education world is doing a major disservice to the broadcast world by pimping an expensive four-year degree to get an entry-level position. There is a private school that has an elaborate OB van that does side-by-side educational training to teach students how to mix in a world where those students will not see the mixing console, much less mix an entry-level show for many years.

I question, is it worth it? On-the-job training (OJT) goes a long way in qualifying people to move forward in the industry and is an important place to find role models and mentors. NEP has led the way with internships and OJT resulting in full-time employment for many people. NEP will start you at the bottom, but if you are willing to work, they are willing to teach.

While emphasis has been placed on on-camera diversity, if you can’t see behind the scenes there is little to draw new people into the industry. I think the lack of role models has contributed to a lack of diversity.

I worked for the International Olympic Committee for 27 years. The greatest thing my former boss Manolo Romero did was create the Host Broadcast Training Program to educate, inspire and fill the pipeline for future quality broadcast people. Romero made his entire staff available to teach at several universities to foster a spirit of inclusiveness.

My first teaching experience began at the 1996 Olympics in my hometown of Atlanta at Clark Atlanta and Spellman University. I worked hard to give about 250 students enough basics so they would be useful, but also be an ambassador from the Host Broadcaster to assure everyone would be treated fairly in the selection process. There was a lot of competition for these positions and it didn’t take long for most of them to realize that their first job out of school would be at the Olympics and get paid! I still hear from some former students who have worked professionally in the business for over two decades.

I think that broadcasting is a fair industry where you are judged on your performance at every show—as you should be. I also think the technical side of the industry is largely diverse. But I do think that people can be insensitive and make disparaging remarks, out loud or under their breath and on open headsets. Yesterday, as well as today, this should be considered harassment or a hostile workplace.

Whether said in jest or not thinking that anybody can hear, or saying “it’s just locker room talk,” it’s still harassment and doesn’t foster inclusiveness. There usually is an open microphone somewhere. 

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production. He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com.

Last summer I started splitting residences between North Georgia and North Alabama and decided that I would “cut the cord” in Decatur, Ala., because my internet connection was at least a 1,000% improvement over my DSL-barely usable AT&T internet in Gainesville, Ga. As all of you know DSL is over copper phone lines and after 8,000 feet of poorly maintained copper wire, the data stream in Georgia was slow.

I set up shop in Decatur, ordered my coax internet service and bought my first streaming device. I checked my internet speed, plugged my new streaming device into my Sony 4K television and then tuned into a 4K streaming channel. To my surprise the picture and sound was out of sync.

As a sound guy I am particularly annoyed by lip sync, which is not hard to find anywhere you look these days. We experience sound lagging behind visuals in the real world because the speed of light is significantly faster than the speed of sound. Many knowledgeable engineers would say that lip sync issues started with the digital age, but I first experienced a lip sync issue in the analog world.

TRAVELING TWO PATHS

Often audio sync issues begin when audio and video are traveling down two different paths. I first encountered out-of-sync audio and pictures in the early days of wireless cameras in the NASCAR pits. The video went one way; the audio went another way and after a couple of frame syncs on the cameras there was clearly a sync issue.

In the good old days of analog, audio and video issues were obvious and usually easily fixed because the audio was often ahead of the picture and quality-sounding analog audio delays were relatively cheap, plentiful, sounded good and were in the OB van—plus CRT screens were an excellent reference for lip sync.

There is no doubt that digital audio solved and caused many audio problems. Video is compressed to reduce the data per frame, which maintains a unique sequence of pictures. Audio is a continuous signal and is compressed separately from the video. The next time I began to see sync issues were with mismatched sampling rates. In every single piece of digital equipment and editing software there are menus for sampling rates and operation for sampling rate conversions. If there is a mismatch, expect a sync issue.

There are a host of reasons and places where the audio and the video can get out of sync, but ultimately it becomes the problem of the broadcaster or steaming service. As broadcast professionals we must keep an eye on sync because in the broadcasting world there is digital processing that happens all along the workflow, from remote operations to transmission, which can result in a cumulative delay effect. We can only make sure that the audio and video is in sync when it goes to the transmitter. But let’s face it; so much video is consumed through an internet connection and I can safely say that will not change.

Data compression and corrupt data-related problems are prevalent on the internet and this significantly affects the quality of our sacred broadcast product—content. Digital over-the-air television seems to have fewer audio problems, but I am guessing that the few broadcast engineers that are left out there are taking care of large network operations plants and antennas. I have often thought that there has been a huge influx of people with a computer mindset where problem solving has devolved to a hard shutdown and reboot.

STREAMING MEANS BUFFERING

Cutting the cord has been a learning experience where I quickly found out that streaming means buffering. With my DirectTV in Georgia I did not encounter buffering issues, which either stops the stream or decreases the quality of the audio and video to compensate for a reduced data stream. In Alabama I have measured the data stream and my internet averages between 80 and 100 Mbps, which adequately covers 4K video for most of my viewing pleasures—but I can still have buffering delays. And buffering can complicate sync problems. It’s a processing thing.

As I have discovered, I have all the specs, but still encounter buffering. On the consumer side there can be problems with streaming services, internet connection, browsers or audio/video device drivers and interconnect cables; or all of the above can cause audio and video sync problems. Chasing down these problems can be incredibly daunting for the average consumer and taking the chance that you would call customer service and get someone on the line to tell them you have a glitch, the pat answer is unplug, wait a minute and reboot.

Another issue that is particularly annoying is the occasional repeat, missing word or upcut word that occurs when watching TV. I reached out to a friend and TV Tech contributor Jim DeFilippis, who said that some of the bumps and jumps are due to buffers in the transmission path.

If the transmission is error-free and there is no lost or damaged data, everything is fine but often the data is corrupted and lost. If there is a video loss the decoder jumps a frame or repeats a frame of video, but the audio can’t jump ahead so sometimes audio gets repeated or words get cut out or upcut.

If the audio data is then damaged over time, the audio and video drift apart and, ideally, when the A/V delay reaches objectionable tolerances the decoder should reset the buffer and start over. If for some reason the buffer does not reset then lip sync will persist until a reset.

The ATSC 3.0 specifies that the audio and video are embedded together and this is where I think the old saying fits: “It is fine leaving here” applies. As professional broadcast engineers we already know that there are a lot of things that can go wrong with audio sync.

Audio delay is everywhere and audio sync must start from the beginning to the end. Broadcast engineers understand tolerances of acceptance but many of them have retired. Perhaps they should come back and teach a couple of things to the computer jocks.

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production. He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com.

Who would have thought that we would end 2020 with accelerated entertainment and sports schedules, athletes, judges, contestants and TV crews wearing masks—along with virtually no audiences or spectators in venues? Here’s my assessment of what the past year has meant to our industry: 

THE GOOD

I really don’t mind empty seats with close shots of the athletes to disguise the fact there are few fans, but I do mind goofy cutouts that cover empty seats. My favorite shot of the baseball season was from the 2020 MLB World Series that Joe Carpenter mixed for Fox Sports. Not only did Carpenter have the biggest parabolic microphone I have ever seen behind home plate, but he had a total of three parabolic microphones: not just directly from behind home plate, but also one on the left and one on the right of home plate. Carpenter not only obstructed the prime viewing area of fans that were not there, but he delivered an engaging sound for the TV audience.

This was the season when audio proved to the many video skeptics the true importance of sound. It’s not that audio is 10% of the show and 90% of the problems; and it is certainly not that audio just follows video. When the sound of the fans and spectators are gone it seems the show really suffers.

As a retired road mixer, I fully understand that home time for many weary broadcast engineers, technicians, camera operators and the audio crew is often not enough. As my wife told me over 30 years ago, “You have it made. You don’t cook, clean or participate in the family.”

What an eye opener—I am still married to a very patient woman.

THE REALLY GOOD

It also seems that many audio professionals have spent some time expanding their audio chops. I have heard from and read about many “audio-ettes” who are spending some education time getting Audinate/Dante Certification and expanding their intercom and mixing desk skills.

More home and education time is a huge benefit to the crews, but for the last four to ve months, I have listened to baseball, basketball, hockey and football and truly believe that it sounds better with fewer people in the venues and sampled sounds.

One of the first times I tried synchronous playback was of pad crunches when I mixed a University of Georgia football game. How can you compete against close to 100,000 exuberant (mostly drunk) fans? The games I have listened to this season with perhaps up to 10,000 fans are dynamic, detailed and interesting for the home listener because you can hear the field along with a nice balance of the fans.

Clearly, the MLB World Series and most basketball coverage took the sound to a different level with alternative microphone placement and practices. For example, using the contact microphone under the floor could certainly lead to interesting Audio Tech Emmy nominations for the 2020 season.

THE BAD

It is hard to understand the lifestyle of the thousands of broadcast technicians in our industry. For the lucky number who can work out of a major sports city, a sport like baseball means 82 home games, which means employment for a large hometown crew (30 to 40 people) for 82 days.

With COVID-19, much of that income was gone for a shortened season. A regular home season for baseball and basketball is a luxury, but there are thousands more who travel to cover sports on a contract for weeks and months at a time, committed to cover college and professional football, basketball, golf, tennis, NASCAR and much more.

I heard from some of my freelance friends who experienced COVID-19 cancellations after being booked before the season had even started or was subsequently postponed, and I was told that many of the major sports networks paid freelancers for missed work—for awhile. The broadcast crew needs to have live sports competitions in order to earn a living. 

With COVID-19, much of that income was gone, but hopefully, only for a period of time.

THE NEW REALITY

What do drones, cameras and microphones have in common? They can all be controlled remotely. Is that bad or another significant paradigm shift? The pandemic accelerated a process that had already begun—remote production. I honestly am surprised that it had not happened sooner. For example, the latency gap is something that Dave Maza of NBC Sports has worked at since before the 2000 Sydney Olympics. Clearly, this season of sports broadcast has proven that remote operation works.

Since the beginning of the pandemic, executive management has seen fewer people in the control rooms and on-site in general, providing ample proof and months of experience that crewing and staffing can be changed. Workow and operator interfaces have been tested and proven and there has been improvement and refinement in all areas of remote operation.

Additionally, there is a broad interpretation of what the term “broadcast quality” has come to be defined along with an abundance of less expensive digital equipment that helps achieve amazing quality for marginal costs. I am astonished at what I have seen on the major networks and often surprised at the quality of many streaming productions. Quality production is only getting better and cheaper.

There is obviously a symbiotic relationship between all the skills that contribute to an Emmy Award-winning production, but job skills and content production have shifted as capture and production have become simplified and decentralized. From the directors, producers, commentators, camera operators, sound mixer, audio assist, stats, spotters and on and on, I advise you to keep one simple thing in mind as you look ahead to 2021... it’s all about entertainment.

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production.

He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com.

These are interesting times for live sports broadcast production. Just as many people prefer to hear laughter from a real audience while watching a sitcom, many sports fans favor a noisy, live crowd over the “fake” crowd noise in the stands during a professional game. Most writers and web events have used sensationalist journalism terms like “fake” to create a very negative description. Using the term “fake sound” has nefarious overtones and I believe sound manipulation for malicious interference has ethical implications. But sound supplementation for the sake of enhancing the entertainment value has always been fair game to me.

SPORTS SOUND SUPPLEMENTATION

Sports sound supplementation is the art of live foley, which can include crowd, sport and venue-specific sound.

Real-time sound supplementation was controversial long before I began using a crowd loop in the early 1980s at NASCAR races. There were always rumors of “birdie” loops at golf events, but there was not much of an urgency to “sweeten” golf sound because major networks had the budget to adequately capture the event live. I spent a decade as a freelancer covering IndyCar racing and was unhappy that the network crew (who only covered one race, by the way) had microphones mounted on the pit wall at every pit zone while I depended on the sound from two handheld cameras to cover up to 30 pit areas. Why? It costs extra money for extra sound.

Motorsports present unique sound challenges because of the excessive noise and compact design of the track and grandstands. For me, sound enhancement began at Bristol Motor Speedway where the sound was so intense your head would hurt. Frustrated with the inability to pull much crowd reaction sound, I built several crowd “loops” to enhance the sound of the start, restart and finish of the race. I knew the real-time sounds were there, but the laws of physics prevailed. It is called “masking.”

Tape loops of sound effects were the shortcomings of sound supplementation because of the inevitable repetition of sounds, but when the music industry introduced electronic samplers that could store real sound and could be played back with a keyboard, everything changed. With instantaneous triggering and access to multiple samples at the same time, the possibility of realistically playing back a complete pit stop from any pit without additional microphones became a reality. What was really cool was adding sounds that were really there but that you never heard before because of masking—such as the air jacks lifting the car and two gear shifts out of the pit.

I am not trying to persuade anyone of anything. I am only trying to enhance the entertainment value of the content. The sound of air wrenches and engines revving is really happening. I even had different samples of motor sounds for different engine manufacturers.

Let’s examine some of the arguments:

Q: Are we documenting an event vs. presenting a game for the sake of entertainment?

A: If the intent is to show the negative impact of the COVID-19 virus on sporting events, then quiet empty venues may be appropriate. But if the goal is entertainment for the listening/viewing audience, then sound supplementation is an applicable solution.

Q: Does sound supplementation enhance the entertainment experience or is the sound a distraction?

A: To my ears, poor sound supplementation is a distraction. I spent from July to October 2020 listening to Premier League Football, MLB, NBA, NHL, NASCAR and WNBA, produced by ESPN. Some sound production is a distraction. For example, my opinion of the sound enhancements for Premier League Football was a half-hearted attempt at sound enhancement with virtually no dynamic articulation.

At the other extreme was baseball, which worked with Sony Entertainment in San Diego to deliver not only a very believable, but entertaining soundscape for the game.

I talked with Kurt Kellenberger, head sound designer and supervisor at Sony in San Diego. Kellenberger is a meticulous and innovative sound designer, and when he undertook the design for a baseball game for Sony, Kurt studied the broadcasters and picked the best aspects of the game as the baseline for his game sound design. A decade later he provided the sound effects components for the primetime presentation for the MLB’s COVID-19 comeback from San Diego. Sony provided approximately 70 different sound samples and Abelton Live software to construct a 4.0 sports sound bed for the game. What I heard was not only convincing, but if you closed your eyes, the soundtrack was close to a perfect presentation of the sound—not too much, but interestingly complete.

THERE’S A DIFFERENCE

Faking the sound as opposed to enhancing the sound for entertainment purposes are completely different sound design concepts. There are technical and financial reasons that make it impossible or impractical to capture the desired sounds in a venue or at an event. Capturing the pit sound was a financial decision and only the largest IndyCar event of the year warranted the additional cost. Even though the budget cuts were made clear to the director and producer, they clearly remembered the sound of the Indianapolis 500 and the experience of hearing the sound from every single pit.

To amplify, enhance and even recreate the sound that already exists is obviously different than creating a soundtrack from scratch, which is what a sound designer does with a film or game soundtrack. For example, when the roof camera shoots a pit stop there is a great distance between the camera’s microphone and the sound. Even though the sound of the pit stop still exists, the microphones on the roof camera cannot capture the sound. This scenario results in no pit-specific sound even though the listener knows and expects to hear the pit sounds from a distant camera shot.

Baseball deals with more subtle sound plus the concept of home and away teams. Not every team or player is popular and not all the crowd reaction is positive, but what if artificial sound is used for reprehensible reasons such as real-time mood manipulation? As you can imagine, a relentless crowd booing or heckling could result in increasing tension on the field.

Kellenberger also reminded me of the writings of the cultural theorist, Dr. Jean Baudrillard, who wrote about how media affects our perception of reality. Baudrillard believed that people often live in the realm of hyper-reality as they are connected deeper and deeper to their television, movies, games and virtual reality. Personally, I suspect that numbness sets in and listeners tune out.

Sound supplementation is nothing new, particularly in certain sports. For example, the Swiss and Fins have used samplers for years to cover ski sounds for downhill and cross-country events. Samplers not only fill in the gaps when there are very long camera lenses and few microphones, but when well-executed, the additional sound brings the viewing listener closer into the sport and event.

FAKING OR SHAPING?

I narrated a BBC radio documentary that was produced by Peregrine Andrews, titled “The Sound of Sports.” One comment I made that became sensationalized was about using samplers to augment the sound at the Olympics. Somehow that comment got twisted to imply that I was using samplers to fake the sound of most sports at the Olympics. This interpretation of sound supplementation escalated in 2012 when I was accused of “faking” the sound of the London Olympics and was even spoofed by late night comedian Stephen Colbert, about the possibility of extremes. You can find it on the web.

Entertainment vs. documentation. These are also interesting times for live sound designers, mixers and producers. Spectatorless stadiums are a first and there have been novel ways to compensate for the visuals with goofy looking cutouts. But usually in sports the sound is supposed to support the picture and what if you are documenting an empty venue? What should the sound be? Clearly, spectatorless venues are a different paradigm that broadcasters and listeners were expecting.

But I have to ask this question: Are we faking the sound or shaping the sound? The sound samples used in baseball are pristine, probably better than real capture. Is this fake sound? I think some journalists and wordmongers who write about audio and know little about sound use these types of words to stoke emotions on one side or the other on the use of extra sound. I was accused once of cheating, but argued that if I did not deliver a high level of entertainment to my listeners, the only person I was cheating was the listener.

Ed Stoltis (A1 CBS Golf) commented that listeners will adapt to a new norm, particularly when it is a pleasant and appropriate soundtrack. I think the sound of CBS golf is wonderful with little crowds and no ice machines. ESPN’s production of the WNBA was appropriate for the picture. The space was compressed with no space for spectators and the soundtrack was the coach, players and commentators. The production was engaging and entertaining. I really like quarter-full college stadiums and always thought too many drunk fans ruined the TV sound of college football.

ESPN was able to come up with a clever presentation, but large venues used by sports like baseball, football and professional basketball have noticeably vacant space and seating for spectators who are obviously not there. Directors will adjust with tighter camera perspectives, but the venues are obviously empty. Who is faking whom?

Perhaps after the stadiums and venues fill back up and the broadcast world has time to ponder the sound extremes, sports sound designers and mixers may have an open mind about massaging the sound with a little sound supplementation. At the end of the day, it’s all about entertainment.

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production. He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com.

I have never been comfortable talking into a microphone. I first noticed how awkward and funny I sounded when I was in the fourth grade in Madison, Ala., and was learning about girls, chit-chat and telephone etiquette over a rural telephone party line. Not only was the telephone in the kitchen, where I believed everyone on earth including my father was listening; but also the neighbors could listen in because there were three or four houses on the same telephone line as ours. Net result? A paranoid introduction to sound at 10 years old.

Later in my professional life, one of the first things I learned about a television set, announce booth or sports field of play was that there are microphones everywhere and some of those microphones may be open—and often they were. There are boom microphones, field microphones, headset microphones, even a “snoop” microphone, so the accountant could listen in and hear if we were working. Needless to say, there were no private conversations on a television set or in an OB van with so many headsets with open microphones lying around everywhere.

MOUTHY COMMENTATORS

In August, a story about a Cincinnati sports commentary made headlines because the announcer made an insensitive remark, which was accidentally streamed out live to listeners. The announcer awkwardly apologized, but ultimately left the announce booth. As in the “The Wizard of Oz,” the microphone had become the insular shield that precluded direct contact between the adoring peasants and the wizard—but ultimately, the bubble burst.

Mouthy commentators getting caught is nothing new. Early in my career, the live sports networks began using C-band satellite frequencies for transmission between the remote sites and network master control, (transmission was continuous from the location even when the network went to commercial break). Broadcast folklore has stories of drunk and misbehaving fans caught on camera accompanied by imaginative commentary being transmitted through cyberspace over satellite dishes. Not only was this unscripted broadcast picked up by master control, it was also heard by many shocked sports fans and snoops from the FCC. This content would not have met FCC censorship rules.

After phone calls to politicians, the FCC and numerous complaints to network management, live television engineers devised an audio-to-air bypass called a “bird-beater,” which took the announcers out of the audio transmission path and put their voices in the headsets of the director and producer who could strategize about the live production off-air.

At the time, the broadcast industry was flying high with little oversight by too many people with too many Emmys and big egos. But in the mid-1980s, the fiefdom came crashing down when freelance operators and technicians replaced loyal and spoiled network employees. Before Disney owned ESPN and ABC, colorful commentators, foul-mouthed directors and producers were common.

As freelancers, we even had a phrase for these types of people, “The Untouchables.” It always seemed (as it does sometimes today) that these folks were immune to the fallout from an open microphone. The untouchables were rampant in the late ’70s and ’80s until someone pushed the button—the record button.

In the early ’80s, I began working with a belligerent producer who, over the PLs (the communications systems), was extremely mouthy to freelancers. After a few years of listening to the B.S., I began to record the intercoms. First it was for a laugh and a cry and usually was recorded with nothing more than a Sony mini cassette recorder capturing an open microphone.

But after the producer called a college intern an unacceptable word, I went to upper management to complain on behalf of the crew. Of course the producer did not remember things exactly as I did, but the tape was damning evidence. However, the tape (recording) not only did significant damage to the producer’s career, but to my career as well. Essentially, the network had to do something and dismissed the producer from live television production for a year or so, but I was flat-out blacklisted.

NOT SACRED ANYMORE

The television intercom communication system is a closed private network and was often considered sacred and immune from any implications of harassment, but an open microphone and a recording has been the demise of not only a few commentators, but also television directors and producers and politicians long before Watergate and Nixon. I would think that if you are going to put on a headset and talk over a microphone, it is obvious that there is a good chance that a wide range of people with different sensitivities could be listening in.

Forget sensitivities, what about conspiracies? Consider this, if I make a recording of actual conversations or comments and alter the content and context of the recordings for fun or possibly for other nefarious reasons, it opens Pandora’s box. Even with my primitive sampling technology in the ’80s, I was able to edit words and phrases enough to construct a conversation, completely out of context, between a producer who was not on site yet and a gullible sports commentator who was on headsets.

A lot of the behavior I witnessed seemed to occur simply because people could get away with it, including my sampler escapade with the sports announcer, although I got an earful from the late, great Fred Rheinstein when he found out about my little prank. I still have Fred’s voice loaded up on my vintage Akai S900 Sampler and chuckle every time I play one of his witty comments. 

It is not necessarily illegal to record someone without their permission and I imagine most seasoned broadcast technicians can remember an incident where two or three people did not remember things the same way. But what about the morality of the recording? When there is little to no accountability the results are predictable. Remember the recording of a presidential candidate boasting about grabbing a person’s private parts? Beware, there may be an open microphone somewhere and someone may be listening and recording you!

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering.” He is currently working on another book about immersive sound practices and production. He can be reached at dbaxter@dennisbaxtersound.com.

I have often pondered the fact that live television is technically easier to set up and get to air today because of digital technology and fiber optics. I could not imagine undertaking the sound of a golf event using what is known as “dry pair,” which was commonly used for audio connectivity just a couple of decades ago.

Forget about the noise and constant maintenance, dry pair is unshielded, thin-gauge copper telephone wire and was very prone to crosstalk between the pairs. If you were lucky and had enough noise on the circuit, you may never notice the crosstalk—I didn’t and got burned, badly.

In the early 1980s, I came out of a recording studio and was stunned at how television audio was really noisy and low fidelity. Further, I was dismayed by some of the practices used in remote audio production, such as using dry pair for show sound and intercoms. After the “shock and awe” of learning that sound fidelity was secondary to the director and producer headsets, I quickly realized the importance of intercoms under some difficult circumstances.

Back then, in the early days of live remote broadcasting, the telephone company was usually hired to provide “clean” dry-pair circuits with minimum noise and crosstalk specifications. Over time, the audio crews learned dry pair by trial and error. Dry pair at major event facilities such as at The Masters in Augusta, Ga., and the Daytona 500 track were usually permanently installed and professionally maintained by the telephone company.

PUSHING THE ENVELOPE

Problems seemed to arise as the TV crews started to push the envelope with large events such as golf and car racing with a system designed for low-fidelity telephone communications. A couple of years into my career, I was audio mixer for a car race taking place over a weekend at a large track in Alabama where the commentator booth was a mile away from the outside broadcast van. The network tried to save some money and the audio crew had to set up the dry-pair infrastructure.

After setup and a full Engineering FAX, I determined to turn my attention on getting good sound. I did not detect any problems for a couple of days because the cars were always running and the noise masked the problem. But on Sunday morning, when there were no cars running, you could hear the producer talking to the announcers in my mix speakers. Unfortunately, the producer channel was crosstalking into the commentator microphones’ circuit and going over the air. The next four hours of television was the most painful experience of my professional life. Needless to say I never worked for that network again.

AUDIO AND INTERCOMS

Audio routing, recording and transmission along with intercom systems have gotten bigger, manageably complex as well as more flexible. Clearly with the digitization of all things audio and intercoms, the performance standards have radically improved and with presets and resets, everything has gotten a bit easier to set up and prepare for a live transmission.

For decades I have debated the issue of whether or not communications should be lumped into the world of audio. Intercoms are the backbone of any studio facility and OB van, and clear communications and intercoms are critical to your success and the success of the production. Accordingly, communications will be a major concern to the director, producer and the engineers working on the production, and it seems that everything with a microphone and a headset is the audio supervisor/A1’s responsibility.

There is a good and logical reason for this: The A1/Audio Supervisor is unique because the role is equal parts engineering and production. The production team and network expect the A1 to make the show sound good. Sounding good is often qualified subjectively by production, but significantly the A1 is often the bridge between production and engineering.

Remember, most directors, producers and broadcast production types are not technically savvy, but they can sure tell you who they need to talk to and who needs to listen to them. Fortunately, the engineer-in-charge can help with intercoms and can make adjustments and changes in real time without distracting the audio mixer during the production.

Yes, for the A1/mixer, the engineering setup has gotten physically and technically easier to accomplish, sometimes as easy as pushing a button, but one aspect of live TV remains a challenge. The audio mixers seem to be listening to more and more noise and less sound. Many people in the audio community, including me, believe that it is hard enough to mix a live event, much less have to listen to a director, producer and all the other chaos.

Yes, it is easier and faster to set up all the equipment, but I have been told that it is still just as difficult—if not more so—to mix a show because of ear fatigue from poor monitoring facilities and excessive intercom usage.

LISTENING FATIGUE

I remember working with a director whom I swear got paid by the word—so much so that my friend and fellow A1, Pete Addams, coined the phrase “word economization,” which translates into not giving too much information for the situation and slowing down the delivery of the message. Remember the mixer is trying to listen to the mix and make dynamic adjustments while filtering through all these distractions.

Every top tier audio mixer I talk to brings up ear and listening fatigue—fatigue not only from listening to audio that is too loud, but also listening to too much. Even for a seasoned mixer who practices good volume management, it is easy to let the sound pressure level creep up as the director and producer get excited and louder. Next the announcers are talking louder and the mixer is adjusting speaker levels everywhere. Soon your head is spinning as you are straining to hear everything—hence “listening fatigue.”

As the live television business goes back to work, I consider an industry that has always depended on headset communications between the operators, technicians and the producers. I recently thought about headphone etiquette after I heard a news story about how people speaking loudly facilitates the spread of the coronavirus. I have worked with people who thought that the louder they talked, the better I could hear and understand them. The old analog intercom systems had a compressor so that the louder someone yelled, the softer the yell became in volume resulting in not hearing the yeller as well. Loud, compressed, distorted intercoms equals fatigue. Additional contributors to noise fatigue are poor speaker selection and placement along with little acoustical isolation and treatment in the OB van.

I have often said, you can mix the best sounding show of your career, but if the producer’s headset is mucking up, what is the producer going to remember about your show? Intercoms have been the demise of more than one audio practitioner—including yours truly.

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering.” He can be reached at dbaxter@dennisbaxtersound.com.

With all the utilitarian advances in digital mixing console technology, there is no doubt that the digital mixing desk should be the audio control center in the modern television studio

and OB van. In the long run, a versatile mixing desk saves everyone money because today’s digital mixing desk is capable of the sound management of the entire studio and OB van—from I/Os for routers, recorders, transmission points, pre-hears, to speaker feeds and headset feeds—essentially all things audio will probably pass through the mixing desk.

The digital mixing desk consolidates the processing and routing of audio signals and with dense channel audio formats like MADI and AES67, along with fewer and cheaper connectors, the digital audio mixing console saves time in the installation (copper and connector costs + labor) and saves money in the operation of an OB van simply because of the reduction in weight from less permanent cable and connectivity.

Soft patching through the mixing desk facilitates an easier and faster setup for the operators and fewer interruptions for the sound supervisor/sound mixer. From a retired OB sound mixer, for me an unheralded aspect of a digital mixing console is that only the control surface needs to be in the audio mix room. This not only saves space in the audio room, but the noisy CPU fans and other electronics can be located elsewhere in the OB van and away from the ears of the mixer.

As studio complexes expand and remote broadcasts swell, the need for equipment to be able interconnect and exchange digital audio and data has never been greater. A significant development for digital equipment and particularly the digital mixing console was a protocol that established a standard language for audio transport work over IP. The standard

is codified under AES67 and insures and simplifies interconnectivity between different manufacturers of equipment.

IP interoperability has been a major effort of Philipp Lawo through one of his brands Ravenna, who along with AES, Dante, Q-Sys, Livewire+ and WheatNet-Ip from Wheatstone share in the 2020 Technical and Engineering Emmy Award for the development of synchronized multichannel uncompressed audio transport over IP networks.

Lawo and Wheatstone digital mixing desks manage I/Os through an IP network, which has become a common platform for resource sharing when operations have to scale up and down or change over to another show within minutes. There is no doubt that high-performance IP interoperability will affect all engineering technologies in television.

TECHNOLOGY SHIFT

The audio and broadcast industry is witnessing a technology and operational shift with the next generation of digital mixing consoles coming online. When first-gen digital consoles rolled out they somewhat mimicked the analog architecture, flow and format that digital was replacing. Eight-buss architecture was never an issue until the broadcast format went to immersive sound.

The interest in and production of immersive sound had been gaining traction during the time that console manufacturers were slowly working on next-gen digital mixing consoles, even while some immersive sound productions were underway. In the June 2019 issue of TV Technology (“Hearing It, Believing It”) I wrote about NHRA’s immersive sound production using concentric 5.1 mixing buses on an eight-buss mixing console. There are a lot of mixing consoles with this architecture because eight-buss architecture has been the standard for several decades. But now it seems the stars have aligned and the major console manufacturers have made true immersive sound a reality.

Localization using the concentric 5.1 method of immersive sound production has proven adequate for some sports and entertainment productions, but precise localization is only possible with three-dimensional panning. A significant creative aspect of advanced mixing console technology is that the live mixers can spatially pan 3D audio in the horizontal and finally vertical axis, also known as XYZ panning, something Christian Struck, senior product manager at Lawo already offers.

“Lawo can offer XYZ Panning on up to 128 channel strips, supporting various channel-based mixing formats including 5.1.4 and 22.2,” Struck said.

USE OF PLUG-INS

Precise localization is the basic foundation of immersive spatialization; however, beyond panning is advanced processing, which can be applied to audio signals within the console through plug-ins, a specialized code for an audio task or processor that can function in the mixing console’s operating system. Plug-ins are usually developed by third-party developers and are both inexpensive and beneficial to the audio industry. These programs have been useful for up-processing, spatialization and other sonic applications that are not native to the mixing desk such as volume, routing, panorama, equalization and dynamics.

Plug-ins save time and money. For example, at the Grammy Awards, the artist comes into the audio OB van and gives John Harris and Jay Vicari a USB stick with the plug-in settings of virtually every single channel and instrument. This is obviously a significant time saver, and with instant recall the turnaround time for musical acts is little more than the time for a commercial.

Sports has seen an increase in the use of plug-ins because often production music for bumpers, features, billboards and breaks comes to the venue, OB van or studio in stereo and needs to be up-processed to surround and immersive formats. I have seen this done more often on a laptop that feeds the mixing console although there is no reason this application cannot be done in the mixing console.

I have been amazed at the little use of plug-ins by broadcast sports mixers, but my most recent survey of live mixers showed that in the last two years there has been an modest increase in the use of plug-ins—particularly as the age demographics tick down with new and younger mixers.

I also think that the up-and-coming generation of audio mixers have better listening facilities in the OB van and studio than ever before and are motivated to work on the nuances and sonic quality of their sound.

Finally, sound management of multiple audio formats and speaker systems is necessary for a successful dimensional production. You have to be able to hear what you are mixing across all the channel and speaker formats you are supporting.

The live audio world needs a mixing desk that is affordable, has a modest footprint, versatile control surface and lots of capacity to tame the demanding world of immersive sound. Oh, and did I say easy to operate? I think we are getting there.

Decades ago, I had the opportunity to mix The Talladega 500 for CBS on a 32-channel Ward Beck console with a 12-channel vertically racked side mixer for tape machine playback. Many years later I asked my friend Fred Aldous, long-time mixer, sound designer and

Emmy winner for Fox Sports, how many channels of audio he averaged on a NASCAR race. He shook his head and said “Hundreds.”

It just makes my analog head hurt thinking about it.

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy awards and is the author of “A Practical Guide to Television Sound Engineering,” published in English and Chinese, and is currently working on a book about immersive sound practices and production.

He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com. 

There are sounds associated with sports—like the crack of the bat and the swish of the net—that are expected by the casual television listener. Unfortunately, in the last decade, the crack of the bat became more difficult to capture after the parabolic microphone behind home plate was displaced by the fan seating that got even closer to the field of play.

New venue designs, changes on the field of play and challenging venue and team management have always forced clever audio practitioners to adapt different technologies, ingenious packaging and some audio adventurism to capture that Emmy-winning sound.

MADE FOR TV SOUND

Good sound requires a major effort to capture and an unwillingness to accept any compromise. After Truist Park (formerly Suntrust Park), home of the Atlanta Braves, changed the seating behind home plate that ousted the parabolic microphone, then Turner Sports Senior Audio Specialist Jim Budka convinced management to let him build “houses” to permanently place the parabolic microphones on both sides of home plate where there was more room. Typically, there was only one parabolic microphone, but Jim adventurously added another parabolic microphone. Now there are two microphones symmetrically on the left and right of home plate. The additional microphone resulted in a clear left-right spatial imaging to his mix.

Television is the best seat in the house at any sporting event. How can you beat different camera angles, replays and the sound? It is capturing and delivering unique event-specific sounds that bring the listener into the sport with complete immersion and belief.

There are a couple of givens in sports audio—you have to have a big crowd sound, and you have to hear and understand the announcers. After that, the sound is the imagination and drive of the audio team.

Film and television are unique because while the listener has some expectations, he/she is definitely willing to be entertained. You can hear the crack of the bat at the ball field, but the swish of the net? Just like the cup sound from the golf green, it is made-for-TV sound.

THE SWEET SPOT

There is no one that has pushed the sonic boundary of baseball more than Joe Carpenter, senior audio mixer and sound designer at Fox Sports. Joe and his team have put microphones in the bases, around the outfield, on the yellow poles and as close to the players as permitted.

Home plate is the “sweet spot” of action. During the 2019 MLB World Series, I noticed that the sound was more open and natural—almost like you were standing next to the batter. Looking around the field, I found that in addition to any microphones from the sides that were aimed at home plate, Joe had shotgun microphones suspended in the protective netting aimed at home plate. This simply was the best coverage of home plate that I have ever heard! (By the way, Joe’s sound design and microphone plan would easily translate into an impressive immersive sound mix.)

Placement of the microphones has always been problematic because a faction of directors and producers do not want to see the microphones, but premier events like the World Series accommodate advance and aggressive microphone plans and pave the way for better sound than from the weekly shows.

Setup time is always limited and there is little time to test something new, which validates learning from the big events like the World Series and from Joe Carpenter. You should build your sound design on the experience of those that came before you and adopt that knowledge for your situation.

The sound of sports has significantly improved over the last decade with more models and types of microphones than ever. Shotgun, lapel and wireless microphones have long been the primary tools for sports sound capture, but nonconventional transducers like the boundary and contact microphone have been useful in capturing close and interesting sounds.

The contact microphone attaches directly to a surface and captures the sonic resonances in the surface.

A few examples of audio adventurism: I first used contact microphones in the 1996 Summer Olympics in Atlanta on the wooden Velodrome track, in 2004 on the balance beam in gymnastics, and later in the ice for figure skating in 2006. My friend, Ron Scalese, used them on the skateboard and bicycle ramps at the X Games and even worked on improving the design from a cheap off-the-shelf transducer. Clearly, the best sound from the 2018 Olympics was the sound of figure skating, where a contact microphone was frozen in the ice of each corner creating a spectacular presence in the sound as the skater approached the cameras.

ATHLETE, COACH PERSPECTIVE

Sound design from the perspective of the athlete would include microphones in boxing corners, on players and coaches, under benches and in dugouts. But the ultimate is microphones on the competitors.

Sound designers for sports have long wanted microphones on the athletes and competitors and have made significant progress over the years. NASCAR, NFL and NBA have adopted protocol for these microphones and the fan appeal is high. Additionally, microphones on the athletes will capture closer field of play sounds like with the NFL, but wireless microphones for sports have always been controversial because of colorful language and the chance of the competition cheating in real time.

Sports professionals should consider TV friendly sound—why not? It is television that made these athletes and coaches multi-millionaires. ATSC 3.0 (aka NextGen TV) provides for interactive channels which provide income opportunities. For example, coaches from the NBA are often seen and rarely heard, but a real-time coach channel offers income opportunity to the broadcaster and the coach and skirts any language issues. Cheating can be curtailed with effort, encryption and enforcement.

AMBIANCE AND ATMOSPHERE

Sports venues tend to have poor acoustics and excessively loud PA systems, making it more challenging to capture atmosphere. Ambiance and atmosphere sounds are the foundation and glue to a sports mix. Efforts to place microphones off-axis of PA speakers are recommended. Additionally, plan on multiple microphone locations because there will be zones of activity like the band and student section or home and visitor areas, which should sonically overlay creating dimension and the perception of depth.

To create a convincing dimensional ambiance, multiple pairs of stereo microphones like the Audio Technica AT4025 and/or 1st Order ambisonic microphones like the Sennheiser Ambio should be placed at the zones of interest and off-axis of the PA as much as possible. Ambisonic microphones offer the advantage of plug-and-play dimensional sound than can be spatialized as desired in the OB van.

Immersive sound is in development and the big question is: What sounds do you put above the listener/viewer? More crowd? It is typical in baseball to hang microphones from the announcer booth over the crowd capturing a stereo perspective. Joe Carpenter added crowd microphones to the outfield and suspended microphones from under the seating decks. Perhaps creating zones of interest is a solution. (Personally, I would like to hear the vendors in baseball above me shouting, “Cold Beer!”)

Every event must have a fresh approach to sound design. Immersive crowd sound for sports will have to be more than just progressively distant microphones from the sound source, plus this effect can be created with reverb and acoustic room simulators.

Finally, capturing quality sound is difficult because of the polluters. Excessive PA, ice machines, generators, blimps, and drones, are roadblocks to perfecting the sound of the sport. When I tune into sporting events, I am shocked at how much extraneous and obnoxious sound there really is. My first broadcast golf tournament was “The Masters” on CBS and I learned that the “birdie” microphone was nowhere close to the clutter that surrounds the support of the golf course.

One thing that is obnoxious and unnecessary is excessive PA levels and chatter. In my decades of live television, I have fought the PA. But I still hear from all my colleagues that the volume of the PA is excessive, further making the sound mixers’ job more difficult. There has to be a balance between venue entertainment and quality broadcast which begins with a good relationship between venue management and the venue PA operators.

At the end of the day, good sound requires a major effort to capture the sound and an unwillingness to accept any compromise—then add the spirit of audio adventurism. My advice? Think creatively about the sound effects that you want to achieve and how that enhanced sound will entertain your listeners.

As we come to the end of another decade in broadcasting I think this period of time was as dynamic and influential as any for the world of audio. Not so much with new or better technology, but with a convergence of the technologies to make possible advances in multichannel audio production, delivery and consumption a reality.

THE OLYMPIC EFFECT

The pace of adopting the next generation of sound is on schedule. In 1996 we heard the full implementation of stereo sound at the Summer Olympics and in 2008 we saw 2K and heard 5.1-surround sound from Beijing. At the Summer Olympics in 2020 there will be the full rollout of 4K and 9.1 immersive sound showing a clear trend where every 12 years the broadcast industry uses the Olympics to introduce or prove technologies. There is no doubt that immersive sound is quickly finding its way to the consumer through streaming and from progressive networks around the world such as NHK in Japan, Korea’s KBS and NBC in the United States, who understand the future of broadcasting.

This past year I looked into various immersive sound applications and technologies and will continue this exploration into dimensional sound in 2020 when world broadcasters roll out the Olympics in full 4K HDR and immersive sound.

NBC is planning to mix immersive sound from a control room in Tokyo and will feature content from high viewership events such as beach volleyball and opening and closing ceremonies. Karl Malone, Audio and audio systems engineering designer for NBC Sports and Olympics said that NBC has benefitted from its experience at the 2016 and 2018 Olympics Games plus the 2019 Notre Dame football season where immersive sound was produced. I think Karl has a plan and NBC will commit the resources for a successful immersive sound production. I’ll make sure to cover it.

The future for dimensional and interactive audio is bright, but recently I read something that disturbed me: A sound manager for a large 2020 sporting event was complaining about inadequate resources to produce immersive sound. Resources or not, the requirement for immersive sound is on the table just as the requirement for surround sound was on the table in 2008—even with woefully inadequate resources and an inexperienced crew.

CREATING IMMERSIVE SOUND

After this year-long exploration of immersive sound in the pages of TV Technology I am further convinced that it is not hard to create and produce immersive sound with minimum resources and would like to offer a couple of observations:

First, to create immersive sound a mixer needs to be able to hear the dimensional sound-field. Second, the mixer must be able to adequately position the sound elements in the sound-field; and finally the mixer must be able to make and maintain an artistic balance of the elements in the dimensional sound-field.

In the April issue of TV Technology I visited an OB van with less than perfect but adequate immersive monitoring. I can only say that NHRA puts out a credible 5.1.4 immersive sound production as well as 5.1 surround from one OB control room.

An alternative workflow would change the location for mixing and monitoring immersive sound. For more than a decade, FIFA World Cup and NBC have been sending mono and stereo stems from the venues back to master control to be mixed and I can clearly see the further trend to centralized control rooms for multiple remote productions.

There also seems to be some schools of thought that you need ambisonic microphones to create immersive sound. I spent 2019 recording side-by-side a 1st, 3rd and 4th Order ambisonics microphone along with 10 correlated and non-correlated spot microphones. I’ve come to the conclusion that ambisonic microphones sound great, but are not essential to creating immersive sound.

Immersive sound for sports and live entertainment is a subjective balance between venue ambiance and atmo-sphere, as well as relevant event-specific sound. With any sport (event) minimum immersive sound production can be as simple as injecting additional ambiance and atmosphere into the height speakers. To me, American and European football (soccer) are the definitive example of overhead atmospheric enhancements and this method does not require any localization. 3D panning is helpful for precise localization, but bottom line—convincing immersive sound for most sports does not require precise localization.

Some sound above the viewer/listener will usually create a sense of aural space for the 2D picture, but sound designers are wrestling with the concept of what sounds should be heard above the viewer when there is no obvious reason. Sure, we can put some sound up in the height channels, but how long is more crowd sound going to be a compelling reason for immersive sound.

SUCCESSFUL IMMERSIVE SOUND

Immersive sound production will develop and evolve with creative sound design and imagination. The picture is still two-dimensional and sound could become a production differentiator where “made for TV” sound such as NHRA will become a viewer draw.

There is no doubt that field sports such as football have no sports-relevant sound in the vertical axis, only ambiance and atmosphere, and artificial embellishments are probably not appropriate. But consider that perhaps enhanced ambiance is enough of an embellishment for football fans—immersive sound does not need to be over the top to be effective.

Hoping for interesting and successful immersive sound is not going to advance our audio agenda. NHK in Japan has spent almost two decades preparing for immersive sound. Dreaming, theorizing, research, testing, evaluation, planning, preparing new sound designs and documents have contributed to and facilitated a successful implementation of immersive sound within the Japanese broadcaster. NHK will produce the entire 2020 Olympic Games in 22.2 immersive sound.

Immersive sound will be proven one step at a time. Immersive sound is not new, but the concept of immersive sound is evolving. Remember, I said NBC would produce 9.1 immersive sound and NHK would produce 22.2; both formats claim to be immersive sound. What is the difference on a soundbar?

Advance audio designs are eminent but will be built on proven successes. What is the channel configuration for immersive sound? Is it 5.1.2, 5.1.4 or 22.2? I do not know. I am currently testing sound schemes that concentrate the listener’s attention forward and uses the height element to draw the focus beyond the normal left and right peripherally. This is an interesting concept for immersive sound and seems to work well with soundbars.

The migration to immersive sound must be planned, methodical and most importantly successful. Immersive sound must be marketed to the consumer and easy to install with minimum wires. Stereo and surround sound had growing pains, but I could not imagine going back to mono sound or black-and-white TV.

Immersive sound has a promising future and I look forward to covering it in the pages of TV Technology.

Dennis Baxter has spent more than 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production. He can be reached atdbaxter@dennisbaxtersound.comor atwww.dennisbaxtersound.com.

In the August issue of TV Technology, I covered the topic of monitoring and metering as a visual reference for audio mixers (“For Audio Monitoring, Seeing Is Believing”) This column continues the discussion of monitoring and the art of listening.

Critical listening in small and acoustically challenging spaces can be tricky at best and almost impossible in an OB van—almost. The problems range from poor speaker placement, improper speaker calibration and alignment, inadequate acoustic treatment in the OB van, frequency masking from extreme ambient noise including HVAC along with excessive chatter on intercom channels. Note: All of this can result in deceptive mixes and mixer fatigue.

IT’S ALL ABOUT DESIGN

Overcoming the myriad of problems associated with inherently deficient listening spaces begins with speaker designs for critical listening. Speaker design has generally been about originating all frequencies from the same location in space and time. This clearly means that the speaker, enclosure and acoustics should not change the direction and dispersion of the sound waves. There is no doubt about the difficulties of keeping the low and high frequencies constant in the horizontal and vertical planes because of the vast difference in the size of low and high frequency sound waves.

Many speaker manufacturers have adopted the term and principles of a shaped wave guide, which physically contours the speaker and/or enclosure to guide the propagation of the sound waves. The intended results are controlling the directivity and dispersion of the sound waves in an attempt to more focus the sound at the mixer while neutralizing reflections off the adjacent surfaces including walls, floor and ceiling as well as equipment. The downside is that a tighter focus of the sound waves may also result in a narrower “sweet spot.”

Multispeaker enclosures use separate transducer drivers for the high and low frequencies, which can lead to uneven reproduction, phase and distortion issues.

By locating the transducers as close as possible to each other and to the listener, this positioning helps the brain aurally perceive that there is a single point source. Point source is a relatively common speaker principle because of the advantages for phase coherence, while the ultimate point source design is coaxial speakers.

Coaxial speakers use a single driver to reproduce the high and low frequencies and tend to minimize some of these problems. According to Thomas Lund, senior technologist with Genelec, “From a perceptual point of view, organic (sound) sources above 300 Hz, like the human voice and musical instruments, radiate this way naturally.” As we know, the dominant sound in both film and broadcast is the human voice.

Even though coaxial speaker design has been around awhile, so have electrostatic and variations on ribbon type speakers. The inherent advantage to electrostatic speakers is that one diagram (like the coax speaker) uniformly drives the HF and LF, but electrostatic designs tend to be large and fragile and not really an option for OB use.

Ribbon transducers are based on the same design principles as typical magnetic- type moving coil speakers, but use different materials than typical tweeters. An interesting variation on ribbon designs is a concept called “Air Motion Transfer,” because AMT moves air differently and this design sounds different than typical tweeter designs. AMT is a folded diaphragm that moves air up to four times faster, resulting in quicker transient reproduction which, in turn, means more efficiency, less distortion and potentially higher frequency response.

NEAR-FIELD SPEAKERS

Speaker location has been a combination of room space and speaker design. In the last 30 years, I have seen improvements in the OB van audio space, but even with these improvements it still falls short in adequate soundfield reproduction and lack of proper acoustic treatment. Near-field speakers try to address these problems because near-field speakers, as the name implies, means the speakers are closer to the listener to minimize environmental acoustic coloration.

Generally, small speakers require a sub-woofer to reproduce the low frequencies, while mixing for film and broadcast require a low-frequency effect (LFE) speaker. I will suggest that this is the origin of some confusion between using the sub-woofer for bass management and the role of the subwoofer as the LFE.

The overall sonic experience of live sports audio for home listeners has improved with soundbars, but I believe that live sports sound tends to have an unnatural rumble because the sound mixer cannot properly hear full frequency 5.1 surround sound. Significantly, many up-processing devices push some of the sound into the LFE channel and a lot of this content that is up-mixed is definitely not LFE content. I’ll discuss this in a future column.

All listening rooms need to be tuned and virtually all manufacturers incorporate some sort of equalization and level contours with fixed set of switches to try and tame the speaker imperfections and room acoustics. I could not find another manufacturer that uses a dynamic software control in each monitor like Genelec. Software is used to configure, calibrate and control each monitor in the system for frequency response, level and time-of-flight or distance of the speakers from the listener.

Wave guide, coaxial and ribbon are viable speaker designs along with proper equalization and alignment, but now we must consider the size of the enclosure. Most OB vans have implemented sound monitoring within small rectangular spaces, but above speakers for immersive sound are going to be deceptive and problematic. Because of the nature of the OB van audio mix room, most speakers are relatively small and the addition of more speakers for immersive sound is tricky. Speaker placement is critical where the distance necessary for an accurate reference is a consideration.

The Ones speaker system by Genelec, unlike any others, offers a variation of point source and coaxial principles, which can deliver an accurate listening reference at a distance as little as 50 cm and an immersive impression in a 2 meter square.

Speaker sonics, high listening levels, distortion, unnatural imaging, all occur while the sound mixer must listen to a wide range of audio sources including program audio plus communications. There is no doubt all contribute to listener fatigue—a topic rarely discussed. Appropriate speaker design is a good start along with considering alternative workflow and ear training, all of which will facilitate immersive sound as the new standard for broadcasting. Immersive sound production is demanding in the OB Van, but not impossible—and it’s time to get ready.

Dennis Baxter has spent more than 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He can be reached atdbaxter@dennisbaxtersound.comor atwww.dennisbaxtersound.com.

The 600 MHz wireless spectrum auction has come and gone, taking some television stations off the air, forcing more than a thousand others to relocate, and causing repercussions that are starting to trickle all the way down to wireless device users. It was already apparent that owners of equipment operating in the auctioned spectrum would need to replace or reband their devices in order to avoid interfering with the mobile devices that will soon be operating there, but many were planning to put this off as long as possible.

With announcements coming from the auction winners of accelerated rollouts of 600 MHz mobile devices, there is no longer any time to waste. Wireless device issues are already on the increase as new devices are lit up and tested, with users puzzled as to why their previously fine wireless systems are suddenly giving them trouble. Though this has resulted in problems for wireless users, it has, perhaps inadvertently, created a place for engineers who can advise customers on their wireless issues, especially if they are able to perform wireless frequency coordination. Some of these calls have come my way, which caused me to take a fresh look at the problem and the newly available tools to help us deal with these issues.

INTERMODULATION DISTORTION

At its most basic, coordinating the frequencies of wireless devices is as simple as ensuring that all transmitters are operating on different fundamental frequencies, but of course, things are never that simple. Those fundamental frequencies have harmonics with enough strength through at least the fifth order that they can interfere with the frequencies of other nearby transmitters, resulting in intermodulation distortion. This typically manifests itself through popping or other unwanted noises that emanate from the receiver, usually at the worst possible time. 

[Read: FCC To Hone Wireless Mic Rules]

Calculating the harmonics of fundamental frequencies is easy, with the second harmonic double that of the fundamental, the third harmonic triple, and so on. For instance, a fundamental frequency of 626.200 MHz has a secondary of 1252.400 MHz (1.2524 GHz), and a third of 1876.600 MHz (1.8766 GHz). A single transmitter operating on its own in an area of clear spectrum will have no problem, but as soon as a second transmitter is powered up things get complicated, and the complexity increases with each transmitter added to the system. As soon as more than one transmitter is online we must determine whether the frequencies of each will interfere with the other, and the best way to do this is by using frequency coordination software made specifically for this purpose.

However, before we can coordinate the frequencies of the devices we know about, we need to figure out if other devices might also cause interference, a process done by performing a frequency scan of the area using an RF spectrum analyzer. These devices monitor and display RF spectrum activity over a specified frequency range. After running the scan, the analyzer generates a report which gets imported into the frequency coordination software along with the data from our known devices. 

Most frequency coordination software also includes the ability to pull in data for local television stations, which needs to be factored into the intermodulation calculations along with all the other data we’ve gathered. Once all the data is loaded, the software calculates any conflicts and suggests changes to the fundamental frequencies of the transmitters in order to get all the devices to play well together.

THE AUTO-SETUP OPTION

It is theoretically possible to eliminate almost all of this work by using the auto-setup features included with some wireless systems, since it polls the wireless spectrum and automatically assigns the best frequencies for all transmitters in the system. Unfortunately, auto-setup typically requires that all devices be from the same manufacturer and be banded in the same frequency range. Most real-world audio systems contain devices from multiple manufacturers with transmitters banded across multiple ranges. 

Building a system of wireless microphones, in ear monitors, and wireless intercom stations from a single manufacturer, that meets end user requirements, can problematic if not impossible. Still, auto-setup solutions are an excellent option for simple systems, or systems engineered this way from the outset, but most systems which have evolved over time are far more complicated than auto-setup can handle. Still, anyone who has decided that now is the time to replace their transmitters should consider new systems that are frequency-agile with the ability to manage frequency coordination without additional tools.

The good news for anyone given the task of wireless frequency coordination is that RF analyzers and coordination software tools are more widely available and affordable than ever before. RF analyzers now cost from tens of thousands down to hundreds of dollars and coordination software from hundreds to tens of dollars. The precision of the analyzers and comprehensiveness of the software is what sets them apart, with the more expensive options offering more features and generally making life easier by reducing the amount of work required.

Back when I worked for a large media company, our coordination work was done with a very high-end Anritsu analyzer coupled with the Intermodulation Analysis System (IAS) software from Professional Wireless. These days my toolset is far more modest, with an RF Explorer analyzer, Touchstone analyzer software for RF capture, and Shure Wireless Workbench software for coordination. Using Wireless Workbench with devices from other manufacturers is not a problem, but it requires building device profiles for those not included. The process itself is much as described earlier—connect the analyzer, gather RF data, pull all data into the coordination software, run the analysis, then make any necessary frequency changes to the transmitters. It doesn’t hurt to run another coordination once things are changed, just to double-check that everything is working without conflict.

While wireless coordination will most likely be needed for 600 MHz devices, it may also be necessary for 470–512 MHz devices if you happen to be in one of 11 large US metro areas, because that spectrum has been designated for public safety (T-Band) land mobile radio use.

Jay Yeary is a television engineer who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology magazine or at transientaudiolabs.com.

The beginning of a new year is an excellent time to reflect on life and ponder mysteries—such as why we make a big deal out of one year starting as another ends, and whether time is just an imaginary construct in the first place. Of course, it’s also a good time to look back at the tumultuous year that just passed and consider what the new year will bring.

Next-generation audio’s biggest impact will be realized through earbuds and headphones.

NEXT GEN AUDIO
Next Generation Audio was top of the list of audio revolutions that made headlines in 2017 while making almost no difference to the end user. That will change in 2018 as ATSC 3.0 begins to roll out to consumers and the age of immersive and personalized television audio begins.

The greatest challenge for the companies tasked with delivering NGA technology will be to make the interface user-friendly and as brain-dead-easy as television is supposed to be. If they deliver real usability to the masses we could see a serious resurgence of interest in broadcast television, and if they don’t, people will simply turn back to their streaming boxes.

As for the technical side of NGA, it would be wonderful to see immersive audio transform multitudes of living rooms into surround environments, but its greater impact will almost certainly be realized through earbuds and headphones. People already love their individual listening devices and immersive audio will deliver a very personal and important television audio experience.

STREAMING TV AUDIO
It will be interesting to see how things go in the world of streaming television audio this year, especially since streaming has amazing potential as a delivery platform for personal immersive audio. The AGOTTVS group within AES has done a fantastic job of setting guidelines for content creators and streaming providers to follow, but whether those guidelines will be followed across the board remains to be seen.

When Congress mandated loudness management through the CALM Act, it had little impact in Silicon Valley, but if streaming audio loudness problems aren’t solved voluntarily, it is entirely conceivable that similar legislation could be put in place for streaming.

AOIP
Audio and video-over-IP will continue to grow this year as television facilities add more IP- and IT-based equipment in place of traditional equipment. As this happens, television facilities will begin to look a lot more like IT facilities.

As the level of IP-based equipment increases so does the need for engineers with IT skills while the need for traditional television engineers decreases, something everyone working in television engineering needs to keep in mind.

At the same time, SMPTE, AES and other professional organizations continue to develop and refine standards to make the myriad television IP standards function together and interoperate.

IMPACT OF FCC CHANGES
The FCC was certainly busy last year, and at times their decisions didn’t seem terribly pro-television. On Dec. 14, 2017 the FCC reversed 2015 Title II Order, which deemed the internet to be a public utility, ending what is widely known as “Net Neutrality.” Prior to the 2015 order, internet providers were actively throttling traffic of various types.

Now that the order has been reversed we’ll have to wait and see what impact, if any, this decision will have on our ability to successfully deliver streaming television content to customers. Of course, 2017 also brought the reality of the wireless frequency repack and sell-off of spectrum, which displaced nearly 1,000 stations and caused chaos and confusion throughout the industry.

In the world of wireless microphones, belt packs and IFBs, 600 MHz spectrum is already gone, with T-Mobile announcing plans to start utilizing those frequencies at any time.

Between the loss and shuffling of spectrum and the possibility of throttled streaming, it feels like a very bad time to be in the business of delivering television content.

AUGMENTED REALITY
Virtual reality was quite a fixation in 2015 and 2016, and it hasn’t really gone anywhere. But a technology that is far more interesting, which will make big waves this year, is augmented reality. AR doesn’t replace the world around us with something different as VR does; it adds dimension to the world we’re in.

This technology has actually been around for years—there were AR apps for older smartphones, and Pokemon Go is an extreme example of the potential of AR technology.

Augmented reality may prove to be an enormous benefit to healthcare since doctors can have medical information immediately at hand using goggles or glasses, but the potential for augmented television content is there as well.

With consumers already watching content on smart devices, AR is a technology just begging to be utilized. For audio professionals, this area is especially intriguing because AR and VR experiences are hollow without an audio soundscape to accompany the visuals.

iZotope’s Neutron track assistant

MACHINE LEARNING
Artificial intelligence was the subject of many, many discussions in 2017 and it will continue to be in 2018 as people worry that robots will become sentient beings and take over. Machine learning, on the other hand, sounds far less ominous so we’ll use that term instead.

Machine learning can best be described as building a machine that can learn and then letting it do so. Google’s language translation AI is a perfect example of machine learning because it has created its own internal language in order to translate languages quicker.

The fruits of machine learning have already started to appear in some commercial audio products, but there is a far deeper movement using it for all kinds of audio work. Acoustics, signal processing, language and signal analysis, music classification, signal and source separation are just some of the areas where machine learning is already being applied.

My first inkling that machine learning was in use for audio software was when iZotope released Neutron, their mix analysis and processing tool. Experiments with Neutron’s track assistant have caused me to rethink some final mixes. Machine learning has made its way into several iZotope products and they’re not the only audio company using it. No matter how you feel about it, machine learning is here to stay, so expect to see more of it in 2018.

Jay Yeary is a television engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology magazine or attransientaudiolabs.com.

As mature as the television industry is, it appears to be going through a very disruptive identity crisis right now. Cable providers are hemorrhaging subscribers while over-the-air broadcasting is being rediscovered; broadcasters and content creators are rolling out their own streaming services to ensure they have direct access to customers without the need for third parties; the journalistic integrity of traditional news organizations is under attack while tech companies deliver uncorroborated information to users while denying any responsibility for determining its credibility; and viewership of traditional sports broadcasts seems to be waning while interest in eSports is on the rise.

Yet despite this changing content landscape, viewing of television content seems more popular as ever, even as the array of devices used to consume it and the possible environments it gets consumed in appears unfathomable.

NEXT GEN TV
The impending rollout of ATSC 3.0 promises to bring together broadcast and streaming content in ways we haven’t experienced up to now, with the ability of immersive and customizable audio to be delivered to practically every device, and to be enjoyed through speakers or headphones. Unfortunately, not all streaming content is quite ready to become a part of the broadcast ecosystem, either due to poor quality or because it doesn’t meet broadcast standards.

One area where this is apparent is with streaming audio loudness, since the disparate streaming services and online outlets use different loudness settings. The good news is that most outlets now implement some sort of loudness management for content; the bad news for content creators is that this is usually done through automated normalization rather than through monitored gain modification, so audio content could be negatively altered.

AGOTTVS Streaming Audio Loudness Recommendations

The first North American attempt at setting loudness standards for streaming content was done by the AES Study Group on Streaming Loudness, headed by recording engineer Bob Katz, who wanted to set loudness standards for music streaming.

In October of 2015, the group released the document, “Recommendations for Loudness of Audio Streaming and Network File Playback,” which recommended that target loudness of non-metadata encoded music reside between –16 and –20 LUFS. While this was a great start, it was immediately realized that all other streaming audio content would benefit from similar loudness recommendations, especially the audio accompanying streaming video.

So, in early 2016 the AES Audio Guidelines for Over the Top Television and Video Streaming (AGOTTVS) Study Group was formed, chaired by Jim Starzynski, the director and principal audio engineer at NBC Universal. His previous foray into loudness management was as part of the ATSC committee that created A/85, which serves as the core of the CALM Act and regulates broadcast audio loudness.

The group is made up of members from broadcast and cable networks, content providers, device manufacturers and content delivery services, as well as other interested parties, all working together to gain agreement on standards across the entire content chain, from creation to delivery to the consumer.

Preliminary AGOTTVS guidelines were published in September 2016, and on October 19, 2017, the AES announced an update to those guidelines that appear to be on their way to becoming an AES recommended practice. AES Technical Document AESTD1006.1.17-10, titled, “Loudness Guidelines for OTT and OVD Content,” can be found at AES.org and it is worth downloading and reading all the way through.

The document’s objectives are spelled out clearly in section 2, namely to “Provide consistent loudness across different Programs, provide appropriate loudness range for devices and listening conditions, prevent … processing from degrading audio quality, preserve the artistic intent, and improve the listening experience.”

Basic recommendations are along the same lines as A/85: use ITU-R BS.1770 measurement tools and make use of metadata with content whenever possible to take advantage of available dynamic range control and loudness management.

LONG FORM VERSUS SHORT FORM
For North America, loudness measurements of long-form content should be done by measuring the anchor element, typically dialog, while full program measurements are recommended for short-form content. For long-form content where the anchor element cannot be determined, it is recommended that the full program be measured.

The guidelines themselves have recommendations for six possible content scenarios: delivery and distribution of properly prepared content; systems with questionable metadata capabilities; devices or environments with limited dynamic range; and for content created with no regard for standards or recommendations. Quite a few of the guidelines mention prior arrangements, which have been made between the content creator and distributor, and in those situations the document recommends that loudness and true peak values should be measured and should not exceed recommended regional targets.

Specific targets can be found in Annex B and vary according to region. Loudness targets for North America are –24 LKFS, plus or minus 2 dB, with maximum peaks no higher than –2 dB TP. Maximum loudness is never to exceed –16 LKFS, plus or minus 1 dB, with peaks measuring no higher than –1 dB TP.

While these recommendations should help moderate loudness of television and video content, the document notes that the loudness of sounds generated in the system itself, such as phone and tablet alerts, may differ from content loudness and requests that developers work to match loudness of all sounds across the device in future versions of software, though emergency alerts are expected to be louder than other content due to their nature.

Also noted is the fact that some devices utilize codecs, which may not support metadata, resulting in undesired playback loudness from the device and suggests that they follow CTA CEB11 and EBU Tech 3344 recommendations to mitigate this.

Finally, the document goes into the differences between fixed and portable devices, their environments, and typical expected capabilities.

This document is a major step forward in sorting out the loudness issues between different streaming services and the many devices on which their content is consumed. While AGOTTVS has a wide cross-section of industry participation, it’s not everyone. Some are waiting on a standard to be created before implementing changes, and while this document will only be a recommended practice, so is A/85.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology magazine or attransientaudiolabs.com.

For the past 13 years, on a Saturday in mid-autumn, the AES Atlanta Section has held their workshop for students. The classes are taught by audio professionals and are held in working audio production rooms.

The workshop concluded with a discussion featuring all of the instructors, giving the students a final opportunity to ask questions.

This year’s event took place Sept. 23 and I was honored to be among the instructors, teaching the “Critical Listening” class alongside composer Tanya Ostrovsky. This class has historically been taught by multiple Grammy Award-winning engineer Jim Anderson, so we had some rather large shoes to fill.

Joining us as instructors this year were Michael Cardillo (Creative Waves) teaching “Pro Tools Fundamentals,” Miles Walker (Gwen Stefani, Coldplay) covering “Mixing Fundamentals” and Matt Still (Elton John) and Bob Gillespie (Turner Studios) digging into “Fundamental Microphone Techniques.”

The students rotated through the four classes throughout the day, ending up with a broad cross-section of audio instruction for six hours.

CRITICAL LISTENING
Critical listening is one of those intriguing topics you see mentioned every so often, but the ability to discern sounds is one of the defining characteristics of professional audio engineers. It is a skill that can and should be developed by everyone planning to work in the audio field.

The idea of listening critically as a discipline is something I find compelling because we sometimes get so wrapped up in the technical aspects of our jobs that we forget to slow down and really focus on what we’re hearing before making adjustments.

Needless to say, being a part of this class, which forced both the students and the instructor to simply listen and analyze, was utter joy.

My goals for the workshop were to get the students to really think about how to listen; get them listening critically with fewer assumptions; and to, hopefully, get them to start listening to sounds in the real world without constantly having transducers stuck into their ear canals.

I sometimes wonder if the sounds of nature, rural communities and even cities will one day become unrecognizable as we continue consuming audio content primarily through the devices we tote along in our pockets.

Tanya and I took turns teaching the Critical Listening class, handling two sessions each. My classes were loosely based on the book, “Critical Listening and Audio Production” by Jason Corey and were taught from my perspective as an engineer. Tanya’s sessions were taught from the viewpoint of a composer, with much heavier emphasis on music theory, with materials and listening tests from the “Golden Ears” course by former TV Technology audio columnist Dave Moulton.

Watching speech on the spectrogram was particularly interesting because it showed how wide the frequency and tonal range of the human voice truly is.

Because music creation and sound engineering are so intricately intertwined—and to aid in pinpointing and understanding the overall concept of frequencies—students were first given diagrams showing the frequency ranges of musical instruments and the human voice, along with the 10 octaves of hearing.

This led to a discussion of how instrument frequency ranges overlap, how timbre, pitch and resonance are among the characteristics that help us distinguish instruments and other sounds. From there we discussed the benefit of arranging music and soundscapes so that sounds with similar tonal characteristics don’t compete with each other in the mix because it tends to make the mixing process more difficult and the results often less pleasing.

In Corey’s book, he calls for active and analytical listening on the part of audio engineers and that is what we attempted to develop through the playback of a range of audio material, including music, speech and sound effects. Each example was selected for its distinctive instrumentation, to demonstrate a specific audio focus such as dynamic range, distortion, noise, reverberation and rooms, etc., and to generate discussion about what we were hearing. The goal was to recognize the unique characteristics of each example and to distinguish what certain sounds actually were.

UNDERSTANDING AND VISUALIZING FREQUENCIES
To help the students understand and visualize frequencies, all examples were presented through a spectrogram plug-in, which allowed everyone to see the frequency and intensity of sounds as they occurred. Watching speech on the spectrogram was particularly interesting because it became obvious how surprisingly wide the frequency and tonal range of the human voice truly is.

The one problem with the spectrogram plug-in was that I periodically had to remind students (and myself) that closing our eyes allows us to listen more intently. Once we worked our way through the audio elements, we started making systematic sweeps, then boosts and cuts, using a parametric equalizer to hear how sounds change when adjusted at given frequencies. This was in preparation for the final part of the class, which was a simple boost and cut equalization test using Train Your Ears EQ Edition software and a couple of new audio elements.

Some of the students attending the workshop were already doing audio production to varying degrees. Others were intrigued by the idea of working in audio as a career, and some didn’t know much about it, but were there to learn more.

By the end of the day, after attending all four classes, they had been inundated with a lot of practical, useful knowledge and had been given the opportunity to spend time with some incredible audio talent.

This workshop always concludes with a round table discussion featuring all of the instructors, giving the students one final opportunity to ask questions of the audio professionals they’ve spent their day with. Invariably, those questions include how to get hired into the business; how the instructors got into the business; and how they became successful.

Yet there are two other, far more important things, that have remained constant in all the years I’ve participated in the workshop; the curiosity that eventually rises to the surface once students understand that their questions are being answered openly and honestly and the palpable excitement they exhibit after they realize they actually could spend their lives making music, manipulating audio and staying close to their passion.

It always reminds me of how I felt in my early years in audio and also makes me wonder whether those of us working in this business realize how truly fortunate we are.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE and TAB. He can be contacted through TV Technology magazine or at transientaudiolabs.com.

Recently I ran across an interesting piece from the Joint Taskforce for New Media (JT-NM) on the “dematerialized facility,” which envisions broadcast facilities built entirely from commodity IT equipment or with everything outsourced and no onsite equipment at all. Prior to the widespread availability of high-speed internet, the prospect of a disembodied broadcast facility would have seemed preposterous, but that is no longer the case and it is a very real possibility that broadcast facilities as we know them now will become relics of the past.

Dematerialization is essentially what happened to the large majority of recording studios and post houses—they no longer exist in the forms they possessed in the past, with recording shifting to home studios and post work being outsourced to freelancers working from personal facilities. It is unlikely that broadcast and cable network master control chains will move to homes, but it is absolutely conceivable that the technical infrastructure and content delivery chains could be outsourced.

Is this the boring future of all broadcast facilities?

A TALE OF TWO STUDIOS
The audio industry is already, to an extent, living in the dematerialized world. The advantages of minimal infrastructure were brought home to me during a just-completed studio move where there were two very different audio studios to uproot and relocate to a different building about a mile away. As usual, the move was done under tight time constraints due to scheduled sessions bookending the move. Fortunately, only one studio needed to be online for the sessions since the other had been relegated as backup in the event the primary studio was down.

To accommodate the booked sessions meant I had around 20 hours to dismantle, move, reassemble, and test at least one studio. The deadline was met and sessions happened as planned, but only because of the design of the studio I chose to bring up first—one that is in line with the vision of a dematerialized facility.

The other, more complicated studio is one a friend and I designed and assembled over 20 years ago. It is equipped the way small broadcast audio studios used to be built—with a standard mixing console, patch bays, audio meters, a limiter/compressor, an amplifier for unpowered main speakers, telephone hybrid, and an ISDN codec. Its original implementation also included reel-to-reel, cassette, and DAT recorders, but these were removed as their technologies were supplanted by hard disk-based systems.

The second, newer studio was built by another engineer, is much simpler, and takes advantage of advances in technology in the two decades since the first was built. At the center of this studio is a computer running recording software, a small USB audio interface, a small audio mixer for signal routing, a high-quality microphone preamp, balance boxes for level matching, powered speakers, and a telephone hybrid. The ISDN codec in this studio has been replaced by Source-Connect Pro, with third party bridging for the rare ISDN session that does occur. Both studios have talkback and headphone distribution to a portable recording booth, and the older studio has custom-built telephone and ISDN control, so session connections can be managed remotely without leaving the booth.

Using the portable recording booth allowed us to set up the recording space in an untreated room without making any physical changes to the room itself. It also meant we could get that space ready before either of the studio control areas were assembled. The simplified design of the newer studio enabled quick equipment placement and setup of the work area. Once the equipment and external lines were connected, all that was required was to check device and software configurations, make test recordings, and check the phone patch and Source-Connect connections in the control room and voice booth.

Live event production will always need some sort of fast user interface for mixing and routing, but it may be some time before mix engineers are willing to give up tactile faders.

ROOM FOR IMPROVEMENT
Once testing was complete, the studio was ready for sessions again—and they started almost immediately, just 10 hours from the time the booth and studios were disassembled for relocation. The older studio did not come together quite as quickly. The physical location of the equipment in this studio in relation to the voice booth and the main studio meant that it was necessary to change the racked placement of some equipment and remake some cables. In fact, the sheer number of cables in the older studio were the primary reason it took almost 16 hours to rewire and test it after reassembly of the racks.

Even though the newer studio was much quicker to reassemble and test, it could be made even simpler. The audio mixer could be replaced with a slightly larger, more capable, audio interface which could also handle record feeds, talkback, phone patch, Source-Connect routing, and headphone feeds. The telephone hybrid could be replaced by a VoIP box or software. With Audio-over-IP interfaces now widely available, duplicate recording rigs could be built easily and relatively inexpensively, and could be tied together and extended far more easily, with far fewer cables than traditional recording studios and even some newer computer based systems.

When we ponder the prospect of dematerialized facilities it is important to remember that that future is one we’re already living in. Computers have been transitioning to virtual machines for years. Portable recording booths and moveable acoustic panels not only mean quicker recording space setup, but also less physical modification of rooms. Using off-the-shelf computers, drives, switches, and other IT-centric components means easy replacement and upgrades, and using cloud-based delivery platforms like WeTransfer, DropBox, and Gobbler eliminates the transfer of physical media. Internal routing within interfaces and other devices negates the need for unnecessary external wiring. Replacing outboard gear with plugin emulations not only saves rack space and power, but also eliminates conversion delays in and out of the system.

Live event production will always need some sort of fast user interface for mixing and routing, but it may be some time before mix engineers are willing to give up tactile faders. Future broadcast facilities may be some hybrid of an outsourced, virtualized, IT-based infrastructure, but it certainly feels that, to some degree, audio technology has paved the way.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology magazine or attransientaudiolabs.com.

ATSC Recommended Practice (RP) A/85 is usually thought of as a loudness standardization document full of guidelines we must follow to ensure content loudness complies with the CALM Act. In fact, the document goes beyond loudness to try and standardize the process of creating and delivering audio content for television, similar to standards used in audio for cinema. The process of mixing audio for TV has always been a bit freeform, partly because the lead time on most television productions is short and partly because the immediacy of live television makes the entire process chaotic, so successful mix engineers end up developing their own practices and processes.

One of the key areas A/85 seeks to standardize is control room speaker monitoring levels, to help engineers inherently understand their mix levels without staring at meters. To understand how well the process works I’ve been experimenting with, and attempting to hold to, fixed monitoring levels on everything listened to in the studio, whether on speakers or through headphones. This requires volume controls to be fixed in place and not adjusted when audio is too soft or too loud, a habit which has turned out to be hard to break. When the level of any audio source isn’t correct—whether audio from a studio project, music I’m listening to while working on something else, or content from the internet—it gets adjusted at the source.

After doing this for just a couple of weeks it feels like I now have an immediate grasp of whether levels are correct without looking at the meters, so it appears that fixed monitor settings do help mix engineers gain a better understanding of the audio levels within their system.

CALIBRATION
Properly setting up fixed monitor levels means calibrating your speakers and headphones, a process detailed in A/85, though it first covers the importance of correct speaker locations and recommends room equalization to help control modes.

Reference-level calibration begins in section 10.4 with a quick reference guide for the process in Annex D (Fig. 1). Recommended sound pressure levels (SPL) vary, based on one of five control rooms types, with Category One reserved for large main audio mix rooms, all the way down to headphones in Category Five. When the document was written, headphone settings were targeted at noisy environments like machine rooms and QC stations, but with the explosion of personal audio devices the headphone mix may become as important as the mix coming from the speakers. Items needed to calibrate the system include an SPL meter with C weighting, audio system metering, and audio files which can be downloaded from the links provided in section 10.4.

Fig. 1: ATSC A/85 Rooms and Reference SPL

Once meters and audio files are ready, calibration is done by following the steps outlined in the document. It is important to adhere to the SPL recommendations for each size room since the same SPL will be perceived differently in rooms of differing sizes, and to ensure that mix engineers feel comfortable with the calibrated levels in each room. Once level calibration is complete make note of and mark gain settings so the room can be returned to its calibrated settings if they are inadvertently changed.

Fig. 2: The G.R.A.S. “artificial ear” is designed to accurately test headphones, microphones telephones and loudspeakers.

USING HEADPHONES
Those who want to perform a quick calibration should look to A/85 Annex D.2, which is titled, “Really Quick Reference Guide for Monitor Setup.” My experiment began in an even simpler, less scientific way—by playing audio files through the system, ensuring that they measured –24 LKFS on a loudness meter, then adjusting monitors and headphones until the audio level was comfortable and not loud, since audio with any dynamics will contain louder segments. Measuring the SPL from speakers is relatively straightforward but determining how to accurately measure the SPL of headphones is a bit more problematic. A/85 doesn’t really clear it up by stating that we should, “use 2 cc coupler and set 440 Hz level to 74 dB.” It turns out that these audio couplers are designed for measuring hearing aids, but getting them to work with headphones looks a bit awkward and they seem to work best when coupled with a device called an “ear simulator,” (Fig. 2). A homemade alternative might be to rig a dummy head and SPL meter. Unfortunately, none of these are devices we tend to have sitting around the control room.

Fig. 3: NIOSH Sound Exposure Limits

One thing is certain, it is especially important to make sure that headphone levels aren’t too loud because the transducers sit close to the ears and anyone making their living in audio must protect their hearing. According to the National Institute for Occupational Safety and Health (NIOSH), we reach our maximum noise dose for the day after listening to audio at 85 dB(A) for just eight hours (Fig. 3). Every 3 dB increase in level above 85 dB(A) cuts listening time in half, so keeping headphone listening levels well under the NIOSH maximum recommendation seems like a prudent method of working.

A/85 recommended we start doing this with its release back in 2009 so it feels a bit like I’m late to the party on this one, yet most mixers I know still don’t work this way. This experiment has changed how I work in the studio and has converted me into a fixed-monitor-level believer. It doesn’t always go smoothly, especially when its necessary to grab an onscreen volume control to adjust a piece of loud content or when something is so soft it never reaches the correct level, but those tend to be the exceptions rather than the rule and they are easily dealt with in the studio.

The most surprising revelation of this experiment is that I’m experiencing and enjoying dynamics more because I have a better feel for the target. If you’re like I was, someone who reached for the monitor volume control whenever it was convenient, I encourage you to give fixed monitor levels a try. You’ll find yourself enjoying listening more and looking at meters only as an occasional reference, exactly as we’re supposed to be doing.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology magazine or attransientaudiolabs.com.

The face of television audio has developed some interesting new wrinkles since we last looked at loudness—namely, immersive audio, object-based audio and audio for personal devices. We’re just beginning to unwrap the specifics and demands of these new delivery formats as well as learning how to produce for them, but loudness is an important factor of each.

The best news on the loudness front is that measurements are still based on the ITU-R BS.1770 standards we’re already using. Existing loudness meters remain valid, though some updates will likely be required. First up, we’ll take a fresh look at the foundations of loudness before we examine the impact the new formats will have on it.

LOUDNESS METERS
Loudness meters are neither volume unit (VU) nor peak meters, but are K-weighted meters designed to measure loudness in a manner that more closely approximates how humans hear. They display Loudness K-weighted Relative to Full Scale (LKFS), where each unit of LKFS (aka Loudness Units (LU)) is equivalent to one decibel.

Fig. 1: Loudness meter

Measurement is done by first applying a pre-filter that simulates how a spherical human head acoustically influences audio; then a second-order high pass filter is applied. Next, a mean square calculation is performed on all channels before they are summed, with each surround channel getting a 0.5 dB bump in the calculation and the LFE being dropped from it.

Gating was added in BS.1770-2 to ensure that quiet passages and silence don’t unduly influence the reading, so these gates make up the final stage of measurement. Samples are taken in overlapping 400 ms blocks, with the first gate at –70 LKFS, and the second at –10 dB relative to the first gated measurement.

THE MEASUREMENTS
As we look at the measurements audio engineers will actually encounter, it’s important to remember that all loudness measurements are made over time. Therefore, each measurement will fluctuate throughout the program and displayed values will rarely, if ever, remain static.

The Momentary (M) reading is an ungated measurement of loudness over a sliding 400 ms window and is a good representation of loudness at a given moment. Short-term (S) is also ungated and displays loudness over a three-second continuously sliding window. Integrated (I) is the gated overall loudness of content from beginning to end and is the measurement used to ensure CALM Act compliance.

NEWER MEASUREMENTS
A few less familiar measurements include Loudness Range (LRA), Peak to Loudness Ratio (PLR) and Maximum True Peak Level (TPL).

Loudness Range is the overall variation between quiet and loud sounds within a given piece of content, but it’s really a little more complicated than that. According to EBU Technical Document 3342, Loudness Range is “Based on the statistical distribution of measured loudness.” So the measurement looks at the entirety of the content and uses a combination of absolute and signal-dependent gates to ensure that individual loud or quiet moments don’t skew readings.

Peak to Loudness Ratio measures the dynamic range of a mix and is primarily used for music production, with some interesting potential as a broadcast tool for mixers who want a measurement of the dynamic range of their mix. PLR is a real-time measurement, usually applied to the entire piece of content, though it could also be used to determine the dynamic range of individual sections of audio. Higher PLR numbers indicate a mix with a wider dynamic range and values closer to zero indicate less dynamic range.

Maximum True Peak Level is simply the highest audio level measured at a given point within the content. It should not be allowed to exceed either the –2 dB TP specification in ATSC A/85 or the True Peak value of the content delivery specifications.

NEW FORMAT LOUDNESS
When it comes to measuring loudness for immersive and object-based audio formats, testing seems to indicate that the loudness of immersive mixes closely track the loudness of rendered 5.1 or 7.1 full mixes.

Fig. 2: MasterCheck Pro meter showing PLR measurement

In e-Brief 352, from the AES Berlin 2017 in May, the authors presented data showing the loudness variation between an immersive movie mix rendered into a Dolby E-AC-3 5.1 mix. When decoded into multiple Speech Gated Loudness configurations the variation was a maximum of 0.3 LU, and maximum variation of 0.7 LU when decoded and replayed into multiple Relative-gated Loudness configurations.

Additional evidence that current loudness tools will work for NGA audio formats can be found in the Atoms Production Suite manual, which states that, “You should use the 7.1 full mix re-render for loudness measurement during post production to ensure that content meets delivery specifications.”

Suggestions have surfaced for specific multichannel loudness tools for NGA formats, but if measuring surround mixes is equivalent to measuring NGA mixes, then these new loudness tools may not be necessary.

PERSONAL DEVICE LOUDNESS
The outlook is a little murkier for personal devices such as mobile phones and tablets because there’s so much variation between them. With differing acoustic and electronic outputs, the potential listening options seem almost limitless. Content services seem to recognize the need for loudness management because most have instituted normalization, though not at the same loudness level.

Fortunately, work is already in progress to address the technical needs of these devices. The AES has one study group working on loudness standards for streaming and the AGOTTVS subcommittee is working on loudness standards for OTT and video streaming. Both AES groups have published preliminary guidelines and have broad support from the broadcast industry and content services. However, the sheer variety of available devices makes the labor of both AES groups extremely involved and technically challenging.

A lot has changed in the 14 years since Dolby introduced the LM-100, the first broadcast loudness meter, yet ITU-R BS.1770 and its refinements are holding up well in the face of new audio delivery formats, proving that it was a wise choice for modern loudness management tools. We’ll likely see more refined measurement algorithms in the future, as well as additional specialized measurements, but for now it appears that the loudness tools we’re using will continue to serve us for the foreseeable future.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted throughTV Technologymagazine or attransientaudiolabs.com.

Every April, thousands of us make a pilgrimage to the desert to learn from gurus who pass along insight and direction, give us glimpses into the future, and help us make sense of it all. We learn to discern viable technology from the vapor versions, while balancing countless miles of walking with unhealthy amounts of sitting in uncomfortable chairs.

This year more than 100,000 of us journeyed to the NAB Show in Las Vegas to learn, catch up with colleagues and gawk at the newest, shiniest equipment from vendors.

In recent years, I’ve shifted focus from the gear toward the amazing number of learning opportunities that coincide with the NAB Show. Conferences, training classes and vendor-sponsored events kick into high gear the weekend before exhibits open. For me, that weekend is consumed by Avid Connect and the DTV Audio Group meeting.

AVID CONNECT
Avid Connect has become increasingly intriguing as the company works to implement the Avid Everywhere initiative announced at the inaugural event four years ago. Attendees get a far more in-depth look at new features than is possible at the NAB Show, as well as the opportunity to interact with product development teams.

Avid announced a number of new developments for Pro Tools at Avid Connect just before the NAB Show.

A number of positive announcements were made at this year’s event. First is that, sometime in Q2 of 2017, Pro Tools will gain multi-seat access to Avid’s Nexis storage, officially enabling multiple Pro Tools systems to share projects and stream audio from the same Nexis storage pool as editorial users.

Shared network storage is often an unsettling option for Pro Tools users because a server’s network connection can get congested with traffic from all the small audio files and waveforms the workstation uses.

During a session on centralized storage, it was mentioned that the Pro Tools and Nexis teams worked together to ensure that Pro Tools waveforms are now cached in a manner that keeps them from saturating the network.

Next up was the announcement of native Atmos support in Pro Tools HD along with the release of two Dolby Atmos software packages—welcome news to anyone concerned about the lack of available tools for immersive and object-based audio production.

The Atmos Production Suite can be downloaded directly from the Avid Store and contains the tools necessary to create Atmos content in Pro Tools. The suite includes three panners, the monitor application, a VR transcoder and the Atmos renderer, along with renderer send and return plug-ins.

The basic bed track format is 7.1.2, allowing two channels for overhead/upper speakers rather than the four specified in ATSC A/342, and up to 118 audio objects. Object busses auto-fold down when there is no renderer, though busses must be configured first.

Some additional items of note came from the session on Atmos workflows. Demos were run in a pre-release version of Pro Tools 12.9, which included an Atmos peripherals tab, the ability to show Objects in the Clips menu, and a menu choice to “Duplicate Atmos Plug-in Automation to Pan Automation,” which converts existing Atmos automation into the type used in the Production Suite.

Other features include the ability to pan between mix and object busses and improved stem rerecording capabilities. These Atmos-specific items are not present in Pro Tools 12.7.1 with the Production Suite installed, so they’re likely coming in an updated version of Pro Tools, though it’s also probable the new version will differ from what was shown at Avid Connect.

The second software package, the Dolby Atmos Mastering Suite, is intended for DVD/Blu-ray mastering and consumer end-product creation and is available only from retailers. It requires either a Dolby RMU or the hardware equivalent of an RMU.

THE DTV AUDIO GROUP
The DTV Audio Group meeting is an afternoon spent gleaning important information from people who are driving change in the industry. Kicking things off this year was Thomas Edwards of Fox Networks, with an update on IP video. He showed how SMPTE IP standards are jelling into the SMPTE ST 2110 media-over-IP standard, with AES67 chosen as the PCM linear audio stream format for ST 2110-30.

The DTV Audio Group covered a range of next-gen audio developments.

Edwards noted that ST 2110 requires zero offset start times for AES67 streams and that the 2110-31 AES3 nonlinear transport is far from sorted out, so input from professional audio organizations is being sought. He concluded with a discussion of discovery protocols, noting that NMOS IS-04 would likely be the discovery and registration protocol used in ST 2110.

Next up was Kevin Gross of AVA Networks with an update on AES67, describing how it fits into the family tree of professional open standards. Responding to Thomas’ earlier comments, he pointed out that AES67 does not require zero offset streams because random offsets are more secure, but that in fact, many AES67 devices already use zero offsets. So compliance for ST 2110 is certainly doable.

Kent Terry from Dolby Labs delved into “Audio Metadata in Live IP Streams,” specifically addressing ST 2110-31 and offering KLV metadata as a potential solution.

Results and implications of the wireless spectrum auction were covered by Jackie Green, president of Alteros, who stated that devices using the 600 MHz frequency band must go quiet soon. Theoretically, they may be used for 39 months after the auction close date, but in reality all usage must cease as soon as the auction winner begins testing devices.

Rob France from Dolby explained Atmos production techniques used for the National Premier Soccer League, including creating 5.1.4 beds using Soundfield microphones. A progress update on AES audio guidelines for OTT was led by Jim Starzynski of NBC, followed by a recap of audio production at the Rio Olympics and a preview of the AoIP-heavy audio design for the forthcoming Pyeongchang games from Karl Malone, also of NBC. The meeting wrapped with a lively panel discussion on Next Generation Audio production.

Between the Pro Tools Atmos software announcements and news of the upcoming SMPTE ST 2110 media-over-IP standard with its inclusion of AES67, the weekend was, as usual, both informative and time well spent.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE and TAB. He can be contacted through TV Technology magazine or attransientaudiolabs.com.

Think about how we consume television now. In our house, cable is simply the pipe that delivers high-speed internet, and content arrives via a combination of over-the-air antenna and a handful of streaming services. We rely on smart devices to locate and play what we watch because we no longer know when it airs. Into this world of distributed, multisourced content our nextgeneration television delivery system is being born, a world where broadcast runs the risk of becoming an afterthought, at least until the internet service goes down.

Both the promise and challenge of ATSC 3.0 is to merge broadcasting into this internet- centric media landscape and, in the process, deliver high-quality content with properly delivered mixes to every device. Accomplishing this requires changing virtually every technology currently in use that delivers television to the home and brings with it new structure and expanded metadata.

Information about the audio elements of ATSC 3.0 can be found in three documents that make up the finalized ATSC A/342 standard. Part one covers the elements common to all Next Generation Audio systems, part two specifies AC-4, and MPEG-H is detailed in part three. These documents contain virtually all the currently available information regarding what audio engineers will be dealing with in ATSC 3.0. In this column, we’ll look at the structure of the common elements as they relate to AC-4, some of the new metadata parameters, and briefly touch on a few things that remain unanswered.

Fig. 1: In the Next-Generation Audio world, the ATSC A/342 standard treats audio as data.NGA STRUCTURE
Let’s start by looking at the audio data structure of NGA systems. Currently, when prepping a mix for television, audio is the source and audio is the final output, whether it’s a stem, submix, iso, or full mix. In the NGA world, audio is treated more like data, and in fact that’s how A/342 refers to it, as audio data (see Fig. 1). Source audio is now officially referred to as “Audio Signals” and each signal may or may not have metadata associated with it. The Audio Signals and their metadata are formatted into either Channel-Based, Object-Based, or Scene-Based sets called Audio Elements.

Channel-Based Audio Elements are traditional-style, fixed output mixes that can be anything from single-channel mono to immersive 7.1+4. Object-based elements consist of Audio Objects with positional metadata that allow them to be placed—statically or dynamically—almost anywhere in the sound field. Scene-based elements model an actual or simulated sound field.

Audio Elements also carry positional or personalization metadata as well as rendering information that helps the system format audio data for the final receiving device. Groups of Audio Elements make up Audio Program Components which may consist of Complete Main mixes, Music and Effects submixes, Dialog only submixes, Video Description Services, and other audio feeds and mixes which are delivered in one or multiple audio elementary streams.

Finally, single or combinations of Audio Program Components constitute an Audio Presentation, which is the audio portion of an ATSC 3.0 program. There can be more than one Audio Presentation per program but one must be designated as the default presentation. By design this layered structure provides the flexibility necessary to deliver immersive audio, personalized content, and emergency alert information to the listener.

NEW METADATA PARAMETERS
There are several new metadata parameters in addition to extensions of current ones. Control parameters for dialog now include Dialog Enhancement to give users more control of independent and premixed dialog to help improve intelligibility of content after it reaches the home.

Dynamic Range Control (DRC) keeps existing E-AC-3 elementary modes but adds modes that properly render audio to the seemingly limitless variety of target devices such as portable audio systems, flat screen televisions, and home theaters.

Loudness metadata has been expanded to include parameters for true peak and maximum true peak, relative gated loudness and speech gated loudness, dialog gating type, momentary and maximum momentary loudness, and short-term and maximum short-term loudness.

Intelligent Loudness Management (metadata) in AC-4 means the system can now verify whether associated metadata matches audio content and then pass along to decoding devices that the loudness metadata is correct so no changes are made to the final audio. If the system is unable to validate loudness metadata, a real-time loudness leveler can be enabled to ensure loudness standards are met.

Extensible metadata is supported in AC-4, allowing user data, third-party metadata, and application data to also be carried in the bit stream while alternative metadata can be associated with objects, in addition to their regular metadata, to enable presentations to create different versions of the object for their use.

CONTENT DELIVERY
There are two types of presentation streams and two types of decoding modes available in AC-4:

An Advanced Single-Stream presentation enables a single stream to carry multiple Audio Program Components inside it, whether those components are part of the same presentation or from multiple presentations. All mixes, submixes, and versions of a presentation may be carried in the single stream or the stream may be used to carry several different, possibly even unrelated, programs.

Multi-Stream Hybrid presentations send Audio Program Components over multiple paths, with the primary component delivered via broadcast and other components delivered via a secondary path of some sort, of which broadband is an option. Core Decoding delivers a simple, complete audio presentation to target devices—such as phones and televisions—that have simple playback capabilities. Full Decoding mode delivers complete presentations to devices like home theaters that have more complex decoding capabilities.

Fig. 2: This graphic represents one possible option for a 7.14 control room speaker setup. Actual control room speaker layouts have not been specified by the ATSC yet. The documents that make up A/342 give us an amazing amount of information, far beyond what is presented here, but there are still gaps. For instance, proper placement of speakers for 7.1+4 setups for television audio mix rooms remains unspecified and the entire rendering process needs to be clarified (Fig. 2). Fortunately, there are training initiatives taking shape from manufacturers and professional organizations geared to help mixers and technical staff get a handle on NGA and prepare for rollout, which may make this the most challenging and exciting time to be a television mixer since the switch to digital.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology magazine or attransientaudiolabs.com

Fig. 1: Simplified ATSC 3.0 layer model

IP transmission will begin making inroads into North American broadcast facilities starting this spring, even if they have no plans to implement an IP-based infrastructure, thanks to the finalization of ATSC 3.0 specifications and the introduction of compatible consumer products at this year’s CES. ATSC 3.0 is an IP-based transmission standard designed on a five-layer stack (see Fig. 1) akin to the seven-layer OSI network stack that allows easy technology replacement and substitution. This month we’re looking at what needs to be done to prepare existing infrastructure to handle the Next Generation Audio (NGA) formats coming in ATSC 3.0.

The audio system chosen for North American implementations of ATSC 3.0 is Dolby AC-4, which has three Audio Element Formats: channel-based, object-based and scene-based audio. Channel-based audio is essentially what we have now with mono, stereo and surround formats, though with the addition of height channels it also serves as the base for immersive audio mixes. Audio objects consist of audio signals and positional metadata for use in immersive mixes or for audio program customization.

Scene-based audio is a sort of soundfield snapshot from a high-order ambisonic source. All audio components get rendered, then encapsulated, along with video, into an HEVC H.265 stream for broadcast and a synchronized MPEG-DASH stream for broadband. The system is designed so that delivered audio can be played back anywhere, from home theaters to handheld mobile devices to headphones, because decoders adjust playback parameters for the end user’s speaker configurations and devices. This end device rendering may finally mean the end of downmixing.

Controlling all of this and making it function properly requires lots of metadata, so those of us who designed our infrastructure around static metadata will have to rethink and likely rework it.

Some things about the new system are familiar, though with updates. The system sample rate is 48 kHz, with support added for 92 and 192 kHz. Dynamic Range Control remains in the system and loudness management is still LKFS-based. Welcome additions to loudness management include a feature that verifies whether metadata parameters and content measurements match, and there is now an optional leveler.

IMMERSIVE AUDIO
Immersive audio in AC-4 begins with 12 audio channels in a 7.1+4 configuration with speaker locations designated as Left, Center, Right, Left Side and Rear, Right Side and Rear, LFE, Upper Left and Upper Rear Surround, and Upper Right and Upper Rear Surround. Upper channel speakers in this format are placed above their lower channel counterparts to provide height imaging.

Setting up a 5.1 mix room for this configuration requires the addition of six more speakers and will likely require replacement or modification of the current monitor controller. Outfitting a dedicated production control room for 7.1+4 is certainly possible, but doing so in a mobile unit raises a host of concerns including whether any ambience in the height speakers will be distinguishable from the ambience bleeding through the truck walls.

More interesting than immersion, and possibly more challenging, are the uses and potential configurations of object-based audio and customization. One of the most discussed uses for customization is modification of the dialog track to allow swapping of the primary dialog track with one or another language, listening to a secondary commentary track, or turning the dialog level up, down or off.

Other uses include providing assistive services such as descriptive video or audio versions of emergency notifications. Audio objects are also meant to be primary and secondary components in immersive audio mixes. Providing this array of options means being able to create enough mix minuses, submixes and stems, and have enough paths to move them through the facility.

Fig. 2: ATSC 3.0 example broadcast operating profiles

ASSESSING OUR FACILITIES
A look at some suggested ATSC example broadcast operating profiles gives us an idea of the feeds required to provide these services to consumers (see Fig. 2).A 15.1 channel M&E with two dialog tracks and one video descriptive service tosses 19 input elements at the audio encoder just for one stream, 11 more than a 5.1+2 mix.

ATSC 3.0 allows multiple simultaneous streams, so the element count could get quite high, depending on what profile is used for each stream.

This information helps us assess whether current production facility technologies will handle the workload, and, in fact, most remain useful to some degree. All infrastructure paths still seem valid with AES the most limited and embedded SDI good for feeds up to 16 channels wide. MADI, with 56–64 channels per link, should be sufficient for most current broadcast productions, but moving beyond 64 channels in one path means moving to some form of AoIP.

Speakers may need to be added and metering will need to be updated for immersive mixing and monitoring, not just in the audio mix rooms, but also in editorial rooms and QC stations. QC audio monitoring may be awkward given the potential output options available, so it may be the time to consider building a separate audio QC room.

Monitor controllers will need to be assessed to see if they can handle the mix and submix options, and room size and room acoustics should be reassessed once speakers are added for immersion. Digital audio consoles with internal routers may be the Swiss Army knife for solving signal flow and feed issues for production since they can be outfitted to directly interface with most types of facility I/O, including AoIP, though additional DSP processing and I/O may need to be added to the console system, and the monitor section will need to be updated to monitor objects and immersive mixes.

The most challenging part to making all of this work may be proper metadata authoring, control and QC. AC-4 is highly metadata-dependent and it seems unlikely we’ll simply be able to provide a static metadata product to the consumer, certainly not if other broadcasters deliver the value-added product this new system can deliver.

Of course, there is currently no pressure for any facility to make drastic or wholesale changes right away or move to full immersive audio with multiple languages out of the gate, but now is certainly the time to start planning.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology or attransientaudiolabs.com.

At the 2016 Audio Engineering Society Convention in Los Angeles, I ran into a friend who was manning the booth of a broadcast audio console manufacturer. I had finally managed to get to the show floor for a quick walkthrough of the exhibits before heading back to another meeting room. He expressed consternation about why they were there since it wasn’t a broadcast show and few attendees would be looking for a high-quality digital audio console.

Along with seeing demos of new products, the AES Convention has become an opportunity to dive into the technology of what we do and see how it is being used. This struck me as odd since there were many years where that was exactly why I went to the show. Manufacturers of larger audio products now tend to stick to broadcast- focused shows since recording and post engineers have gravitated to working inside the box with emulations of hardware, including features from audio consoles.

As I pondered the implication of my friend’s words—that the convention dedicated solely to audio is no longer the place for audio consoles with amazing sound and incredible technology—I realized that the dynamic of the convention has changed for me as well. It’s no longer primarily about getting to play with fantastic new audio products, but it has become an opportunity to dive into the technology of what we do and see how it is being used.

AUDIO FOR VR

Merely judging the show by exhibit space alone gives an incorrect impression of what is going on at the show because the technical program of workshops, papers, tutorials and technical tours were incredibly busy. In addition, the AES Audio for Virtual and Augmented Reality (AVAR) technical conference was held in conjunction with the convention this year, which meant some of us were standing around with boxy, modern day Viewmasters on our faces.

VR and immersive audio seem to be exciting people and offering opportunities in television, film and gaming that may not have previously been on their radar. VR, while an interesting technology, is not one I’ve seen much practical value in until this show when I ran across a small company called Audio Fusion who deservedly won the AES Silver Award in the student design competition.

Audio Fusion has created a virtual studio training environment by modeling an analog recording studio to provide hands-on training for those who don’t have access to an actual studio. Trainees use headsets, headphones and custom controllers to manipulate audio consoles, patchbays and other studio equipment.

NEXT-GENERATION AUDIO

The status and production of immersive and object-based audio, now collectively referred to as next-generation audio (NGA), was discussed in several sessions.

Dolby AC-4 has been selected as the next-generation audio format for the United States, while MPEG-H will be used in the country with the most aggressive timeline for ATSC 3.0 rollout, South Korea. This is the first broadcast format designed to deliver broadcast content to devices of all types, not just to televisions, including new devices as they arrive on the scene.

Mobile devices in particular bring with them an array of level and dynamic range problems due to the nearly limitless number of viewing environments they could be used in. AC-4 will manage these mobile environments as well as static listening environments, by utilizing multiple metadata profiles and rendering audio at each device. MPEG-H proponents remain skeptical as to whether the metadata in AC-4 can survive the distribution process despite the benefits of audio control and interface customization.

Audio Fusion has created a virtual studio training environment by modeling an analog recording studio to provide hands-on training.AUDIO FOR OTT

At last year’s convention, the AES Technical Committee on Transmission and Broadcasting published their technical document for audio music streaming and this year their subcommittee, AGOTTVS, has released technical document AES TD1005.1.16-09, which covers loudness for OTT streaming. “Audio Guidelines for Over-the-Top Television and Video Streaming” provides more than the lengthy acronym for the subcommittee; it also provides initial loudness recommendations for a problem-fraught delivery medium.

There are four recommended practices in the document: the use of agile or static metadata when devices are full-range and distribution is able to support it; a list of how to handle content when the distribution system does not have metadata capabilities along with a recommended loudness setting of –16 LKFS for devices with limited dynamic range; the recommendation that all loudness implementations be tested for anomalies; and recommendations for versioning of the same material with metadata encoded versions left at full range and reducing the dynamic range of versions without metadata.

AGOTTVS is made up of broadcasters, manufacturers and streaming companies, and is headed by NBC’s Jim Starzynski, who helped forge the A/85 loudness recommendations that form the core of the CALM Act. It is a testament to this group that they have been successful in engaging some of the streaming providers in the standards process. Hopefully all of them will get involved as the group’s work develops into a standard.

AUDIO-OVER-IP

Sessions on AES67 drilled down into this maturing technology, covering some large deployments of it in real-world live events. One important point that was brought up, and one I’m not sure I’ve stressed enough, is that AES67 is not a competitor to other AoIP technologies, but is meant to help them all work together.

Video is certainly a big part of the broadcast world, but it was not originally mentioned in AES67 despite the possibility that it could be included later. AES67 has now been adopted by the Joint Taskforce on Networked Media (JTNM) in its Video Services Forum (VSF) technical recommendations TR- 03 and TR-04 for inclusion into the upcoming SMPTE ST-2110 standard. This means that manufacturers adhering to AES67 technical recommendations now have access to the television market as long as they also adhere to the Networked Media Open Specifications (NMOS) from the AMWA.

There were many other interesting sessions this year, and we’ve only scratched the surface of the few covered here, but we’ve run out of space. December is traditionally a time for gift giving and making resolutions, so I encourage you to consider giving the gift of education in some form this year because it truly is the gift that keeps on giving. Keep on learning!

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology or attransientaudiolabs.com.

The entire process of sound for film has always fascinated me partly because the working environment seems so extravagant compared to someone who has spent the majority of their professional life working in the trenches of broadcast audio for television.

The mystique perpetrated by cinema audio is that the budgets are enormous, audio is recorded on gigantic sound stages, mixed in huge mix rooms and staffed by large specialized crews; and for studio-backed widerelease films, these generalizations are fairly accurate.

Broadcast audio engineers have a totally different working experience. In broadcast audio post, a single person may end up handling the music and dialog editing, sound design and final mix; whereas in film each of these is handled by a different person. Some broadcast facilities have relatively large studios, but the budgets and crews are much smaller than those in cinema, and audio control rooms are sometimes no larger than the closet we’re able to wrest away from the video guys.

A Sony mix stageFRESH LOOK

In September of this year TV Technology sister publication Mix magazine held the third annual “Mix Presents Sound for Film and Television” event on the Sony Entertainment lot in Culver City, Calif., and it seemed like an excellent opportunity to get a fresh look at the world of cinema audio. I was particularly interested in seeing how much immersive audio work was being done and what impact the proliferation of OTT shows is having.

As soon as I arrived on the studio lot it was apparent that production for streaming has made its way to Hollywood because billboards for OTT shows are prominently displayed alongside those for broadcast networks and feature films.

The event itself made for a very full day, beginning with the keynote from Gary Bourgeois at 9:30 a.m. and finishing up 12 hours later after the screening of 13 eight-minute film segments, each introduced by the sound crew or another person involved with the film. There was a mini-trade show, sound cart demonstrations and technology master classes, but the most compelling parts of the event were the panel discussions: “Sound Inspiration Within the Storytelling Process,” “Sound Mixing Panel: Workflow for Musicals” and “Building an Immersive Room: Small, Medium and Large.” The panelists for these sessions were excellent and the audio podcasts are now available online at http://soundworkscollection.com/.

Of particular interest was a session titled, “Remastering Cinema Audio for ATSC HDTV,” led by Lon Neumann, which was, at its heart, a review of ATSC A/85. It seems the film community is still coming to grips with the CALM Act and translating cinematic- style mixes into one that plays well in the home. Neumann introduced a couple of terms I had not encountered in reference to sound for picture: “allocentric” and “egocentric,” but they perfectly sum up the differences between cinema and television.

The Sony Entertainment lot Seeing a film in the theater means participating in an allocentric event specifically designed to be shared with others, with speakers distributed around the room for wide dispersion, where everyone in the theater shares a similar experience.

Television on the other hand—especially with the increasing consumption of content on personal media devices— is a highly egocentric medium where everything, including speakers, is focused to one spot in the room creating an experience best enjoyed by one or a few people.

Creating content for these two opposing methodologies necessarily dictates mix rooms with vastly different designs and speaker layouts in order to emulate the end-user experience, which additionally complicates the translation of material between the film and television mediums.

It turns out there is now so much television, streaming and game work being done in L.A. that mix rooms are in short supply, so new ones are being built by studios as well as by mixers who have opened their own facilities. Most of these new rooms are built primarily for television and game work so the rooms being built tend to be smaller. This presents challenges for mixers who move between large and small mix rooms because they not only have to adjust to completely different spatial environments, the rooms tend to have less than ideal dimensions so low frequency build-up must be dealt with, and the smaller rooms require mixing at lower monitor levels.

Even these smaller rooms are larger than most broadcast audio mix rooms, but film sound mixers are starting to get a taste of the issues the rest of us deal with on a daily basis.

ADDITIONAL COMPLICATIONS

The Sound Inspiration Panel, from left: Carolyn Giardina (moderator), Will Files, Mark Stoeckinger, Ben Cook, Paula Fairfield, Harry Coen, Paul Menichini

One enormous additional complication presented by these smaller mix rooms when compared to broadcast mix rooms is that the majority are built for immersive audio production. That means not just more speakers, but additional acoustic design complications. The panel on building immersive rooms stressed how important it is to make the rooms deader than they would for a standard film mix room, including deadening the ceiling surfaces, to help with localization of sound sources. Every mixer who spoke about immersive audio spoke effusively about how much they enjoy working in it and how it is becoming a regular part of their workflow for film, episodic television and gaming. The general consensus among the mixers is that it’s here to stay.

Another aspect of film sound that has changed is the jobs themselves. Thanks to the smaller budgets of today’s television shows, the sheer amount of work to be done and the workflow efficiencies gained by being able to work completely in the box, the lines are starting to blur and individuals on sound crews are more often filling multiple roles.

It was a pleasure to see how OTT and gaming has reinvigorated the film sound community, but the most encouraging thing was the large number of young people attending this event who are working in the industry—something the broadcast industry seems unable to replicate on a large scale. However, if we’re lucky, the abundance of interesting content, gaming, immersive audio and fresh faces will trickle down from studio lots to the wider broadcast community.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE and TAB. He can be contacted throughTV Technologyor attransientaudiolabs.com.

Anyone who has been an audio engineer for a substantial period of time does so because they love what they do, not because they can’t move on to something else. Being an audio engineer in the world of broadcast means that we also have to deal with all that pesky video equipment and the signals that accompany it.

A depth of video knowledge may actually be the broadcast audio engineer’s least-known skill since, just to get our jobs done, we often end up learning more about video than anyone realizes. Still it can be tempting to just stick to the basics of what we need to know and end up puzzled by some of the stuff we run across.

This month we’re going to take a look at some video terminology that we see, but may not be familiar with. Let’s start out with a general overview of garden variety video connections: plain old analog video, as composite and component, standard-definition video (SD), and high-definition video (HD).

Video starts out with individual red (R), green (G) and blue (B) channels, which, depending on the application, either remain individual full bandwidth signals or get matrixed into a full-bandwidth luminance channel (Y); and two lower bandwidth color difference channels (B-Y, R-Y), which minimizes loads on video processors.

An array of video connectors with composite as CVBSCOMPOSITE AND COMPONENT
Composite video combines the three channels onto one cable while component leaves them separate. Composite connections are usually just called video, though I recently ran across a device that labeled it CVBS (color, video, blanking and sync). Component connections are most often designated Y/G, Pb/B, Pr/R and may or may not be labeled component. The simplest way to think of SD and HD video is that they are 8- or 10-bit digital versions of composite video that each move down one Serial Digital Interface (SDI) cable typically terminated in BNCs, so even the cabling is similar to composite analog.

The superpower of SDI video is that it can carry 16 channels of embedded audio along with it. Native SD video is always in the 4:3 aspect ratio with an interlaced frame size of 720x480 at 270 Mbps. Native HD is in 16:9 widescreen video format and is most often seen in progressive frame sizes 1280x720 (720p) or 1920x1080 (1080p) at 1.485 Gbps.

Interlaced video places odd lines on the display first, followed immediately by the even lines, while progressive scan video displays all lines in order. As with audio, connecting digital and analog video signals together requires conversion.

Another video format that needs addressing here is the high-definition multimedia interface (HDMI) as its ubiquity has allowed it to creep out of the consumer space into professional facilities. HDMI passes high-definition video and audio between devices on one cable, but quality can be hampered by poor quality cables and transmission hindered by high-bandwidth digital content protection (HDCP).

Content-encrypted with HDCP will not display on devices without an HDCP license, or those it considers unlicensed. Unfortunately, nonsensical failures are regular occurrences due to this embedded copy protection so HDMI should be used sparingly and always be tested.

Video has, of course, moved past HD to 4K and higher resolutions, with high dynamic range the current flavor of the day. 4K, which is being marketed as “ultra-high definition,” offers a minimum of four times the pixels of HD. Actually, UHD displays reproduce 3840x2160 lines to maintain a 16:9 aspect ratio, while video capture and image creation will likely be done in the actual 4K resolution of 4092x2160 lines. Higher resolutions and bit depths require infrastructure upgrades to beyond 3 Gbps and, depending on the codec, potentially more data storage space.

Uncompressed 1080p HD at 59.95 fps consumes disk space at approximately 1.53 TB per hour, whereas uncompressed 2160p 4K video would take up approximately 5 TB per hour. These storage requirements almost certainly mean that most 4K content will be shot and stored in some compressed format.

THE PROBLEM IS...
A bigger problem is that video carrying this much data cannot be moved around within most infrastructure as it currently exists. There was talk of a new multi-cable standard for 4K early on, but, as with most other technologies in the plant, IP distribution seems to be the way forward. HDR technology, which increases contrast and expands colors available in images, is being touted as a way to improve picture quality without increasing resolution. I’ve seen HDR in cinema demonstrations and it is very impressive, with incendiary whites and seemingly bottomless blacks and tons of detail where it was previously missing.

Every once in a while I run across a device with an asynchronous serial interface (ASI) connection on the back, also called a digital video broadcasting-asynchronous serial interface (DVB-ASI). This is a one-way data link for streaming compressed video and audio between digital devices. An ASI output on one device connects to the ASI input on another and, since it is a compressed data stream, it is not compatible with SDI connections.

Finally, there are the video sampling formats 4:4:4, 4:2:2, 4:1:1 and their seemingly ceaseless variations. This is actually related to our earlier coverage of luminance and RGB. The first number in each of these trios refers to luminance, while the other two are chrominance.

For instance, a 4:4:4 signal has the luminance (Y), red minus luminance (R–Y), and blue minus luminance (B–Y) channels each sampled four times, while 4:2:2 samples luminance four times, and drops sampling of each chrominance channel to two times. Following this pattern, it is easy to figure out the other sampling formats, though if you encounter one with a fourth digit (i.e. 4:4:4:4) then a key channel has been added.

Luminance sampling must remain high because we are far more sensitive to light than we are to color, so color sampling can be reduced with little concern. You may have noticed that all matrixed RGB lacks a green channel, which is not necessary since green can be derived from information in the other luminance channels.

Jay Yeary is a broadcast engineer and consultant who specializes in audio and now wonders if video did indeed kill the radio star. He is an AES Fellow and a member of SBE, SMPTE and TAB. He can be contacted through TV Technology or attransientaudiolabs.com.

Comrex BRIC-Link II IP audio codec

In my August column, “The Audio World Without ISDN,” we looked at the impending demise of ISDN and the options available for those who can no longer get service. With telcos pushing everything into the packet-switched world, it certainly appears that any solution to this dilemma will be IP-centered.

Manufacturers of ISDN hardware and IP software obviously want us to move in that direction since they already have solutions available, and, in many instances, the hardware is designed to drop into the studio as a direct replacement for an ISDN codec.

Whether we move from ISDN to IP is no longer a question but more a matter of time, so we need to examine concerns about how to make the transition smoothly.

YOU WANT IT WHEN?

Reliability seems to be the greatest concern when it comes to using the internet to provide audio services. ISDN provides nearly rock-solid reliability with uptime the norm, not exactly how most of us would describe our experience with internet connections.

There is little we can do if the network goes offline, but choosing the most reliable provider who can deliver sufficient speed and bandwidth for our needs is the first step, though anyone in a large organization will likely have no choice in the matter.

When shopping for bandwidth, just about any modern urban internet plan will suffice because audio data is relatively small in terms of network traffic. A peek at some of the larger internet providers shows that the current slowest business-plan download speed is 10 Mbps, which is more than sufficient for receive data. However, the upload speed for that plan is just 1 Mbps, which gives no headroom for outgoing audio data if the network is performing subpar that day, so a plan with more upload speed, say 5 Mbps, makes more sense.

Anyone who wants a really robust connection has the option of leasing a dedicated network line between the provider and the studio. It is extremely important to make sure the provider doesn’t have data caps that will shut down your service if you’ve done too many sessions and exceeded your cap.

Fortunately, now that Google is expanding its fiber network, subsequently inspiring previously hesitant telcos to do the same, faster network upload and download speeds should be on the horizon for most everyone.

PACKET LOSS

A source-connect now IP audio session in progress There are a number of technologies built into IP audio codecs to ensure they are robust enough for professional use. Remember that our audio data is being broken into packets, then sent across the internet to a specific destination where they are reassembled in order, despite the fact they may have taken vastly different paths to get there. Due to the fragmented nature of the network there is a high probability that jitter will occur because packets rarely arrive in order, but this is dealt with by introducing a buffer delay prior to reassembly.

There is also the possibility, dependent on a host of factors, that some packets won’t arrive at all, resulting in packet loss. Buffering introduces latency in the path, which can be a problem for live audio sessions; and packet loss is what everyone dreads because it means some of your audio data has gone missing. In reality neither of these are significant problems in IP audio streams because all the codecs use some form of data compression to minimize demands placed on the network while they actively manage buffers to minimize delay.

Virtually all IP codecs use Forward Error Correction (FEC), which puts duplicates of packets inside adjacent packets so the data can be reproduced even if a packet, or a string of packets, is lost.

For larger organizations there can be problems using the business LAN to send data, though the problems are not nearly as prevalent with audio as with video. It is possible to have the IT department prioritize traffic using Quality of Service (QOS) tools within the router. The bigger issue when passing media data on business LANs tends to be with firewall rules that limit traffic or close ports.

These issues can usually be handled by working with the internal IT department, but be aware that sometimes updates are pushed out and device refreshes are done by IT that may not include the changes requested for audio traffic, so it’s important that they include those changes in their base firewall and router configurations.

PRACTICAL SOLUTIONS

At the practical end of the spectrum is how these things work in everyday use. While I rarely cross paths with ISDN these days, I do spend a lot of time using IP devices for television production. I hope the following example gives some idea of how solid IP audio is.

For a weekly news show, we use two Comrex BRIC-Link II bidirectional point-to-point IP audio codecs using AAC/HE-AAC coding for IFB and backup audio, as well as two Haivision Makito video and audio units passing 1920x1080i video and MPEG-4 48 kHz audio at an average bandwidth of 6400 Kbps for all traffic.

The BRIC-Link IP audio codecs have been rock solid with no loss of audio even though data passes through two networks and myriad firewalls that hide the networks from the outside world. The Makito makes good-looking video, despite the low bit-rate. We occasionally experience blocking in the image, but never have any loss of audio.

The final choice here is whether to choose a hardware or software codec, and this decision likely comes down to what the destination connection will be. Either prefer the same or similar codec at each end though there is some flexibility and bridging is always available. For studios with control over both ends of the chain, a hardware codec may make more sense. Radio and remote television broadcasts are perfect places to use hardware codecs due to their plug-in simplicity. For voice actors and smaller studios, the choice is not so clear and software codecs may offer more flexibility. Regardless, the world without ISDN is here and everything will be fine.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE and TAB. He can be contacted through TV Technology or attransientaudiolabs.com.

I was working on a string of projects back in the mid-1990s that required us to fly in talent to handle Spanish and Portuguese translation. We searched for ways to make the work more efficient and discovered that a technology called Integrated Services Digital Network (ISDN) was being used by producers to record musicians from all over the world and by movie studios for remote ADR. It offered high-quality audio and latency low enough that conversations between the studio and remote artist felt completely natural.

We hopped on board with one of the early black-front Telos Zephyrs, which was compatible with practically everything and a massive APT WorldNet unit that only talked to other APT units, and the codecs went into service doing remote voiceover sessions and live event announcing.

Voiceover artists (now voice actors) saw the potential of the technology right away, so many of them built their own ISDN-equipped studios and offered their services globally, and in the process, became the original “work-from-home” pioneers of the broadcast industry.

RUMOR MILL

Software IP codec comparisons
Click To Enlarge ISDN technology has served broadcast professionals faithfully for nearly 30 years, but rumors have been circulating for some time that the telcos plan to stop providing it because the service is not profitable enough. Up to this point, it has remained just a rumor, but it now appears to be very real.

A voice actor friend is in the process of relocating his studio within the same New York City suburb. He has been told by Verizon, his long-time ISDN provider, that they will not reinstall his service once he leaves his present location, despite the fact that he has paid them thousands of dollars annually for decades for the privilege of having those lines. Unfortunately, no other provider in his area offers the service, so we have been looking at alternatives.

It makes sense that the telcos want to kill off ISDN just as they want to kill off the analog home telephone line. Both are remnants of the circuit-switched public telephone network, which makes inefficient use of the telco network compared to packet switching.

Circuit switching establishes individual point-to-point connections, switching both ends at the same time, and creating a dedicated connection between those two points. Packet switching isn’t interested in making dedicated point-to-point connections; it breaks data up into packets that are sent off on what could be completely different paths, to get reassembled at the far end. Packet switching can also send those packets to multiple destinations at the same time.

One alternative to a dedicated ISDN line is the Telos Zephyr Xport, which connects to a traditional phone line or the Zephyr Xstream ISDN codec. The ISDN lines themselves are copper-twisted pairs that carry digital data instead of analog and there are two variants. The type normally used with audio ISDN codecs is a Basic Rate Interface (BRI), which consists of two 56 kbps bearer channels and a 16-kbps channel for data signaling.

Some codecs gang BRI lines together to improve quality, reduce latency and pass bidirectional timecode and control information. Each bearer channel is connected to the telco through a network interface (NT- 1); and each is assigned a Service Profile Identifier (SPID), which tells the network who it is. The Primary Rate Interface (PRI) version is essentially a full T-1 line carrying one 64-kbps signaling channel and 23 64- kbps bearer channels.

The monthly fee for a PRI is significantly more than a BRI, which is why it’s normally only installed to provide Private Branch Exchanges (PBX) to businesses. PRI, unlike BRI, doesn’t appear to be on the chopping block so far.

AVAILABLE OPTIONS

For those facing the loss of ISDN service, there are a number of options to consider with IP audio codecs seeming the obvious choice. Telos makes an intriguing device called a Zephyr Xport that connects to a traditional phone line, yet it can also connect to their Zephyr XStream ISDN codec. Unfortunately, since it only connects to that particular codec it’s only useful if the studio at the far end has an XStream with the proper version of software installed.

Source-Connect was the first software alternative that directly competed with ISDN, though it has since been joined by SoundStreak, ipDTL and Source-Connect Now. Each requires a computer, network connection and the same software at each end. They are differentiated primarily by their pricing models:

• Source-Connect offers three purchase options and a subscription;
• SoundStreak doesn’t charge talent, but does charge production;
• ipDTL has an array of pricing options, and
• Source-ConnectNOW is free with premium options available.

Most ISDN users are familiar with bridging services, even if they’ve never used one. These services started as a way for users with one type of ISDN codec to connect to an incompatible ISDN codec at the far end. The bridging site establishes connections to the origination and destination codecs using in-house-compatible units, and passes audio between the units. These services now offer bridging from software codecs to ISDN codecs, especially appealing to anyone without ISDN service. Bridging services include industry stalwarts DigiFo – n and EDNet, ISDN To-Go, ISDNBridge and Out of Hear.

There is one way to continue ISDN service without telco support, but it’s only feasible for those who work larger businesses: Generate the ISDN from the business PBX. IDSN lines created in this way work just like any other ISDN line with the exception that they need to dial out of the business network to place calls.

Choosing a technology replacement for ISDN is not easy, with client requirements always the key factor. Studios currently using ISDN should be able to continue using it for the foreseeable future, which means they will continue to require ISDN sessions from voice actors. However, once a studio decides to relocate, they may face this same dilemma. Next time we’ll take a look at making the transition from ISDN to IP codecs.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE, and TAB. He can be contacted through TV Technology or attransientaudiolabs.com.

Click on the Image to Enlarge
KEEP IN MIND
Before proceeding, there are a number of items to keep in mind. First is that AES67 could be considered the lowest common denominator for successfully sending audio across a network using IP. Many pieces required to provide a complete AoIP solution— most notably discovery, management and control—are not included in AES67 and were never intended to be. Click on the Image to Enlarge

One of my missions at this year’s NAB Show was to take a deeper assessment of the state of interoperability of audio-over-IP devices in preparation for an upcoming project. I was specifically interested to see whether a complete system could be built using disparate components from different manufacturers, utilizing whatever AoIP technology had been chosen for the product.

The project is for a broadcast facility audio control room with no significant connectivity to the video router, so the key pieces are the digital audio console, intercom system, digital snakes and peripheral equipment. This system needs be configurable in minimal time with minimal hassle. All components must be guaranteed to work together and need to be as interoperable as possible, which means complying closely with AES67.

As a longtime proponent of AoIP it was my opportunity to take a closer look at what could be assembled into a functional AoIP system.

Fig. 1: OSI layers

It is also important to remember that the internet protocol (IP) can be carried over a variety of transmission paths, including Ethernet, ISDN, ATM, FireWire and DSL, making it quite versatile.

The scope of the network itself should be kept in mind, whether it will be a small internal network or one that expands outside the boundaries of a given subnet, building, campus or continent. AoIP equipment working at OSI Layer 2 will suffice for small internal networks, but any AoIP network that will roam farther afield will need to operate at Layer 3.

Finally, and most important for us, the audio settings of the technologies must match.

My mission didn’t get off to a particularly auspicious start, even before the NAB Show was underway. At the DTV Audio Group meeting the day prior to the show, Thomas Edwards, vice president of engineering at Fox, presented findings from his experiments with AoIP interoperability, concluding that interoperability wasn’t working, primarily due to transport differences.

Fortunately, he didn’t leave things there as he went on to elaborate that this situation was just temporary as video-over-IP standards were beginning to gel around SMPTE 2022 and with AES67 as the likely audio transport.

Later on in the day, Tobias Kronnenwett of Lawo, talked about how they were having continual success with AES67-based AoIP technologies on a large scale with Moto GP races, proving that the technology itself is proven in real-world use.

As I started my own examination of each AoIP technology I made the disheartening discovery that simply claiming compatibility with AES67 does not necessarily indicate that the technology is interoperable.

Table 1: AoIP compatibility matrix

In the case of Dante, upgrades made to their devices means that Dante and AES67 streams can coexist on the same network, which is about the same as having roommates— you share the same space, but that doesn’t mean you get along. In Dante’s case, they legitimately want to integrate AES67 streams, but may be hesitant to modify their robust and very successful core technology just to play nice. The players who do claim full compliance with AES67 include Ravenna and Telos, but each have their own methods for discovery, control and management to fill in the gaps.

TECHNOLOGY CHOICES
When we get down to the AoIP equipment itself we find most manufacturers making definitive technology choices, with Dante appearing to be most popular. Among large intercom systems, Delec and RTS/Omneo have chosen Dante, while Riedel has announced support for Ravenna with their Tango intercom to complement their longstanding support of AVB.

Taking a different tack, intercom vendor Clear-Com has announced their own technology called Instant Voice Core (I.V.Core) which claims AES67 compliance and which, perhaps providing the “Rosetta Stone” of the intercom world, purports to have also been tested for compatibility with AVB, Dante and Ravenna.

The world of audio console players doesn’t provide any real surprises with all the name players getting into the game, though in decidedly different ways. Digico consoles and racks can be outfitted with modules that provide Dante AoIP connectivity.

Calrec has very diplomatically decided the best way to accommodate customers is by letting them have the AoIP technology of their choosing, whether it be AES67/Ravenna, AVB, Dante or SMPTE 2022-6.

Ravenna’s parent company Lawo offers AoIP options across a large portion of their products including consoles, routers, racks and interfaces.

Dante solutions have been arriving from SSL for years and, at the 2016 NAB Show, they were showing their System T high-end, large-format broadcast console.

Wheatstone offers AoIP in five of their audio consoles and other products through Wheatnet, their AES67-compatible solution.

Finally, audio industry stalwart Yamaha offers Dante console cards and remote rack units.

Unsurprisingly, no single manufacturer makes all the equipment necessary to build an entire broadcast audio system. Very few have anything close to an entire AoIP ecosystem within their own product lines, something I find more than a little disappointing since it leaves us using an awful lot of glue to stick onto non-AoIP equipment.

The bright spot is that AoIP works and works well if all components use the same base technology. It can also be successful to a lesser degree with equipment from different manufacturers, but complete compatibility can only be guaranteed when all equipment is using the same AoIP technology.

An all-Dante system will obviously work, as will an all-Ravenna system, and any technology operating in AES67 mode will pass audio to other AES67 devices, but the method that manufacturers utilize for discovery, control and device management may mean that the components won’t know about, recognize or pass control to each other.

The challenge for my project at this stage is to choose a base technology that allows everything to work seamlessly while remaining open enough to accept other technologies as AoIP matures.

Jay Yeary is a broadcast engineer and consultant who specializes in audio. He is an AES Fellow and a member of SBE, SMPTE and TAB. He can be contacted through TV Technology or attransientaudiolabs.com.