Immersive sound is the final journey to the complete auditory experience. Stereo began that journey with the ability to reproduce sounds that would align more with our hearing, but at the onset, audio practitioners seemed to define stereo as two channels of monophonic sound.

I will always remember my first records that separated the instruments into one channel or the other—this did not work for television. When you saw music on television, the instruments were not separated into left or right, but more of a mix in the middle. This may have been because early mixing decks did not have panorama controls.

The Problem With Surround Sound
Once audio technicians ventured into two channels of sound, the problems began immediately with phasing and even to the extreme of phase cancellation. With surround sound, this became even more problematic. Stereo and surround sound cracked the door open for the complete auditory experience, while the next step, immersive sound, blew the door off its hinges with creative possibilities while solving many technical issues.

For the audio producer/broadcaster, immersive sound adds an upper layer of sound over the basic surround-sound bottom layer. Technically, when you propagate the upper channels with sound, you have immersive sound.

Immersive sound is easy to produce—begin with atmospheric enhancements so the listener further believes the “being there” experience. The ambience at sports venues is fairly homogenous and omnidirectional, particularly in the upper stratus, so to capture a stable immersive sound base will require a minimum of four microphones spaced some distance apart and some distance from the source.

There is much discussion of the separation of “spaced pairs” of microphones. I happen to like a widely spaced group of microphones because I like a broad sense of dimension—my experience has been that precise placement is not absolute in sports.

With immersive sound, you do not want to over-mix the ambiance/atmosphere into the lower channel—use all of your channels to separate your sound elements and be careful not to drown out the commentators or other voices with ambiance and atmosphere.

Evolving Sounds of the Game
Immersive sound will evolve as the practitioners and producers gain knowledge and experience. There are sports that are covered with the athlete in full frame from head to toe, which lends itself to specific sounds in the upper front channels. For example with basketball, the net microphones can be placed in the upper front channels, giving a natural soundscape, because the listener does expect that the sound of the basket and net are above their head.

It is often assumed that if you cannot properly hear the sound field, you cannot properly control the sound field. This is partially true, but consider that somewhere in a quality-control or transmission room the entire immersive sound field should be verified, but for simple atmospheric enhancement visual metering is a valuable asset to the mixer.

Critical listening of a transmission mix may dictate your level of sophistication in the immersive mix. For example, basic atmospheric enhancements can be metered and monitored in a QC listening room with a soundbar to determine proper levels of sound and balance to the immersive sound encoder.

In a space without overhead speakers, you can hang a couple of overhead speakers or—depending on budgets and time—a temporary immersive mix area can be assembled for the event. Specifically, any event where “flight pack” equipment is set up, you should be able to configure speakers to accommodate Immersive Sound production.

Digital Advances
Immersive sound was not possible before digitized audio. Additionally, digitizing the audio and video signal solved the problem of transporting the broadcast signal to the consumer. The introduction of ATSC 3.0 audio standards for immersive sound with Dolby Atmos and its competitor, Fraunhofer’s MPEG-H, along with the proliferation of modestly priced encoding “black boxes” like the Liner Acoustics brands, has allowed for immersive sound to be encoded on virtually every electronic entertainment production.

Dolby Atmos and MPEG-H decode immersive sound into the home over speakers, soundbars and ear devices and either codec is capable of decoding virtually any listening configuration—not only immersive but surround and stereo, solving the decades-long concern over “the down mix.”

Entertainment formats and technologies are consumer-driven—no matter how cool the industry thinks a format is, the consumer may think otherwise. The problem for multichannel sound has always been delivering the experience to the home. The early analog “matrixes” from Dolby sounded mediocre at best, but opened ears to the possibilities of the enhanced audio experience.

Fast-forward to today and soundbars with audio decoders are the “go to” audio device for the home consumer. The consumer/listener can tell the difference with even the most basic of sound reproduction devices, and enhanced sound clearly stands out with virtually all content—drama, music, variety and sports. Finally, soundbars are significantly easier to install and set up; ask my mom.

My advice has always been, it is never going to be perfect! It only needs to be entertaining and immersive sound is entertaining.

LOS ANGELES, Calif.—Sonovo started in Norway in 2003 as a music mastering studio and later branched out into post production and music production, primarily for the surround and immersive audio market. After moving to Los Angeles in 2018 to establish an additional studio, Sonovo now serves clients on both sides of the Atlantic and around the world, including Norwegian art-rock band Gazpacho and their recent “Fireworking at St. Croix” performance.

With the live stream complete and the recording slated for release this fall, I was tasked with creating 5.1 surround versions of the stereo tracks for a Blu-ray of the live concert footage, as well as delivering LP versions, streaming files and CD. Additionally, I performed dialogue clean-up for the interviews and mixed the bonus material.

IN SYNC WITH DAVINCI

Having implemented DaVinci Resolve Studio and working with its Fairlight audio post page and a full suite of Fairlight panels, I completed editing, final mixing and delivery of the project, working from the stereo masters and stems provided by the band. 

DaVinci Resolve Studio’s busses allow multiple delivery targets on the same timeline, all in sync and with metering to match levels (stereo and 5.1). Having the stereo versions on their own bus in DaVinci Resolve Studio provided an easy reference to match the timing, EQ and loudness, as well as preserve the feeling of the originals, while keeping everything in sync with the picture from the videographer. 

Having separate timelines for different parts of the project, including dialogue edit, final dialogue mix, Blu-ray music mixes, bonus content mixes, etc., helped keep the project structured and allowed me to try different approaches and settings, all easily accessible up to final approval. 

Additionally, I appreciated having the ability to crossfade between the track layers, as well as having clip- and track-based effects, which allowed global and individual adjustments to get everything to sit right.

I also tackled dialogue editing and clean-up of the bonus material, which included interviews of the band and the cover art artist, as well as alternate mixes of some of the songs. I matched ambiance and EQ and brought in room tone to provide a consistent experience across the extra content.

A TOUCH OF A BUTTON

This project’s workflow was made so much easier with the Fairlight Desktop Audio Editor. With one button press I could accomplish complex tasks that would require a lot of mousing around otherwise, saving valuable time. For example, flying clips around and performing slips, trims and adding room tone to a range was easy and effective. The jog wheel was key here, and the programmable macros allowed me to set up multistep workflows that could be initiated at the touch of a button. 

Mixing is more intuitive and alive when using a console with faders, allowing me to “play” the mix as it progresses and capture it via automation. I think it engages my brain differently when working with a control surface compared to using a mouse and keyboard. Having knobs and buttons to grab and dial while the mix is playing means I’m not interrupting my flow, which allows greater focus on the creative aspects. It might sound like a trivial improvement, but it is significant, especially when working with larger projects with higher track and bus counts. 

For additional information, contact Blackmagic Design at 408-954-0500 or visit www.blackmagicdesign.com.  

TOKYO—The broadcast production of the Olympic Games is mostly a background operation provided by the Olympic Committee’s broadcast operation Olympic Broadcast Services. Their mission is to provide comprehensive coverage of all sports, of all athletes and ceremonies to global rights holders such as NBC in the United States.

OBS is also known as the Host Broadcaster and provides complete coverage including audio, video, graphics and timing/scoring—essentially providing a full broadcast production to their clients, the Rights Holder. Rights holders will then overlay their local identity on top of the Host Broadcaster production and to their home viewers it would appear to be a custom production by their network provider. In the U.S., it looks like NBC does all the work.

BEHIND THE SCENES

In a normal Olympic year, the hustle and bustle during the month of May would signal that there are about 60 days to Opening Ceremonies of the next Summer Games. Many of the vendors have arrived; the army of skilled crews have been preparing the broadcast center and venues for sports and television production; and boatloads of equipment have been arriving at a frantic pace.

Often the behind-the-scene preparation that goes into an Olympic production is unknown to the public and even to many television engineers who have never experienced such a vast undertaking from the inside.

Broadcast operations are organized by function with equal, but separate production, engineering and management departments. OBS’s engineering department is under the direction of Isidoro Moreno who is responsible for the International Broadcast Centre and all sports venues. John Pearce is the director of venue engineering and the venues are divided among five venue managers while all things audio are under the direction of Nuno Duarte who has been the full-time audio manager for OBS since 2009.

The audio manager is responsible for anything technical that impacts the audio of the games including cable, equipment, mounts, the IBC, venue improvement, audio mixing rooms and OB vans. Duarte usually spends four years, with an intense two-year cycle, preparing for the next games. The audio manager usually arrives in the host country (this year, Japan) three months before the games begin. This may seem like a long time, but every venue must be surveyed weekly and even some daily to ensure that control rooms and venue field of plays are on schedule and that every cable is labeled and located where it is suppose to be. During this period the audio supervisor can find problems and variations in plans and hopefully accommodate all impacted parties.

Efforts that should be simple like drilling holes in the floors and walls will often include venue management, equipment suppliers, sports federations and the Host Broadcaster venue and audio manager to get an approval and sign off.

In addition to technical duties, Duarte is responsible for the sound design of each sport and the particulars of implementing the microphone plan into each venue. A great amount of time and energy has been spent on developing and shaping the sound and tone of the Olympic Games.

Years of developing and testing has evolved into refined capture methods that enhance the enthusiasm of a crowd no matter what the crowd size. I am not talking about using samplers to enhance the crowds; I am referring to proper microphone placement.

IN-DEPTH AUDIO PLANS

All Olympic audio plans include mounting and rigging details along with recommended "sweet spot" placement for each microphone. For example, large venues like track and field require microphones to be placed off-axis to the PA speakers. Rowan Smith has been responsible for rigging the microphones at the Olympics since 2000.

Microphones are positioned at different distances in front of a crowd zone with the microphones pointed at the audience. Microphones are often placed on ground stands so they can be moved and tuned to the seating and size of the crowd to present a “being there” impression.

The sound of sports is often associated with sizzling field sounds, PA banter and exuberant crowds. The spring of 2021 saw the return of spectators and posers to the racetrack, the horse track and baseball stadiums; and some would say sports looks and sounds normal again—in the United States.

However on July 23, 2021, the show in Tokyo will begin with Opening Ceremonies and stadiums that are empty or near empty. Crowd size always varies with the host country and any difficulties with travel and visas, but the stark reality is that a half-empty stadium is common. Crowds are usually modest for most Olympic sports anyway. For this summer’s Olympics in Tokyo there has been talk about masked, socially distanced crowds that cannot cheer or chant. I wonder what that will sound like or perhaps it means more noisemakers? I think the Japanese call them "thunder sticks" and when they are struck together it sounds like digital distortion, much more annoying than the Vuvuzelas.

The Olympics are often the stage for new technologies and production practices and the 2020 Summer Olympics, although a year late, is the debut of immersive sound. Tokyo 2020 or 2021 is the culmination of decades of work by the public Japanese broadcaster NHK to roll out Super High Definition television with 8K pictures and 22.2 sound. OBS made a commitment to implement immersive sound beginning in 2022, but will produce basic immersive sound in Japan eight months ahead of schedule.

Conceptually, immersive sound is often described as the venue experience and many of the current immersive sound designs are nothing more than atmospheric embellishments. Along with some production elements like immersive music, this level of production delivers an immersive experience without too much hassle. Immersive sound is still being developed and defined and an empty stadium is no way to practice and develop your sound. Developing a sound is not the objective of this summer’s sound production because clearly there are a lot of technical challenges for the production of immersive sound especially when there are over 30 different venues and an international crew that only works together every two years.

We are only eight months away from the Winter Olympics in China and I suggest that Tokyo 2021 is a dress rehearsal for the 2022 Games. During 2020, spectator-less stadiums were the norm with silly looking cardboard cutouts and audio samples of crowds. At the time of this writing I cannot imagine what Japan might do, although we will soon find out. 

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to  Television Sound Engineering,” published in both English and Chinese. His current book about immersive sound practices and production will be available this fall. He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com.

Since 2013, my first NAB Show stop has been at Avid Connect. This year I was extremely intrigued to see how the event would differ from previous years, since Jeff Rosica had taken the helm of the company following the still-fresh ouster of the previous CEO — who originated the event — along with the Avid Customer Association. It didn’t take long to find out.

Avid Connect has always tended to be product-heavy from start to finish, but this one simply did not feel that way. There was plenty of discussion of Media Composer, Pro Tools and the Media Central Platform — Avid’s business is to make and sell products after all — but the overall event, and the breakout sessions in particular, were far more interesting and educational than prior years, covering a wide range of topics that were not specific to the company and its products.

TECH TRENDS

One session that was surprisingly engrossing, “Technology Trends Musicians Can Utilize for Tracking Royalties.” It covered, among other topics, the pros and cons of streaming.

Panelist Jordan “DJ Swivel” Young made the case that, for those who have a dedicated fan base, streaming can actually be a greater source of revenue than CD or song purchases because the artist gets paid every time the song is played, not just when a purchase is made.

A different technological solution to the issue was presented by Benji Rogers, CEO of Dot Blockchain Media, who claims that current audio file formats are antiquated — WAV is 27 years old; MP3 is 24; and FLAC is 17 — and are part of the problem because they don’t contain persistent metadata.

Rogers proposes a new format and wrapper with blockchain as part of the fundamental framework so that ownership can be tracked from inception through all iterations, sampling and reuse.

Whether or not you’re an artist trying to get paid, the ability to encode persistent, unmodifiable metadata will soon become an essential element of every single piece of media as the ability to replace original audio and video content with completely believable digital substitutes becomes even more mainstream than it already is.

There were several sessions on immersive content and VR, both key elements of Next Gen Audio, and these sessions were far more informative than any I attended at NAB.

The panel of “Creating Spectacular Experiences With Dolby Atmos” included re-recording engineers Doug Hemphill, Ron Bartlett, and Tim Hoogenakker. During the discussion on mixing immersive content, Hemphill pointed out that as sound density increases, imaging and intelligibility decrease, so films with busier soundscapes are more difficult to mix and it becomes harder to make individual sounds stand out. In fact, he sometimes pulls back the density of the sound mix when there are really intense visuals to give people’s brains a break.

When asked what it was like to be able to place sound in so many locations, Bartlett said that they went a little overboard with the overhead speakers at first, but realized that this actually made the film sound more mono and less immersive, so more judicious use of the overheads is now employed.

[Read: 2018 Audio Prognostication—What Lies Ahead?]

Everyone felt that the soundscape in Atmos is more akin to listening in the natural world than standard surround mixes, though every so often the choices made in the surround mix are the right ones compared to the Atmos mix.

While they do verify downmixes as they mix, Hoogenakker said that he specifically checks television downmixes on a soundbar since more people than ever are listening on them at home.

Something that helps keep costs down while speeding up workflow is being able to work in 7.1.4 in smaller rooms to prepare audio for the Atmos mix room. Hemphill brought us all back to reality by reminding us that even though the ability to create amazing immersive soundscapes is now in everyone’s grasp, dialog remains king. Always.

CHALLENGES OF AUDIO IN VR

In the first session on audio for VR, “Techniques for Mixing Audio in VR Content,” the panelists covered aspects of VR production I had not even contemplated. Benedict Green of Ecco VR stressed the importance of preproduction and the fact that, when shooting content, the crew, recording equipment, cables and anything else that might take people out of the experience must be concealed, which is a tad difficult when shooting live actors while trying to capture and playback audio.

Avid Connect has always tended to be product-heavy from start to finish, but this one simply did not feel that way.

Green and Varuna Nair of Facebook both noted that ambisonics is the preferred format for VR audio and there are different tools for working with different order ambisonics.

They also discussed some of the challenges of working with audio for VR, including never being able to use a boom mic on set; and the difficulty of using ambisonics microphones because they cannot move without changing the soundfield. Managing loudness, and keeping audio from clipping is a huge challenge, as can working with nonaudio people who may reorder or mangle the ambisonics tracks. According to Nair, when thinking about VR, audio engineers should “think of it as another tool in the toolbox.”

In the session “Creating a Compelling Immersive Mix for VR Content,” Scott Gershin made the case that realism is not always the best approach, since the purpose of sound is to help tell the story in whatever way necessary.

Like Green and Nair, he also mentioned things I had not really considered. For instance, if you move a sound in the soundscape, people will pay attention to it, so don’t move it if you don’t want it noticed. Gershin does not move backgrounds, but he does group and move all sounds for given characters together to the same place in the soundscape because that is where the character exists at that moment.

Since we don’t really hear things behind us he feels that a 270-degree soundfield is wide enough. As for where this is all heading, Gershin thinks that VR will be commonplace in five years and boring in 10, but augmented reality is really where we’re heading anyway.

Jay Yeary is a television engineer who specializes in audio. He is a member of AES, SBE, SMPTE and TAB. He can be contacted through TV Technology magazine or attransientaudiolabs.com.

[Want more information like this? Subscribe to our newsletter and get it delivered right to your inbox.]

SYDNEY—There’s a new member of the Freedman Electronics Group, which includes audio brands Røde Microphones, Event Electronics and Aphex, as TSL Products’ SoundField has been acquired by the company.

SoundField is a maker of 360-degree surround sound recording technology. This acquisition sees Freedman take control over the entire SoundField company, including the current product line and associated IP. The existing product line will continue, with support being handled through a new entity. TSL Products will retain and rebrand the upmix/downmix processing product range.

In addition, TSL Products’ Product Director Pieter Schillebeeckx will join Freedman as new product development director in January 2017. He has been with SoundField for the past 16 years.

A Calrec Apollo digital audio console was the centerpiece in the Game Creek Video production truck used for the U.S. Open—one of several Calrec consoles deployed for the Fox Sports coverage of the event.

HUDSON, N.H.—Game Creek Video handles mobile television production for the world’s largest TV networks, production companies, and news organizations. We’ve earned a reputation for skillful coverage of sports and entertainment shows, which is why you’ll find our OB trucks and production teams in the truck compound for major events such as the Super Bowl, the World Series, NASCAR races and the FIS World Skiing Championship.

One of the biggest changes I have seen in sports coverage over the past 15 years is an increased emphasis on and commitment to audio by the broadcast networks. The networks have determined that the viewers’ audio experience is as important as the video. Audio has gone from being an afterthought to a key part of the production.

Fox Sports is one of the leaders in that charge. Last year, when covering the 2014 U.S. Open golf championship, we began planning with Fox Sports for coverage of the 2015 event. Game Creek would provide the ad hoc network at Chambers Bay golf course in Washington state, and Fox Sports would operate it. Fox is known for placing special importance on audio, particularly surround sound, because they understand the value of a high-quality audio production.

Brad Cheney, director of technical operations for Fox Sports, said it best: “We are out to capture the sounds of the course in an all-encompassing way, so we put equal emphasis on sound at every hole. It’s so important that we’ve even got additional audio engineers working in our ISO replay rooms to make sure the sounds are consistent on-air and online.”

Today’s productions place enormous demands on audio processing, and those demands would be multiplied for this event. We had to build a network to deal with 118 cameras, more than 200 microphones, and a massive number of line inputs. Audio would be coming in from a number of sources besides the microphones. For example, each replay device would have 16 channels of 5.1 audio. There would be audio associated with graphics. There would be primary and backup systems onsite.

There would also be 24 Calrec fieldboxes (small-format I/O units) sitting around the course, sending 5.1 audio to the trucks via fiber, providing a huge efficiency advantage. Not only would they save us from running DT-12 but fiber is lighter, less obtrusive, and can carry more signals than DT-12.

Fox would be producing content for its linear networks, Fox and Fox Sports1, along with complementary programming for usopen.com, usga.org and foxsports.com. In early talks, it was clear that capturing the sound Fox viewers have come to expect would require a large network of audio consoles and operators. Fox had already done its homework and determined that the only way to do it right was with Calrec consoles, specifically the Apollo.

Game Creek is an all-Calrec fleet. We built the network for Fox with four Apollo consoles, one Artemis console, and two Summa consoles, all inside of our production trucks. This would be the largest network we had ever created.

BIGGEST CHALLENGE

Calrec's Artemis console was used for the U.S. Open release mix.

The biggest challenge with a network of that size was tying all of the assets together and making them work with each other while giving the operators maximum flexibility. We had never attempted to tie five consoles together in a seamless package before. Without Calrec’s built-in Hydra2 technology we wouldn’t have been able to do it.

Fox operators mixed the broadcast with one Apollo desk and did the submix with another. They used the Artemis console for the release mix from the ISO replay rooms and mixed additional digital programming with an Apollo and two Summa consoles. While not on the network, SD10B consoles from Calrec’s sister company, DiGiCo, in the ISO production rooms, were connected via MADI to the main production truck to provide hole coverage via replay into both the main and digital programming.

The network meant Fox’s audio engineers could cover any part of the course at any moment with the full resources that are normally only available to the main show. The importance of this flexible networking capability cannot be overstated.

There were plenty of other benefits to using Calrec consoles. Over time, Calrec has expanded the number of audio sources its consoles can handle, in a way that’s efficient and instinctive for operators. That ensured that we could design a system that got the job done in the most efficient and least time-consuming way. And we knew the system would be robust and reliable—something we had to consider given the Pacific Northwest’s unpredictable June weather.

Another Calrec advantage is the company’s outstanding customer support. Their people are experts who not only helped us specify to configure the system but joined us in the field to help us set up the system to yield the best performance. That’s why Game Creek Video is a repeat customer.

The bottom line: Our business is all about making the customer happy. We know when we purchase Calrec equipment we’re going to get great reliability, great sound, great performance, great flexibility for the operator, and great service, so that ultimately we can help the customer create the very best production.