This year has witnessed an explosion in the use and discussion of artificial intelligence (AI). It is, of course, something that has been around since the 1950s but since the end of last year, coverage has ranged from the serious to hysterical, particularly over its latest manifestation, generative AI (Gen AI).

This more evolved form of AI can create different kinds of content, from text and images to audio and synthetic data, which is more realistically human instead of something clearly produced by a machine. At the forefront of this is ChatGPT (generative pre-trained transformer), which, although only launched in November 2022, has already radically changed the direction of automated speech, text and image creation.

Fast and Precise
As is often the case with “new” technology, many of the features of AI had been available to broadcasters and media producers in the last decade.

“We’ve been implementing AI in our tools for years, so it’s not something we’ve jumped on with ChatGPT,” says Andre Torsvik, vice president of product marketing at Vizrt. “AI is used to make computers do what they are best at, which is being fast and precise. Human beings can find that difficult to do, with, for example, outdoor keying on a sports field. An AI can react much faster and the end result is a much better key, you don’t see flickering or ads on top of players.”

The most common applications for Gen AI in broadcast include: 

• image generation and video synthesis 
• automated production 
• assisted video editing 
• creation of metadata for automatic “logging” 
• subtitling, captioning, segment-specific searches and re-use of media assets 
• automatic upscaling of content to higher resolutions; and
• encoding/decoding of audio and video streams 

Simon Forrest, principal technology analyst at Futuresource Consulting, comments that AI is capable of assisting artists and producers to create content more quickly, enabling “faster iteration, more exploration and delivering media assets that approach a harmonized composition.”

Another assistive application of AI comes in the form of improved archive searches. This has been a particular area of research at the Fraunhofer Institute for Intelligent Analysis and Information Systems (IAIS). Dr Christoph Schmidt, head of Fraunhofer IAIS’s Speech Technologies Unit, explains that it goes beyond keyword techniques, with Gen AI and natural language processing providing “improved ways to find relevant archival content to produce new programs.”

Efficient and Dynamic
In addition to reducing the number of repetitive tasks performed by operations staff and allowing creatives to do their jobs faster and better, Peter Sykes, strategic technology manager at Sony Europe, sees AI as “making the whole media supply chain as efficient and dynamic as possible, helping analyze and then address resource allocation and process steps to optimize them, leading to better business decisions.”

Sony established an AI division in 2020, focusing on projects in the areas of imaging and sensing, gaming, gastronomy (robotic manipulation of food and cooking utensils) and AI ethics. 

For broadcasting, Sony offers its A2 Production system, which, among other features, can identify sports highlights using automated logging and scene detection features. It is also applying AI to software-defined networks through the VideoIPath media orchestration platform developed by its Nevion subsidiary.

Ross Video implements AI across two of its main business groups: robotic cameras and newsroom systems. Both are working on integrating AI into their product lines, with robotics using a more proprietary-style technology built into its products while news is relying on third party AI engines.

Jenn Jarvis, product manager for newsroom computer systems at Ross Video, says this gives customers a choice of which engine they want to use and a framework for how they integrate it into a product. “We’re also now looking at the content creation side, which is part of the newer aspects of AI we’re still exploring and seeing how it fits into a news workflow,” she said.

For robotics, Karen Walker, vice president of camera motion systems, observes that the person managing shot selection will still have some work to do in keeping the presenter in frame or in focus.

“But the next thing for AI—and a lot of people have come out with this—is a ‘talent tracking’ application,” she says. “You don’t have to have any human intervention so its possible to set up pre-sets and where you want the talent to be in that shot. This is done independently of the talent and can be configured for different presenters. It takes away some of that manual intervention and I think it’s where AI has benefits, in taking out some of the mundane tweaking.”

Reducing Tedium
Removing, or at least easing, the amount of dull but necessary elements in live production by employing AI is now a realistic proposition. This is illustrated by Rob Gonsalves, engineering fellow at Avid, who gives the example of Open AI’s Whisper speech-to-text model. 

“It can be applied for live transcription and real-time translation of multiple languages within a broadcast feed, toggling through as many as 100 languages simultaneously,” he said.

Avid has also tested both OpenAI’s CLIP model and the generic GRoIE ROI (region of interest) extractor as part of research into auto-framing.

“This followed the area of a shot that is of the greatest semantic interest,” Gonsalves explains. “The traditional way to enable search is to manually annotate media assets with metadata tags describing what is in the shot. Using AI object or facial recognition can now automate the scanning and annotation process. Semantic search does something similar but, by creating embeddings into clips, it allows an editor to conduct a free-text search for scenarios.”

Sepi Motamedi, global industry marketing lead for professional broadcast at NVIDIA, comments that “live production, particularly for sports, takes tremendous advantage” of AI. “It is used in super-slow motion replays to localize advertisements seen on the pitch, to quickly generate highlights from the game, to deliver an added layer of data through telestration [which involves pitch calibration and player tracking] and, of course, camera tracking.”

Among the first developers to begin applying AI to broadcast applications from the outset was Vertitone. Founded in 2014, the company offers an enterprise AI operating system platform, aiWARE, along with engines for ChatGPT, audio, biometrics, speech, data and vision. 

“We realized there was an opportunity in the media and entertainment markets to begin to index the world’s audio and video content,” comments Paul Cramer, managing director of media and broadcast at Veritone. He adds that once material is indexed, Gen AI can be used to “create a new personalized experience for the consumer” who is looking for custom content. This could be in the form of news footage tailored for a viewer with an interest in, for example, space exploration.

From Broad to Niche
As AI is adopted more widely throughout the media sector, it is being used for both very niche and quite broad applications. Moveme.tv illustrates a highly specialized use: its search platform is designed to help viewers match films to their mood through the use of descriptive words and emojis. Founder and chief executive Ben Polkinghome says the ultimate aim is to enable people to create “their own hyper-personal entertainment channels.”

On a wider broadcast level, BBC’s R&D department initiated its AI in Media Production program in 2017 with a prototype video editing package that automatically selected and assembled shots into a finished piece. This work continues today and was expanded last year with a new data set for “Intelligent Cinematography” to assist in framing and editing. BBC R&D is due to announce its position on Gen AI soon but could not give any more details before TV Tech went to press.

AI is now exploited by all types and sizes of media creators and organizations, both new and old. Video marketing platform Vimeo announced in June it was making a Gen AI-powered “creation suite” that simplifies the process of making videos. The package includes a script generator, teleprompter and text-based editing system that automatically deletes filler words and long pauses. 

On the more traditional side, the All England Lawn Tennis Club is using the Gen AI capabilities of IBM’s Watson platform to produce commentary for video highlights of the 2023 Wimbledon championships on its app and website.

Despite the innovative spirit of this, AI has caused discomfort among media professionals, both in terms of jobs being potentially lost and the risks of the technology being misused. Major news outlets including “The New York Times” and NBC News recently voiced concern over how Gen AI could not only make journalists redundant but enable unscrupulous types to produce fake but believable stories.

The FCC has its own working group on AI and among the topics it’s initially focused on has been using the technology to improve its services including using AI to manage spectrum more efficiently. The commission is hosting a joint workshop this month with the National Science Foundation “to discuss the possibilities and dangers AI presents for the telecommunications and technology sectors.”

In the U.K., media regulator Ofcom, while acknowledging that Gen AI offers benefits such as synthetic data training for better safety technology, has similarly highlighted the dangers of bogus news and other media content. Ofcom is currently monitoring the development of Gen AI to see how its positive aspects can be maximized and also what threat the more negative ones might pose.

As for the human impact, Karen Walker at Ross Video observes that “a lot of creative things have to be done by humans—people are going to work with AI side-by-side.”

Veritone’s Paul Cramer concludes: “Gen AI won’t replace humans but it will replace the humans who are not using AI.” 

ERLANGEN, Germany—LG Electronics has licensed MPEG-H Audio from Fraunhofer IIS, allowing the consumer electronics company to manufacturer products with the audio codec, Fraunhofer announced today.

MPEG-H Audio, developed largely by Fraunhofer IIS, is specified as one of the audio systems that is part of ATSC 3.0. It has been on air in South Korea since May 2017 when the nation rolled out the Next Gen TV standard.

MPEG-H Audio also is specified as part of the DVB standard. In Europe, where many countries use DVB, MPEG-H has been tested successfully as part of the digital television system.

Among MPEG-H Audio’s features are immersive audio support as well as audio personalization.

More information is available on the Fraunhofer IIS website.

LAS VEGAS—If you were a broadcaster walking the aisles of the 2018 NAB Show earlier this month in Las Vegas, no one could blame you if you focused much of your attention when it came to video compression on HEVC, or its extension SHVC, which will help to power Next-Gen TV.

However, high efficiency video coding as implemented by ATSC 3.0 wasn’t the only compression development that promises new efficiencies for those in the M&E industry.

The Fraunhofer Institute for Integrated Circuits IIS used the Las Vegas venue to announce the first JPEG XS codec implementation for professional video. The research institute demonstrated its new codec as an input format for post-production.

Specifically, Fraunhofer showed JPEG XS as a software plug-in for Adobe Premiere Pro CC, providing real-time playback of 4K 60p UHD video in a Windows environment using an NVidia graphics card to accelerate decoding.

[Read: A discussion with Fraunhofer on what’s ahead in compression] 

Fraunhofer is positioning its new codec as a solution to the impending train wreck it sees when the ever-growing data demands of higher res video and HDR meets the limitations of coax cables in studios. Exacerbating the problem is the desire of production companies to move away from specialized switches and cable for video transport to commercial off-the-shelf alternatives that move IP packets via standard Ethernet infrastructures.

The JPEG XS codec, which is being standardized by the JPEG committee for IP workflows in studios, local video networks and VR/AR applications (ISO/IEC SC29 WG1), enables high-res video transfer via standard Ethernet and other wired connections, Fraunhofer says.

It is optimized for use with mezzanine compression applications involving high-res video to be transferred over limited bandwidth or when limited computer resources are employed. JPEG XS is also designed to take advantage of the SMPTE ST 2110 transport layer.

BERLIN, GERMANY—When it comes to advances in telecommunications technology—and specifically video codecs—Germany’s Fraunhofer Heinrich Hertz Institute has been at the forefront. The organization is an integral member of the experts groups, which developed the H.264/AVC video codec nearly 20 years ago, as well as its successor, HEVC. It was also among the team that received numerous Technical Emmys including one in

2017 for the High Efficiency Video Codec.

Thomas Wiegand, executive director, Fraunhofer Heinrich Hertz Institute HHI; Benjamin Bross, project manager at the Video Coding & Analytics department with the Fraunhofer HHI and Detlev Marpe, head of Department Video Coding & Analytics at Fraunhofer HHI

Now celebrating its 90th year, TV Technology caught up with Benjamin Bross, project manager at the Video Coding & Analytics department with the Fraunhofer Heinrich Hertz Institute to discuss the evolution of HEVC and what’s next in video compression.

TV Technology:What are the latest developments on video compression standards work?

Benjamin Bross: The ITU Video Coding Expert Group (VCEG) and the ISO/IEC Moving Pictures Expert Group (MPEG) have already started working together on a successor to HEVC by forming the so-called “Joint Video Exploration Team” or JVET. We currently have a joint exploration codec model (JEM) already in place; this isn’t necessarily the successor codec, but it’s new technology that was proposed and investigated after the standardization of HEVC. This JEM codec provides about 30 percent bit rate reduction for the same objective quality compared to HEVC. So this is clear public evidence that it’s possible to further improve the compression performance compared to HEVC. We have started to work on the successor with a call for proposals and it is possible that we might soon be seeing another factor of two in coding efficiency relative to HEVC.

TVT:What does the call for proposals entail?

BB: Last October, JVET issued a call for proposals for video coding technology beyond HEVC and several companies including us are submitting technology. Each proposal will be independently tested and compared to HEVC and the results will be evaluated at the next JVET meeting in April. That’s when the official standardization process will begin with the goal to finish the new standard by 2020. Besides submitted coding technology, a lot of other influences will be taken into account. One of the main concerns from hardware manufacturers for example is complexity. Even if you find a nice technique that significantly reduces the bit rate, but it’s super complex, chipset vendors will say ‘that’s pretty nice, but there’s no way we can build a chip using that technology at a reasonable cost.’

[HEVC Advances Enhance Cell-Based ENG] 

TVT:Is it the JVET that’s doing this?

BB: Yes, the JVET, ITU and ISO are teaming up to generate a codec that is both efficient and simple enough in order to be able to put the decoder on a chip that can be integrated into TV set-top boxes, mobile phones and so on. The goal is to reduce the bit rate by up to 50 percent of the previous standard for the same perceived subjective quality.

TVT:Can you elaborate on ‘subjective’ vs. ‘objective’ quality?

BB: You cannot perform subjective evaluation for every new thing that you add during standardization—the logistics to perform subjective testing for every new incremental step is just too high. That’s why you have an “objective error measurement” that you calculate, like some kind of mean squared error-based measure that gives you an indication of the quality; but usually these measurements are a little bit more pessimistic compared to subjective quality.

For example, in HEVC, we ended up with 38 percent bit-rate reduction over H.264/AVC for the same objective quality. When we did perform subjective evaluation and verification of that in independent test laboratories, we ended up with 50 percent bit-rate reduction for the same subjective quality perceived by the individuals involved in the test.

TVT:Would you say HEVC has attained its goal, has it been a success in terms of adoption?

BB: From a technical perspective, I would say totally. The support is very broad. Apple introduced it last year, so now every new iPhone is able to decode HEVC up to UHD—almost every Android phone is able to do that—and all of the new TV sets have HEVC built in, as well as new set-top boxes and other media devices. Then there’s the question of whether business-wise it makes sense to employ HEVC. Apple’s support was extremely important, so now we will see how many services will jump on that and actually use it.

For the broadcast community HEVC is pretty well set, especially. When it comes to UHD and new improvements like HDR and high frame rate, HEVC is the codec to use. In Germany, for example, they started the new terrestrial TV service with HDTV that uses HEVC and it’s working pretty well.

TVT:What about the future development of HEVC? Has it fully matured as a standard and is that why you’re working on a new compression standard?

BB: For compression efficiency, when it comes to encoders, there is a steady improvement still to come. When you look at MPEG-2, H.264/AVC, even over the last five years—and H.264/AVC is already quite old—encoders have improved in efficiency; improvements that are also being carried over to HEVC. I don’t think we’re at the end of the line efficiencywise for HEVC, but the encoders that are out there are already mature enough to provide the gain that HEVC promised at the beginning.

TVT:What is Fraunhofer HHI is doing in regards to HEVC?

BB: After we contributed core compression techniques to the standard, we started to develop products for HEVC encoding and decoding to the industry. We have a software encoder that is used in broadcast to encode HD and UHD content live, for example, for satellite and terrestrial television in Germany. Here, we do not sell the appliance, we provide the software encoding component.

TVT:Where do you see advances in video codecs going?

BB: Right now, everyone is following the same concept of hybrid video coding. They’re adding and improving a lot of parts of it, but the general concept remains the same.

TVT:How important is it for Silicon Valley companies like Google, Amazon, Facebook, Netflix and Apple to participate in the development of the successor to HEVC?

BB: It is very important as compressed video data are the by far highest percentage of bits on the Internet and mobile phones. These companies and everybody else who is serious about video compression will highly benefit from the coding efficiency improvements that the JVET process will surely deliver. They and many others are interested in video compression technology improvements that will help their businesses to flourish.

For example, Netflix has provided content to the JVET already. When we develop a standard we need test sequences that are characteristic of the content that is out there; we cannot test or develop a new standard using sequences captured with cameras from the ’90s.

TVT:Are you saying then that it’s to their benefit to participate in this process rather than try and go their own way?

BB: Sure, and they will be looking at the new video codec and realize that the achieved coding efficiency improvements are a big deal. But, if you create such a superior video codec as a result of a joint effort with a lot of players in the industry, there are patents involved. Hence, there needs to be a reasonable way to get a license.

AMSTERDAM—Berlin-based telecommunications research institute Fraunhofer Heinrich Hertz Institute HHI and satellite services provider SES used the IBC show to debut the transmission of virtual reality content over satellite to multiple devices.

The 10K by 2K panoramic video signal is showing up at SES’ booth at the conference where it’s being relayed directly to an Ultra High Def (UHD) TV and VR head-sets for attendees. Viewers can choose to watch the content in multiple viewing angles, zoom in and out, and change the picture via remote.

The VR images were filmed using Fraunhofer HHI's OmniCam-360 camera and are being transmitted via SES’ ASTRA 19.2 degrees east orbital position satellite.

“We use a combination of technologies here exactly to showcase what is possible when using hybrid approaches," said Dr. Ralf Schäfer, Fraunhofer HHI's head of division video, in a statement. "There is no stadium in the world providing enough seats for all enthusiastic fans. So imagine a live event somewhere in the world — filmed with professional cameras like our OmniCam-360 and then delivered to a huge global audience via satellite. And every single viewer at home has the best seat in the middle of the show.”

Thomas Wrede, VP of reception systems for SES, added: "Satellites are the perfect distribution path for these new kinds of video experiences, as they can manage huge volumes of data being offloaded from terrestrial networks. Furthermore, technology standards like SAT>IP not only allow the viewers at home to pick and choose a device - the TV screen, tablet or virtual reality equipment."

This story originally appeared on TVT's sister publication, B&C. 

BERLIN—Virtual reality is adding a bit of the real-world, as Fraunhofer HHI has announced that it has developed a new camera system that allows for 3D human body reconstruction, creating a 3D model of someone and placing it into a virtual model.

The camera system from Fraunhofer is at its base a stereo camera, which records an image with two lenses, similar to what people do with their eyes; the results creates a 3D impression. More than 20 of these stereo cameras are used to map a person, with each camera focusing on a specific part. The different images are then merged to create a single picture.

Once the image is completed it is transmitted into the virtual world, where the person can move freely in the dedicated capture area. Fraunhofer’s eventual goal is to be able to have a person be able to directly interact with the virtual world.

Fraunhofer will demonstrate its new technology at the upcoming IFA fair in Berlin and the IBC 2016 Show in Amsterdam.

PARIS & ERLANGEN, GERMANY—Technicolor and German-based Fraunhofer IIS have announced a new licensing program as part of the market introduction of the next-generation TV audio system developed by the MPEG-H Audio Alliance, which is also under consideration for the ATSC 3.0 standard.

The MPEG-H Audio Alliance TV system enables TV broadcasters to capture and deliver next generation audio and allows consumers to customize content experiences. The MPEG-H Audio Alliance uses a subset of the MPEG-H 3D audio standard, according to Manuele Wahl, head of technology and trademark licensing, Technicolor.

The licensing program will focus on a per-unit royalty, with annual volume discounts for manufacturers of both professional and consumer hardware and software. No patent royalty will accrue for producing , transmitting or distributing content under this licensing program.

Technicolor will administer the licensing program of the standard essential patents for the MPEG-H Audio Alliance TV system. The program builds upon Technicolor’s MP3 licensing program, as well as the company’s experience managing 1,100 licensing agreements across 15 licensing programs.

LOS ANGELES—The technology of broadcasting is due for a major upgrade once the Advanced Television Systems Committee finalizes the specifications for the proposed ATSC 3.0 standard, designed to replace the 20 year old ATSC 1.0 specification (augmented in 2013 with ATSC 2.0). At the association’s meeting last month, the record number of attendees indicated how crucial this new standard is going to be for the future of TV both over-the-air, on cable, and potentially influencing IT delivery.

MPEG-H Audio offers viewers the ability to turn up or down particular audio elements in a program--such as dialogue or sound effects--as they prefer.

“Our schedule is to get to a ‘candidate standard’ level for all the components of ATSC 3.0 by the end of the year,” said ATSC President Mark Richer. “This will then be voted on before the end of next year by our technology group and then by the whole ATSC membership for the final decision.”

AND THEN THERE WERE TWO

The industry is abuzz with speculation about the upcoming video standard in ATSC 3.0, possibly including High Dynamic Range, multiple frame rates, and wider color gamuts. But the audio component of this new standard has received far less publicity.

Yet the field of competitors for the next broadcast—and therefore production—audio standard which once included Dolby Labs, DTS, Fraunhofer IIS, NBCUniversal, Samsung, Technicolor and Qualcomm, has been whittled down to two contenders: Dolby, with its AC-4 technology, and the MPEG-H Alliance represented by Fraunhofer on behalf of its partners, Qualcomm and Technicolor.

MPEG-H is an existing international standard that Fraunhofer is proposing as the audio part of ATSC 3.0, according to Robert Bleidt, general manager of Fraunhofer USA Digital Media Technologies in San Jose, Calif., a subsidiary of Fraunhofer-Gesellschaft, the renowned German R&D organization. “We are trying to use the MPEG-H approach to bring three primary features to TV viewers: interactivity, greater immersion, and universal delivery.”

As Bleidt explains it, the “interactivity” capability will let viewers adjust the presence of various audio objects in the broadcast signal. This could include bringing an announcer’s voice out of the background for greater clarity, or the option of listening to either the home team or the visitor’s native broadcast mix depending on fan preference, and even the potential of hearing programs in different languages.

Robert Bleidt

The “immersive” factor is a quantum leap over Surround Sound because it adds 3D audio components such as many more channels including ceiling or floor speakers. Fraunhofer’s psycho-acoustic tests have shown this is almost as significant as moving from stereo to surround sound.

To get these sounds to the panoply of big and small screen devices that want to receive them, MPEG-H can optimize its audio reproduction to best match the capabilities of the system to which the viewer is listening.

“One of the primary advantages of the MPEG-H system is that it is designed to work with the existing equipment of today’s broadcasters,” Bleidt said. “We’ve established a four-stage process to gradually put MPEG-H into today’s broadcast workflow. In fact, we showed all four stages working live on-the-air at the NAB Show.”

Fraunhofer accomplished this with the help of some MPEG-H audio monitoring units from Jünger Audio after installing new encoding/decoding equipment to handle the MPEG-H codecs. They demonstrated that with MPEG-H there is no need to overhaul either existing control rooms or outside broadcast trucks, and it can be easily edited on the NLE timelines such as Adobe’s Premiere Pro CC.

The four stages to attain full MPEG-H capabilities start with transmitting Surround Sound using current metadata, but with a 50 percent reduction in bit rate over today’s AC-3 model. Then the broadcaster could add interactive objects to let viewers adjust the dialog or sound effects levels in the audio mix. Step three incorporates the inclusion of 3D sound with ceiling channels and the final step includes the option to introduce “higher order ambisonics” (HOA) to represent the sound environment by determining the accuracy of representation of the pressure field. A spatial encoding is performed on the HOA coefficient signals to fit HOA signals into a studio’s existing SDI infrastructure.

To enable flexible implementation, MPEG-H can use any combination of three techniques to get the audio through a broadcast system: traditional channels such as 5.1, 7.1 or more, audio objects, or HOA. 

“Even though this will require viewers to purchase all new equipment to get these advantages,” Bleidt said, “the MPEG-H Alliance feels if MPEG-H is adopted for the audio component of the ATSC 3.0 standard, it will be a great opportunity to remove the shortcomings of the prior system and give viewers the advantages of these new audio technology features.”

BACK TO THE DRAWING BOARD

Dolby Labs’ candidate for ATSC 3.0 audio revolves around the company’s AC4 technology.

“For us it is all about improving today while enabling tomorrow,” said Jeff Riedmiller, senior director of the sound group, office of the CTO, at Dolby Laboratories in San Francisco. “Three years ago when we started developing AC4, we really went back to the drawing board.”

Jeff Riedmiller

As Riedmiller describes it, the vision of AC4 is based on four pillars, the first of which is to improve reach to enable efficient delivery to all types of devices.

While the Surround Sound technique found in its predecessor, AC3, provided a multichannel living room experience, AC4 offers 50 percent more efficiency, enabling delivery of 2D and 3D sound environments to a wider number of homes and mobile receivers. AC3 delivered stereo sound at 192 Kbps and 5.1 surround at 384 Kbps, but for today’s IT delivery, the market is demanding stereo bitrates at 48-64 Kbps, 5.1 at 100-200 Kbps and immersive multichannel audio like Dolby Atmos at 200-300 Kbps, which is what AC4 enables.

The second pillar is simplified daily operational aspects, such as automatic volume leveling and seamless audio for ad insertions. To comply with the CALM (Commercial Advertisement Loudness Mitigation) Act, AC4 has built-in intelligent loudness and multiband dynamic range control without manual intervention.

“These compression curves in combination with the loudness information generated by the encoder produce in real time the dynamic range coefficients desired by the original artist,” Riedmiller said. “The decoder then has the ability to scale across the needs of various playback options ranging from mobile phones to home theater systems.” 

Pillar number three of AC4 allows the end user to choose which components of the audio tracks they want to select to enhance their listening experience, including descriptive video, multiple languages or home field broadcasts, with the ability to enhance and control the center announcer or dialog track’s presence.

Then the fourth pillar delivers immersive 3D object-oriented sound, including height dimensions and potentially speakers providing sound at floor level. 

“Just adding the height element brings us a huge leap toward engaging the senses in a more natural way,” Riedmiller said. “The next system for ATSC sound needs to shoot for ‘audio nirvana,’ sort of a ‘Stairway to Heaven’ beginning with better codec efficiency, improved loudness management, enhanced audio perspective and language substitution, and then the immersive benefits of audio objects throughout the sound field.”

Riedmiller acknowledges that any system based on ATSC 3.0 will not be backward compatible with current ATSC equipment, but added that “we needed to start with a clean slate when designing our audio proposal. 

“TV and set-top box manufacturers should be able to develop additional support capabilities to provide the audience with a satisfactory viewing experience on legacy equipment,” he said.

The proposed technologies from both contenders—Fraunhofer with the MPEG-H Alliance and Dolby Labs’ AC4—are based on years of audio technology experience. Whichever wins the competition for the audio component of the ATSC 3.0 standard will provide a quantum leap in the audience’s enjoyment of future digital audio/video communications.