As UHD, HDR, live sports streaming, immersive audio and even 8K experimentation move into the mainstream, encoding has become a core business strategy. Broadcasters and streaming providers must elevate the viewer experience while reducing bandwidth and infrastructure costs.

Advances in AI-driven optimization, content-aware encoding and next-generation codecs enable operators to deliver higher-quality video at lower bitrates — fundamentally reshaping the delivery of premium video experiences.

Encoding as a Strategic Business Driver
Every additional megabit per second carries a cost — in CDN fees, transport, storage and processing power. At scale, even marginal bit rate reductions translate into substantial operational savings. Conversely, any visible drop in video quality risks churn, particularly in today’s competitive market where viewers can instantly switch services.

The challenge is inherently complex. Service providers must optimize three compression variables simultaneously: video quality, bitrate efficiency/processing power and latency. Improvements in one area often affect another. For example, reducing latency can come at the expense of the bit rate efficiency. Improving video quality by keeping bit rate low can increase computational load. Adding immersive formats increases complexity across the pipeline.

Modern encoding strategies recognize and treat compression as part of the overall delivery strategy, not just a codec setting.

The Rise of AI and ML Encoding Innovations
One of the most significant encoding developments in recent years has been the integration of machine learning into the encoding workflow. Several key enhancements are enabling broadcasters and service providers to deliver higher video quality, lower latency and greater efficiency.

Content-aware encoding
An advanced technique, content-aware encoding identifies visually important regions within video content — such as faces, text overlays or high-detail textures like grass — and prioritizes them for perceptual quality, (Fig. 1). Rather than treating every frame equally, content-aware encoding analyzes content characteristics in real time and allocates bits where they matter most.

Less critical areas receive fewer bits, preserving overall bandwidth while maintaining viewer satisfaction. Sophisticated rate-control algorithms can deliver significant bitrate savings, in some cases up to 50% without visible quality loss.

Real-time VMAF prediction
Today’s advanced encoding solutions can estimate perceptual quality metrics such as Video Multimethod Assessment Fusion (VMAF) during live encoding, enabling service providers to detect potential degradation before it reaches viewers. Real-time VMAF prediction models can achieve high correlation with offline measurements, up to 95%, allowing accurate quality assessment in live workflows and preventive encoding adjustments.

Automated quality analysis
Embedding AI into quality monitoring shifts service providers from reactive troubleshooting to proactive quality management. AI-driven regression testing and automated quality analysis enhance reliability by identifying deviations across nightly and weekly test streams. The result is a more resilient encoding pipeline where quality is continuously optimized. Service providers benefit by delivering better perceptual quality at lower bit rates, reducing distribution costs.

Intelligent node rebalancing
AI-driven algorithms assess system load, content complexity and processing demands to guide dynamic node rebalancing. This encoding approach enables more consistent resource allocation and stable video quality across distributed deployments.

GPU enhancements
GPU-accelerated enhancements play a pivotal role in the next generation of encoding. By integrating AI-driven pre-processing (like superscaling, denoising or deinterlacing) and GPU-enabled encoding control (like fine-grained Quantization Parameter -QP- control into the GPU pipeline), modern encoding platforms can deliver significant gains in performance and efficiency.

Preparing for the Next Generation of Codecs
While AI optimizations improve encoding efficiency within existing standards, broadcasters and service providers must also prepare their workflows and infrastructure for next-generation codecs.

Versatile Video Coding (VVC) promises up to 50% bitrate savings over HEVC while maintaining exceptional visual quality and is the selected codec for next-gen broadcasting standards like DTV+. Historically promoted as a royalty-free codec alternative, AV1 continues to gain momentum in OTT ecosystems with an improved efficiency compared to legacy codecs. And Low Complexity Enhancement Video Coding (LCEVC) offers a scalable enhancement layer that can improve compression efficiency without requiring full codec replacement.

Audio codec innovation further expands the scope of modern encoding platforms. Object-based formats such as MPEG-H and Dolby AC-4 enable immersive, personalized experiences. Dialog separation and accessibility features enable broadcasters and service providers to deliver personalized audio experiences to audiences. Moreover, support for object-based metadata for both MPEG-H and AC-4 enables precise audio rendering and personalization.

To accommodate for all these changes, a key strategic consideration for encoding is flexibility. Broadcasters and service providers cannot afford disruptive, large-scale infrastructure replacements every few years. Scalable encoding pipelines — capable of supporting multiple codecs, base layers and enhancement layers — allow gradual transitions aligned with market and business demands.

Powering Next-Gen Video with High Density, Low Latency and Immersive Readiness
Delivering next-generation video experiences requires broadcasters and service providers to handle intensive workloads with precision and reliability. Advanced encoding architectures are being designed for high-density and error-resilient performance. This, in turn, is laying the foundation for higher resolutions, lower latency, immersive formats and emerging viewing experiences.

Certain applications such as live sports streaming highlight why high-performance encoding architectures are essential. For instance, live sports streaming and interactive applications require high quality and a low degree of latency. Optimized pipelines reduce glass-to-glass delay while maintaining compression efficiency. This is essential for betting integrations, synchronized second-screen experiences and social engagement.

At the same time, experimentation with 8K and immersive video formats is accelerating. Encoding technology providers like Harmonic are trialing OTT profile ladders derived from an 8K source stream, processed in the cloud using both CPU and GPU resources. The profile ladder showcased in Figure 2 was processed in the cloud and would have been cost prohibitive two years ago. These trials illustrate the industry’s move toward higher resolutions delivered efficiently through hybrid compute architectures.

Even if 8K remains niche in the near term, the underlying engineering advances — high-density processing, scalable cloud-native workflows and error-free multi-profile generation — lay the groundwork for spatial computing, VR and headset-based experiences.

The New Compression Imperative
Ultimately, the latest encoding innovations enable broadcasters and service providers to deliver superior video quality at lower bitrates while reducing costs. Content-aware encoding, AI advancements and emerging codecs all have a role to play in helping service providers deliver premium experiences with the utmost efficiency.

In an era defined by subscriber churn, cost cutting and relentless viewer expectations, video compression remains a strategic necessity. Service providers that treat encoding as a core priority will be best positioned to thrive in the next phase of video evolution.

A new tech blog from Netflix highlights the importance of the AV1 open video codec, which now powers about 30% of the platform’s streaming and discusses a variety of opportunities to expand its current use, which is primarily for VOD content.

The blog by Liwei Guo, Zhi Li, Sheldon Radford and Jeff Watts comes at a time when AV2 is on the horizon.

“Looking ahead, we are excited about the forthcoming release of AV2, announced by the Alliance for Open Media for the end of 2025,” they posted. “AV2 is poised to set a new benchmark for compression efficiency and streaming capabilities, building on the solid foundation laid by AV1. At Netflix, we remain committed to adopting the best open technologies to delight our members around the globe. While AV2 represents the future of streaming, AV1 is very much the present—serving as the backbone of our platform and powering exceptional entertainment experiences across a vast and ever-expanding ecosystem of devices.”

The blog revisits Netflix’s AV1 journey to date, highlights emerging use cases, and shares adoption trends across the device ecosystem. It noted that since entering the streaming business in 2007, Netflix has primarily relied on H.264/AVC as its streaming format.

“However, we quickly recognized that a modern, open codec would benefit not only Netflix, but the entire multimedia industry,” they wrote. “In 2015, together with a group of like-minded industry leaders, Netflix co-founded the Alliance for Open Media (AOMedia) to develop and promote next-generation, open-source media technologies. The AV1 codec became the first major project of this collaboration, with ambitious goals: to deliver significant improvements in compression efficiency over state-of-the-art codecs, and to introduce rich features that enable new use cases. After three years of collaborative development, AV1 was officially released in 2018.”

The blog noted: “AV1’s superior compression efficiency was especially valuable for mobile users, many of whom are mindful of their data usage and network conditions. By adopting AV1, we were able to deliver noticeably better video quality at lower bitrates....Launching AV1 support on Android in 2020 marked a significant step forward for Netflix on mobile, making high-quality streaming more accessible and enjoyable for members everywhere.”

Based on that successful launch, the streamer then expanded AV1 support to smart TVs and large-screen devices. “This collaborative effort enabled our AV1 streaming to TV devices in late 2021. Shortly thereafter, we expanded AV1 streaming to web browsers (in 2022) and continued to broaden device support,” the post explained. “In 2023, this included Apple devices with the introduction of AV1 hardware support in the new M3 and A17 Pro chips.”

Today, AV1 is the streamer’s second-most-used codec and on track to become No. 1 very soon.

In part, this represents its “superior” compression technology for delivering 4K and high-frame-rate experience. “On average, AV1 streaming sessions achieve VMAF scores that are 4.3 points higher than AVC and 0.9 points higher than HEVC sessions,” the blog said. “At the same time, AV1 sessions use one-third less bandwidth than both AVC and HEVC, resulting in 45% fewer buffering interruptions.”

The blog also explained that the codec has been important in their efforts to provide more immersive experience with HDR and other features.

“In March 2025, we launched AV1 HDR streaming,” the post explained. “We chose HDR10+ as the HDR format for its use of dynamic metadata, which enabled us to adapt the tone mapping per device in a scene-dependent manner. As anticipated, the combination of AV1 and HDR10+ allows us to deliver images with greater detail, more vibrant colors and an overall heightened sense of immersion for our members. At the moment, 85% of our HDR catalog (from the perspective of view-hours) has AV1-HDR10+ coverage, and this number is expected to reach 100% in the next couple of months.”

The codec has also allowed Netflix to incorporate film grain, which has been difficult to faithfully render in digital video.

“The AV1 specification incorporates a unique solution called Film Grain Synthesis (FGS),” they explained. “Instead of encoding grain as part of every frame, the grain is stripped out before encoding and then resynthesized at the decoder using parameters sent in the bitstream, delivering a realistic cinematic film grain experience without the usual data costs. … In July 2025, we successfully productized AV1 FGS, and the results were astonishing: AV1 with FGS could deliver videos with cinematic film grain at a bit rate well within the capabilities of typical household internet connections. For non-FGS AV1 encodings, even at much higher bit rate, they may not be able to achieve comparable quality.”

Looking forward, the post noted Netflix has primarily used AV1 for VOD, but the platform sees “significant opportunities for AV1 beyond traditional VOD streaming,” including the use of "AV1 in live streaming, as we believe it could help further scale Netflix’s live programming.”

One use might be for events where Netflix is delivering content, such as live sports, to tens of millions of viewers simultaneously.

Another might be “customizable graphics overlay: for live sport events such as football, tennis and boxing, graphics overlays have become an integral part of the member experience—from embedding game statistics to delivering sponsorships. AV1 offers an opportunity to make the graphics highly customizable: layered coding is supported in AV1’s main profile, allowing encoding the main content in the base layer, and graphics in the enhancement layer, and easily swapping out one version of the enhancement layer with another.”

The full post is available here.

WILMINGTON, Del.—InterDigital today announced the acquisition of Deep Render, an artificial intelligence startup with a team of AI experts focused on video codecs. Financial details were not immediately available.

“Acquiring Deep Render means that we’re perfectly positioned to lead the development of the next generations of video technologies, building on our existing leadership in HEVC and VVC,” said Rajesh Pankaj, chief technology officer at InterDigital. “At InterDigital, we have spent years placing AI at the center of our wireless and video research to make networks more efficient and to change the way we consume content, and this acquisition only adds to our research leadership.”

The acquisition adds depth to InterDigital’s existing AI expertise, strengthens its position in video compression and accelerates its AI-native video research. It also adds Deep Render’s patent portfolio in AI-based video coding to InterDigital’s video portfolio. As part of the deal, a team of AI experts will join InterDigital’s Video Lab.

“With Deep Render’s strong focus on research and engineering and on solving some of the most complex challenges in video, we believe this acquisition is an excellent fit for InterDigital,” said InterDigital president and CEO Liren Chen. “The transaction deepens our talent pool in AI and video and extends InterDigital’s leadership in the efficient delivery of high-quality video, which enables consumers to enjoy content on a range of devices and a growing range of streaming and other video-based services.”

Founded in London in 2018, Deep Render has pioneered the use of AI in video and image compression.

More information is available on the company’s website.

U.K. researcher Rethink TV predicts that the valuation of the rapidly evolving market for advanced video codecs in the media and entertainment market will reach $2.4 billion by 2030, driven by royalties generated by the patent pools that have formed around the video compression tools.   

This latest forecast examines the adoption timeline for codecs, focusing on the arrival of VVC, AV1, and eventually AV2. It takes into account recent developments since the researcher last evaluated the market more than two years ago, including what it deems as “the most significant” move when Velos Media exited the VVC patent pool game and shuttered its HEVC pool, leaving just MPEG LA and Access Advance. 

Additionally, “the impact of Dolby’s Via acquiring MPEG LA and establishing the Via LA organization, cannot be understated,” the researcher said, adding that with Dolby’s involvement in both organizations, it expected that the two pools will eventually merge, down the line, at least to cut down on administration costs. 

“There is a question of whether a single pool option might help snare more licensees, but the practice of direct licenses with IP-holders, outside of the patent pool system, will continue apace,” Rethink TV said.

Regarding TVs that support the VVC codec, Rethink says those sets have arrived slightly ahead of its previous predictions, citing the availability of MediaTek’s Pentonic SoC, but adds that  Qualcomm has been slow to add AV1 to its flagship silicon in the mobile sector. 

“Potentially the most seismic shift has been Google’s relenting in Chrome, finally allowing HEVC in the browser, and opening the door for a possible explosion in the mobile sector for the codec,” the researcher said.

Rethink TV says the LCEVC enhancement layer has moved slower than it expected, but the codec “seems poised to announce a breakout year of deals.”

(Read: How Multi-layer Coding Standard MPEG-5 LCEVC Can Enable High-quality Metaverse/XR Applications)

“Given its software-only approach, it can move much quicker than the fully-fledged codecs, and its novel business model should demonstrate clear returns on investment for the video services that implement it,” it said. 

The researcher also noted the current debate over “fair contribution,” with both fixed and wireless ISPs calling for OTT platforms to pay their “fair share” for the use of traffic on the platform. 

“We have a dim view of these positions, largely because these same ISPs are averse to pursuing Multicast ABR on the basis that they ‘can just fire up some dark fiber,’ and maintain their familiar comfy unicast model,” it said. “‘Fair share’ or ‘fair contribution’ smacks of double dipping, and it speaks volumes that these ISPs would prefer to be paid than invest in technologies that might solve their bandwidth problems. To this end, the demand for new codecs seems to have been muted, and to wear our most cynical hat, there will be little pull from the ISPs to speed up the codec roadmap—as this would undermine their fair share argument.”

It concluded that its conclusions are based on the fact that manufacturers keep the rate of adoption and revenues garnered from patent pools close to the vest.

“Assuming that every device sold has paid all the royalties that these pools believe they are entitled to is naïve, and so any modeling done on device sales requires a judgement call about the success rate of codec royalty reclamation,” it concluded.

Click here to download the executive summary of Rethink TV’s Media & Entertainment Codecs Market Forecast 2023-2030.

DEVENS, Mass.—Comrex has announced BRIC-Link III, the latest addition to is BRIC-Link family of codecs first debuted at the 2022 NAB Show in April.

BRIC-Link III is an updated version of the codec that takes advantage of the latest improvements in IP audio transmission technology, the company said.

A true codec, BRIC-Link III offers a full-duplex stereo encoder and decoder in each box. A jitter buffer manager is incorporated that automatically balances delay and stability, dynamically increasing and decreasing delay based on network performance. BRIC-Link III offers a wide range of encoding algorithms, including stereo and mono linear mode, FLAC modes,

AAC/HE-AAC modes, Opus, G.722 and G.711, the company said.

Compatible with all Comrex IP audio codecs, BRIC-Link III also works with Gagl, a new audio contribution service available for purchase. Gagl turns any Comrex IP audio codec into a hub for up to five remote participants, it said.

BRIC-Link III includes CrossLock VPN technology, Comrex’s proprietary suite of reliability tools. In addition to bonding technology, CrossLock also includes a Redundancy Mode for mission-critical applications. BRIC-Link III can take advantage of HotSwap, meaning users can set a 4G/5G modem as a backup connection to only be engaged when the primary internet fails, it said.

An optional license to use the Comrex SwitchBoard private server that enables easy connections between IP codecs is available as a one-time purchase per codec, it said.

With a user-friendly HTML 5-based interface, BRIC-Link III has a compact chassis and can easily fit into a studio setup. A dual rack kit accessory for housing two BRIC-Link III codecs on a rack shelf side-by-side will be available for purchase, the company said.

Multiple operating modes, including normal (UDP), HTTP (Streaming Server, Icecast/Shoutcast, Multistreaming), RTP, TCP, IP Multicast and EBU 3326/SIP are available, it said.

More information is available on the company’s website.

ORLANDO, Fla.—“All I need is a QuickTime file.” How many times have you heard this from a client? Unfortunately, that’s not enough information. Here are the reasons why.

Digital files consist of format wrappers, data, and metadata. A format wrapper defines the type of file it is, as identified by the file extension, like .mpeg, .mov, .mp4, .mxf, etc. Think of a wrapper like a videocassette tape. It earmarks which player application should be compatible with that file, just like a VHS cassette told you that a VHS player was required. The data is the audio/video content contained within that format wrapper. The metadata is information about that data, such as date/time stamps, color profiles, and more.

UNDERSTANDING THEM

Audio and video signals are encoded into digital files using codecs—shorthand for compression/decompression. Most video files use data compression, whereas the audio signal in professional formats is typically uncompressed. On the other hand, consumer audio formats such as .mp3 and .m4a use a highly compressed codec. While we tend to associate certain codecs together with specific formats, like ProRes and .mov, they are not synonymous. For example, files can be encoded with the ProRes codec and stored in .mxf wrappers.

Video compression is defined as either lossy or visually lossless. This is determined by the codec type, data rate used to encode the file, frame rate, and frame size. File sizes will be smaller when a lossy method is used. But, they will display compression artifacts visible to the eye, because so much signal information is thrown away. Streaming services like Netflix and YouTube use lossy methods to get the signal to your home over the internet. Visually lossless codecs, like high bit-rate versions of the Avid DNx or the Apple ProRes family of codecs, are used in camera acquisition and post production. These codecs employ a high data rate for compression and will have significantly larger file sizes. However, compression artifacts are generally indiscernible to the eye compared to uncompressed video.

The vast majority of codecs used in production and post today are the proprietary intellectual property of companies or associations. They are not open source or open standard, even though their use may be ubiquitous. The use of a codec, especially to encode data to that codec, requires a licensing agreement. Typically, this is transparent and appears to be free to the user, but rest assured that an arrangement between companies has been made. As with all intellectual property, this can result in a codec no longer being available within an application if such an arrangement has ended between companies.

LIBRARY COMPONENTS AND THE 64-BIT TRANSITION

When a video file is played by an application, its data is decoded on-the-fly and displayed as RGB pixels to your screen or a viewer within the interface. This requires a set of installed library components that the application draws upon in order to read, decode, and display the video data. These components may be part of the computer’s operating system or they are custom-installed components that only function for that one application.

Over time, Apple and Microsoft have dropped or “deprecated” support for older codecs within their own operating systems. For example, Apple’s Catalina is a 64-bit operating system, with no support for 32-bit applications and library components. This means that certain codecs - including many still in active use, like DNxHD/HR—can no longer be decoded (played) through any application that depends on the 32-bit QuickTime framework used in previous versions of macOS. This is not an issue of the codec itself, but rather the library components used.

In order for such files to work within Catalina, Apple or the application developer has to write new 64-bit library components to play such files going forward. In the case of DNx, Avid and Adobe can read and write these files in the .mxf format, but Apple’s own applications, like Final Cut Pro X or QuickTime Player, are not able to do the same. Expect that to be updated later this year.

WHY CAN’T I WRITE CAMERA RAW FILES IN POST?

Digital video cameras convert raw sensor data into RGB pixel information and record that to a digital file using a defined codec and format. Processing is done in the camera to “bake in” the conversion of the Bayer pattern sensor data to RGB, along with its native ISO and a color profile. These files are then easily played by most professional editing and player applications.

Ever since the launch of RED Digital Cinema’s RED One camera, compressed raw codecs have grown in popularity. When you record a camera raw signal, the conversion/processing step to RGB is skipped, thus enabling access to more latitude for color correction in post. It also generates higher quality images for an equivalent data rate and file size. The trade-off is that camera raw files tax the hardware systems used in post. That’s because the conversions that would have been done in the camera are now performed in real-time by the computer.

Raw codecs store information generated by a camera’s sensor and are only intended for image acquisition. You cannot write or re-encode camera raw files in post, because there is no sensor data available. Camera raw codecs are also proprietary to individual companies, including RED, ARRI, Apple, Blackmagic Design, and others. For now, there is no video equivalent to CinemaDNG, a photographic raw format developed by Adobe and then offered up as an open standard to the community. Although it could be argued that GoPro’s CineForm RAW codec is a comparable solution for video.

The video from non-raw codecs can be adjusted through color correction, but you aren’t actually altering the underlying color processing of the file itself. You are skewing the RGB information that is already there. In contrast, camera raw codecs offer the opportunity to leverage the way the sensor data is actually decoded, using various color science and adjustment schemes. This requires each company to create a camera raw plug-in specific to their proprietary codec, which the user can access in order to “develop” the image. Software engineers have several options: 1) keep that proprietary process isolated to their own applications, 2) create plug-ins for other companies to use, 3) create OS components that everyone can tap into, or 4) offer an SDK and let other companies write their own tools to use within their own—often competing—applications.

Codecs like REDCODE are widely supported in most applications, thanks to RED’s plug-ins. Others, like ProRes RAW and Blackmagic RAW, are in a transition state as NLEs add support over this coming year. In a practical sense, if you own a camera that records raw video, don’t assume that your favorite editing tool or post house can deal with those files. As always, do your homework and ask the right questions.

WAKEFIELD, MASS—A group consisting of some of the world’s largest technology companies has announced the formation of a technology alliance to develop next generation video codecs for IP-based video. 

Founding members for the “Alliance for Open Media” include Amazon, Cisco, Google, Intel, Microsoft, Mozilla and Netflix. The group says its desire for an “open and interoperable” next generation video standard will “meet growing demand for top-quality video, audio, imagery and streaming across all devices of all kinds and for users worldwide.” It says the “open source project” is designed to develop new formats “in the public interest.” 

Although the word “standards” is nowhere to be found in the group’s announcement, it’s clear that the group is using its technology heft and worldwide reach to avoid the financial obligations that come with deploying current next-gen standards such as HEVC, the standard most likely to handle 4K. “The Alliance for Open Media brings together the leading experts in the entire video stack to work together in pursuit of open, royalty-free and interoperable solutions for the next generation of video delivery,” the group said in its announcement.

The new codecs will be based on current alliance video member formats such as Mozilla’s Daala, Google’s VP10 and Cisco’s recently announced Thor royalty-free codecs and will target the increasing use of high-resolution video, such as 4K, over IP. “As resolutions and framerates increase, the need for more advanced codecs with ever-better compression ratios will only grow,” said David Bryant, in a blog on Mozilla’s website. “We believe that Daala, Cisco’s Thor and Google’s VP10 codec combine to form an excellent basis for a truly world-class royalty-free codec.”

The initial project, according to the alliance, will focus on developing a new codec based on member contributions, along with binding specifications for media format, content encryption and adaptive streaming. More information on the alliance, including how to join, will be made available later this year.