CAMBRIDGE, Mass.—Akamai Technologies Inc. has announced that Ateme is the latest organization to join the Akamai Qualified Computing Partner Program. 

The Akamai Qualified Computing Partner Program is designed to make solution-based features that are interoperable with Akamai's cloud computing services easily accessible to Akamai customers. The services are provided by Akamai technology partners that complete a rigorous qualification process to ensure they are readily available to deploy and scale across the globally distributed Akamai Connected Cloud, Akamai explained. 

As an Akamai Qualified Computing Partner, Ateme will be able to offer Akamai customers end-to-end high-quality video-delivery solutions and greater choices in their digital transformation. The smooth, one-stop source facilitates cloud migration, as well as customers' search for a relevant Akamai-supported solution, the companies said. 

"We are honored to become a part of the Akamai Qualified Computing Partner Program," said Ricardo Minari, director of global partnerships at Ateme. "This partnership highlights Ateme's commitment to ensuring our customers deliver the highest quality of video content to global audiences, even during their ongoing digital transformation journeys."

With Ateme's high-quality video-delivery solution running on Akamai, joint customers can benefit from low latency and better throughput thanks to the power of Akamai Connected Cloud, Akamai's massively distributed edge and cloud platform, the companies explained. 

As a qualified computing partner, Ateme offers workloads that can be deployed with a range of origin options on Akamai, giving customers the flexibility to choose either a self-managed or an Akamai-managed origin. In addition, Ateme's compression efficiency combined with Akamai's cloud computing services' competitive data-transfer rates can help customers enjoy the cost benefits of reduced bandwidth and storage requirements, the companies said. 

"Akamai is pleased to make quality video compression and delivery easily available to Akamai customers through the Akamai Qualified Computing Partner Program," said Matt Berk, vice president of cloud computing at Akamai. "Pay TV operators and content providers are now able to captivate their audiences with high-quality video and scale it across Akamai Connected Cloud, the massively distributed platform for cloud computing, security, and content delivery."

Looking to a life after co-ax connected video signals has many wondering which path to take—SMPTE or NDI? How can it be we’re at a fork in the road for IP video production?

The simple fact is this: for decades, the evolution of digital video production was a fairly standard and uncompetitive process. The Society of Motion Picture and Television Engineers (SMPTE) took the lead and developed standards which the industry has relied upon for many years. With each evolution new higher resolution formats were supported but continued to use co-axial cable with BNC connectors. If you needed to throw a particularly long distance, then a baseband fibre alternative was also an option. 

So why are we at a juncture where system architects are left making a judgment call when developing new production facilities? The answer is simple: Neither NDI or SMPTE IP is “optimum,” and thus the right choice is much harder to select.

An Evolution
The first important IP format, SMPTE ST 2022, appeared in the last decade and was basically SDI packetized into an IP stream. Though not hugely flexible, it did—and still does, have uses. Then came SMPTE ST 2110; it provided greater capability by separating video, audio and metadata into individual IP streams, allowing designers to create facilities with simplified workflows and significantly less hardware. There are other advantages, too many to mention, but a critical one opened the door to remote production with cameras and production galleries located in different countries with a helpful telco in between; global televised sport never looked back.

The latter part of the last decade saw manufacturers, including ourselves, natively supporting SMPTE ST 2110. The choice was simple, the future was bright… wasn’t it? The reality is that SMPTE ST 2110 demands very high-speed IP infrastructure with cost to match. Away from the highly organized environments of studios and outside broadcast, SMPTE ST 2110’s fibre transport can show its fragility making decision makers nervous. Remember it took years to get location drama to stop using film!

This backdrop has been to the advantage of NDI, emanating from NewTek and now a commercial tech spin-off owned by Vizrt. Its mantra could be summed up as “keeping it copper—IP video should be transported over cheaper Cat5/Cat6 networks.” Here the cables are robust and familiar, and the networking infrastructure doesn’t require $40K of training to manage. 

There is, however, a big “however" the video is compressed. Purists in broadcast engineering dismissed this with a sniff and sharp turn of the head, so NDI sat in the dark longer than it should have. Nearly all routes to the consumer result in video being compressed so why should compression matter further up the workflow? Well, it probably doesn’t, thus we are seeing growing numbers of manufacturers supporting NDI. One of the “cons” of NDI was its limited acceptance but this is becoming historical. 

The compression itself can be confusing as the NDI brand has multiple types. NDI, often referred to as “Full NDI” is based on something known as SpeedHQ supporting variations SHQ2 and SHQ7 (both 4:2:2 video types), resulting in HD bitrates around the 100Mb/s. NDI’s other version HX is H264/H265 depending on whether it is HD or 4K (UHD). NDI-HX is a much lower bit rate and may rule itself out for more “premium” production environments wishing to avoid the cost of SMPTE ST 2110, but the non-HX NDI is pretty damn good, when you consider it wasn’t long ago the audience was “enjoying” interlaced Standard Definition.

More Complex Than It Looks
Those “purists” I mentioned will protest that I’ve oversimplified the issue and have clearly overlooked important factors such as compression latency. They’d be right of course; but the true picture is much more complex. Film, CGI and drama workflows are still going to demand minimal if not zero compression and I strongly suspect that in location production, baseband 12Gb/s SDI will have a long life to come. 

Eventually we will all implement IP and the hardware will only ever be present at the “edge” with all processing, distribution and storage being firmly in the cloud. Which format will win is of course difficult to predict. If the discovery model (how you identify and find multimedia IP devices) for SMPTE ST 2110 and its eventual successor matures, then an uncompressed video, standardized by SMPTE, may take the lead. 

For NDI, taking its place in the competitive market might come down to the commercial approach. Having spent some time talking with senior figures in the NDI team—past and present, I believe it’ll be the fee manufacturers must pay to wear the NDI logo that will dictate the take-up.

We are likely to see a battle over the next five years and the resulting uncertainty will make choices difficult. I won’t make a prediction, but I will say that having previously introduced support for SMPTE ST 2110 in our video technology, we are currently putting the final touches to our NDI support ahead of this year’s IBC.  

WATERLOO, Ontario—SSIMWAVE, a developer of viewer experience QC and automation technology, has been awarded a patent for image and video banding detection. 

U.S. Patent No. No.63/008, 257, "Image and Video Banding Assessment," is intended to help live and VOD streaming services detect and eradicate one of the most difficult-to-solve video quality impairments in OTT streams. The solution is already deployed in production by some of the top streaming platforms to optimize for Banding across billions of hours of viewed content.

The patent describes the method by which the SSIMPLUS products and metric address banding effects in an image or video, including:

• Assessing content for banding impairments;
• Determining the significance and location of the impairment;
• Detecting the banding pixels;
• Creating a banding map; and
• Determining the intensity level of each of the banding pixels.

"Banding's effect on content impacts viewing experiences, of course, but it is particularly frustrating for content creators," said Dr. Abdul Rehman, CEO and Co-Founder of SSIMWAVE. "Establishing a patented method for detecting banding will help preserve the content community's creative intent, driving all of the high engagement and deep emotion that was intended when the content was produced."

Banding is a common compression impairment that usually appears in smooth regions within encoded videos. All popular codecs, such as H.264/AVC, VP9, H.265/HEVC, and AV1 can introduce banding. Content-aware encoding approaches typically introduce more banding as content regions that are prone to banding are generally deemed low complexity by such encoders and are compressed even more than traditional approaches, resulting in even more banding. The impairment is noticeable in both SDR and HDR content, especially on high-resolution displays.

SSIMWAVE measures banding on the SSIMPLUS Banding Score, a 0-100 scale in which 0 means no visible banding is detected and 100 indicates situations in which banding makes the video hard to watch or even unwatchable. SSIMPLUS can detect banding at any point in the delivery chain from ingest to playout.

SSIMWAVE will show banding detection in action at Stand 1.B10 at IBC 2022 or via this video. SSIMWAVE also will showcase its newest live sports streaming benchmarking capabilities at IBC.

"Banding impairment is prevalent across all content, including tentpole titles, streaming services and display devices. We are excited to be partnering with content creators in preserving creative intent by monitoring and optimizing content delivery workflows to eliminate or reduce banding," said Dr. Abdul Rehman, CEO and Co-Founder of SSIMWAVE. "The patented ability of SSIMPLUS to detect banding will help the industry provide a superior video product that will raise the bar on consumer satisfaction."

IRVINE, Calif.—Disney Research and University of California, Irvine, computer scientists have developed an artificial intelligence-enhanced video compression model that they say is capable of competing against established video compression technology.

Researchers working on the project showed an early phase of their AI video compression model at the Conference on Neural Information Processing Systems in December 2019, that yielded less distortion and smaller bits-per-pixel rates than classic coding-decoding algorithms, like H.265, on specialized video content. On downscaled, publicly available YouTube videos, it had comparable results.

The compression model created by Disney Research and UCI is designed to downscale the dimensions of the video using a “variational autoencoder,” a neural network that processes each video frame in a sequence of actions that results in a condensed array of numbers. The autoencoder then tries to undo this operation so that the array contains enough info to restore the video frame.

The next step is for the algorithm to use a deep generative model to guess the next compressed version of an image given what has gone before. It then conducts an operation to encode frame content by rounding the autoencoder’s real-valued array to integers, which are supposed to be easier to store. 

The final step is the application of lossless compression to the array, allowing for its exact restoration. The algorithm is informed by the neural network about which video frame to expect next for greater efficiency, per Disney Research and UCI.

“Intuitively, the better a compression algorithm is at predicting the next frame of a video—given what happened in the previous frames—the less it has to memorize,” said Stephan Mandt, UCI assistant professor of computer science, who first worked on this project while employed by Disney Research. “If you see a person walking in a particular direction, you can predict how that video will continue in the future, which means you have less to remember and less to store.”

Mandt continued: “The real contribution here was to combine this neural network-based deep generative video prediction model with everything else that belongs to compression algorithms, such as rounding and model-based lossless compression.”

Work is still progressing toward a real, applicable version of the video compressor, with Mandt indicating they may need to compress the neural network itself, along with the video.

Few could argue that MPEG-2 has not been a workhorse of professional media compression. Created in 1988, it helped to power the commercial success of DVD disc players and is still used daily by U.S. TV broadcasters as a vital –and mandated—part of their DTV transmission service.

But what if I told you H.264 Advanced Video Compression (AVC) will have a run that matches or exceeds that of MPEG-2? One that ensures AVC will be with us for the rest of our adult lives?

Perhaps, you are raising an eyebrow, smirking skeptically or simply saying to yourself “No way!” But keep an open mind and consider these three reasons why H.264 compression will be with us for a long, long time.

No. 1: Hitting the sweet spot. When you consider factors like processing power and cost, it’s hard to beat H.264 encoding. Sure, H.265, or High Efficiency Video Coding (HEVC), outshines AVC in terms of pure apples-to-apples compression.

A BBC Research & Development blog posting, “H.265/HEVC vs H.264/AVC: 50% Bit Savings Verified,” makes the case for HEVC in the title. The posting discusses subjective comparison testing of the two compression standards based on international recommendations for assessment of video quality.

“The overall average bit rate saving achieved by HEVC compared to AVC of the same subjective quality was found to be 59% as supposed to the 44% gain show with objective quality metrics,” the blog says. It also notes that bit rate savings were greater for larger picture sizes.

While impressive, the blog doesn’t address the cost of encoding. At this point in time, it’s not out of the ordinary for a professional quality HEVC encoder to cost four times as much as a comparable AVC encoder. Although HEVC encoders will likely drop in price with time, as do the vast majority of electronics, so too will AVC encoders.

Thus, in terms of bang for the buck, H.264 encoding occupies, and likely will continue to occupy, the compression sweet spot for a very long time.

No 2: Changing viewing habits. No one could argue that TV viewing hasn’t undergone a major transition. While the living room TV remains important, people now enthusiastically watch television, streaming entertainment and movies on their phones, tablets and laptops—something that would have been unimaginable a few years ago.

Helping to fuel this transition is H.264 compression. Equally adept at powering video streaming on a Raspberry Pi-based IoT device, an iOS platform or Android device, H.264 unlocks a world of possibilities for reaching people with video where they want to consume it.

Further, because of its flexibility, H.264 can be used to simultaneously stream a high school or college basketball game at 360p or 480p while encoding a 720p or 1080p version to be edited later and uploaded to a YouTube channel.

Not only has H.264 evolved into a web delivery format, it is widely used in high-end cameras as one of many supported output formats. It’s also the fuel powering a variety of affordable, yet-powerful production setups used to produce everything from corporate and government events to video game tournaments and club sports—all of which nurture changing consumer viewing habits.

No. 3: Archiving. Perhaps the biggest reason H.264 will remain relevant for decades is its role as an archival format. It’s widespread use for archiving by universities alone ensures H.264 will be around a long time. But it’s also widely used for first-time digitization of content to be archived outside of academia.

All of that HD content shot over the past 20 years –whether 1080i or 1080p— is going to be with us for the rest of our lives regardless of the growing adoption of 4K and maybe 8K. As long as it is, H.264 will remain a vital compression format to archive that content, assuring its longevity one day will challenge and likely surpass that of MPEG-2 as a workhorse compression format.

LOS ANGELES—In the 20-plus years since video compression became available, improvements in compression efficiency, CPU utilization and quality of experience have been many, steady and required a lot of effort.

However, a new approach to improving compression algorithms that leverages artificial intelligence has emerged and is already producing benefits that touch terrestrial broadcasters, OTT service providers, IPTV services and satellite and cable TV operators, said Harmonic Senior Product Manager, encoders, Jean-Louis Diascorn.

Speaking Oct. 21 on the first day of the 2019 Annual SMPTE Technical Conference & Exhibition at the Westin Bonaventure Hotel in Los Angeles, Diascorn discussed how Harmonic is exploiting AI to achieve these improvements.

During his “How AI Technology is Drastically Improving Video Compression for Broadcast and OTT Content Delivery,” Diascorn described a two-step process using AI to optimize the style used for compression as well as the optimization of encoding for frame rate and resolution considerations.

The first step, which can take hours or even days, is an offline learning process, he said. Many test panels are fed into an AI system at this stage to produce a prediction model, which is downloaded into a live encoding system.

“The second step is running the live system… [which] will use the prediction model to produce a better compression for viewing,” said Diascorn.

The Harmonic encoder expert laid out three predictive models: Dynamic Encoding Style, Dynamic Resolution Encoding and Dynamic Frame Rate Encoding.

Dynamic Encoding Style aims to reduce bitrate and maintain quality. Using the two-step process, test files are loaded into the AI system to develop these encoding styles. “Encoding styles are in fact encoding configurations,” he said. After a lengthy time running AI algorithms on the files, the system produces an encoding style prediction model that is downloaded into the live system. While running on the live system, video analysis provides information to the prediction model, which in turn modifies the encoding core, he explained.

A Tier One satellite provider has deployed Dynamic Encoding Style-optimized compression and has achieved about a 20% bitrate reduction, said Diascorn.

Dynamic Resolution Encoding relies on the same two-step process, but in this instance, AI leverages the relationship between movement and resolution in developing a predictive model.

In low movement video, the human eye sees detail; however, in high movement video, such as in sports, the eye is unable to recognize the same level of detail, he explained. “So, it can be useful to have the appropriate resolution,” said Diascorn. “This is what Dynamic Resolution Encoding does, selecting with AI the best possible resolution for [a] given video.”

Dynamic Resolution Encoding is primarily intended for use by OTT services, and the benefits achieved are primarily in CPU savings, he said. “For complex content, we found a 50% CPU savings, and for simple content we found 42% CPU savings,” said Diascorn.

Dynamic Frame Rate Encoding is “a little like the previous one, but the other way around,” explained Diascorn. In video segments with a lot of movement, it is desirable to have a high frame rate to avoid any jitter in the picture. Conversely, in video with little movement, “why spend all the bits?” he asked.

Once again, the two-step method was employed to develop Dynamic Frame Rate Encoding. This time the process was tuned to create a prediction model based on the best frame rate for the amount of movement in video.

Appropriate for satellite, IPTV, OTT, cable and terrestrial broadcasting, Dynamic Frame Rate Encoding saves on average about 30% of frames, which translates into about a 30% savings on CPU usage. “In terms of bitrate savings, we are about 10% for AVC and 5% of HEVC,” said Diascorn.

Diascorn noted that it is possible to combine the AI-optimized predictive encoding models, depending on the application.

“Dynamic Encoding Style and Dynamic Frame Rate Encoding both provide bitrate savings—significant [savings] for Dynamic Encoding Style and [a] better quality of experience is brought by the Dynamic Resolution Encoding,” he said.

“CPU savings is brought by DRE and DFE, and in terms of interoperability the Dynamic Encoding Style and the Dynamic Frame Rate are for all applications, whereas Dynamic Resolution Encoding is mainly for HD OTT.

The implementation of AI to enhance the performance of compression technology is in its early stages, he noted. “We have an exciting future ahead of us because we are just starting,” he said.

LAS VEGAS—Compression, image processing and security specialist intoPIX will introduce the JPEG-XS standard with a demo of its TICO-XS FPGA and ASIC IP-cores at CES 2019, Jan. 8-11, in Las Vegas.

The company, which will also unveil CPU and GPU software development kits for the standard, co-developed JPEG XS based on intoPIX’s TICO RDD35 codec. It foresees TICO-XS as being an important enabler for development of video devices because it solves problems facing CE manufacturers, such as how to manage more and better pixels, realize cost and power savings, simplify connectivity and preserve quality with no latency.

For 4K and 8K, JPEG XS can carry video over existing bandwidth with lossless quality and imperceptible latency, the company said. The standard also saves on cost and power in regards to in-device video transport by reducing internal links and memory.

It has undergone rigorous testing by the ISO JPEG committee and has been shown to provide fully transparent quality at compression ratios of 1.5 bits per pixel ~ 16:1 for TV, the company said.

See intoPIX at CES 2019 booth 1129 at Westgate Paradise Center.

More information is available on the company’s website. 

For all the latest news from the 2019 CES, visit TWICE’s CES Hub.

To access all the latest CES news on Twitter, follow #MyCES2019.

Cobalt Digital is one of the recipients of the 70 Annual Technology & Engineering Emmy® Awards that will take place in partnership with the National Association of Broadcasters (NAB), at the NAB Show at the Wynn Encore Hotel and Spa on Sunday, April 7, 2019 in Las Vegas, NV.

Cobalt was one of the companies selected to receive the Emmy ® Award for “Pioneering Reliable Transmission Method for Live Contribution and Distribution TV Links”. This type of technology allows the use of the Internet as a cost-effective means of live, low-latency contribution and distribution for broadcast content.

A number of Cobalt products include the option for reliable delivery over the Internet, developed by Dr. Ciro Noronha, Cobalt’s Director of Technology and one of the pioneers of this field. Dr. Noronha, who holds a Ph.D. in Electrical Engineering from Stanford University, said, “With the advances in compression technology (which bring the data rate requirements down) and in Internet infrastructure (which bring the network capacity up), it has become technically possible to use the Internet as a high-quality, low-latency contribution and distribution link. The protocols used in the Cobalt products complete the picture and make it a reality.”

“Cobalt is a firm believer on open standards”, added Dr. Noronha. “This technology is now available using the Reliable Internet Stream Transport (RIST) protocols from the Video Services Forum.” Cobalt Digital is one of the contributors to the RIST standard.

“Cobalt has been a supplier of signal processing solutions, to the broadcast industry since 1997, and we are pleased and honored to be recognized with a Technology and Engineering Emmy Award”, stated Gene Zimmerman, President of Cobalt Digital.

About Cobalt Digital

Cobalt Digital Inc. designs and manufactures award-winning 12G/3G/HD/SD and IP-based conversion, terminal, throwdown, and multiviewer technology for the live production and broadcast television environment. As a founding partner in the openGear® initiative, Cobalt offers a full range of openGear video and audio processing card solutions for applications such as closed-caption compliance monitoring, OB production, master control, HD news production, signal transport, audio loudness, and color correction. Cobalt’s Blue Box Group™ line of interface converter boxes streamlines and simplifies a wide range of 12G/3G/HD/SD and IP-based conversion and signal transport tasks. The company’s multi-image display processors enable multiviewer capabilities in the most demanding studio and remote broadcasting environments. Distributed through a worldwide network of dealers, system integrators, and other partners, Cobalt Digital products are backed with a five-year warranty. More information is available at www.cobaltdigital.com .

LAS VEGAS—If you were a broadcaster walking the aisles of the 2018 NAB Show earlier this month in Las Vegas, no one could blame you if you focused much of your attention when it came to video compression on HEVC, or its extension SHVC, which will help to power Next-Gen TV.

However, high efficiency video coding as implemented by ATSC 3.0 wasn’t the only compression development that promises new efficiencies for those in the M&E industry.

The Fraunhofer Institute for Integrated Circuits IIS used the Las Vegas venue to announce the first JPEG XS codec implementation for professional video. The research institute demonstrated its new codec as an input format for post-production.

Specifically, Fraunhofer showed JPEG XS as a software plug-in for Adobe Premiere Pro CC, providing real-time playback of 4K 60p UHD video in a Windows environment using an NVidia graphics card to accelerate decoding.

[Read: A discussion with Fraunhofer on what’s ahead in compression] 

Fraunhofer is positioning its new codec as a solution to the impending train wreck it sees when the ever-growing data demands of higher res video and HDR meets the limitations of coax cables in studios. Exacerbating the problem is the desire of production companies to move away from specialized switches and cable for video transport to commercial off-the-shelf alternatives that move IP packets via standard Ethernet infrastructures.

The JPEG XS codec, which is being standardized by the JPEG committee for IP workflows in studios, local video networks and VR/AR applications (ISO/IEC SC29 WG1), enables high-res video transfer via standard Ethernet and other wired connections, Fraunhofer says.

It is optimized for use with mezzanine compression applications involving high-res video to be transferred over limited bandwidth or when limited computer resources are employed. JPEG XS is also designed to take advantage of the SMPTE ST 2110 transport layer.

LAS VEGAS — Aperi Corp. will display native IP sports technology for live remote production at the 2018 NAB Show, along with a new SMPTE 2110 network security application.

With a focus on live IP Media Function Virtualization (MFV) solutions, the company will present case studies profiling how top U.S. sports leagues and others use its virtualized technology to link venues and production bases to form practical, live remote production workflows, as well as how it can also benefit complex distribution applications.

[Read: Aperi, Video Clarity to Test H.264 Compression at VidTrans 2018]

Aperi will also show a new SMPTE 2110 native firewall app (with up to five in a single RU) to support 128 bidirectional flows of content per microserver; a suite of 2022 gateway and standard codecs, including H.264 and J2K; and a new TICO 4K compression app. They will also show a new single-RU multiviewer app that monitors hundreds of IP inputs, while creating five independent outputs in IP, SDI or HDMI.

Visitors will also see the latest V-Stack, now with expanded functionality to support multiple processes, such as feed monitoring, content logging, license management, as well as third-party orchestration and control systems. Aperi’s A2101 real-time dense network-edge media processor and gateway server, applicable to mobile production facilities and local-loop networks, will also be featured.

Aperi Corp. will be in booth SU11710.

NEW YORK--With giants such as Netflix, Amazon, Facebook, Google and Microsoft in its corner, the Alliance for Open Media (AOMedia) has released the 1.0 version of AV1, a next-gen, royalty-free codec that claims to have a 30 percent bit-rate efficiency edge over current-gen technologies like VP9 and HEVC.

The consortium is releasing AV1 in the hopes that it will deliver improved bandwidth efficiencies amid the move to video formats with more pixels (like 4K and 8K), better pixels (High Dynamic Range and Wide Color Gamut), as well as the emergence of 360-degree video, virtual reality (VR), augmented reality (AR) apps and services that need to be smooth and are poised to ratchet up bit rate requirements on TV screens, laptops, tablets and smartphones.

Though AV1, which is also claimed to have a 65% bandwidth savings over AVC/H.264, could eventually find itself a home inside set-top boxes and, perhaps even further out broadcast TV signals, it’s expected to initially find adoption on web browsers and other types of OTT streaming devices.

Though AV1, which is also claimed to have a 65% bandwidth savings over AVC/H.264, could eventually find itself a home inside set-top boxes and, perhaps even further out broadcast TV signals, it’s expected to initially find adoption on web browsers and other types of OTT streaming devices.

“Video is changing the way the internet is evolving,” Gabe Frost, the executive director and a founding board member of AOMedia and a principal engineering manager for Microsoft’s operating systems group, said in an interview.

Another chief aim for AV1 and its royalty-free model is to eliminate some of the pricing uncertainty that has enveloped codecs like HEVC, which has been saddled by multiple patent pools and little in the way of uniformity on rates.

The current codec market is “gummed up” due to the cost uncertainty, Frost said, and has likewise caused friction between technology innovators that those that want to charge for patented technologies that can help them.

AOMedia, launched in 2015, was formed with the core idea that the underlying video codec technology had become a commodity, and that a royalty-free model represented the best path forward.

“We all use it; it all gets baked into silicon,” Frost said.

The plan with AV1 then focused on how a new approach, despite being royalty-free, could still “create a viable business model to move the industry forward, with some certainty around the costs of producing these codecs,” he explained.

[Read: Report: HEVC Poised To Become Codec Of Choice]

While that idea was spawned by early meetings with Microsoft and Google, it later expanded to include several other in the ecosystem, including chipmakers, distributors, web services companies and video equipment makers that would be willing to provide their intellectual property under the royalty-free model.

Examples of founding members of AOMedia include Amazon, Apple, ARM, Cisco Systems, Facebook, Google, IBM, Intel, Microsoft, Mozilla, Netflix and Nvidia. Promoter members include Adobe, CableLabs, Bitmovin, Hulu, Vidyo, and Broadcom, among others.

Frost said every member company of AOMedia signs an agreement that licenses the essential technology that’s used in the final AV1 codec on a royalty-free basis. And those that implement it likewise sign a cross-patent license agreement that says they’re free to use AV1 at no cost, but that if they hold any essential patents, those are licensed to other AOMedia members on a royalty-free basis.

“We wanted to make sure we had a licensing infrastructure set up where we could create a durable, royalty-free ecosystem,” Frost said, noting that companies with patents used for codecs such as HEVC and H.264 are free to license those same patents under other terms, such as those that are governed by AV1.

The hope is that the model will accelerate the adoption cycle of VC1 and spark advancements across software, silicon, devices and connectivity and push the market forward for all stakeholders.

“Royalty-free doesn’t mean that people aren’t making money off the technology,” Frost said.

[Read: ITU, ISO Prepare For Next-Gen Video Codec]

As for near-term expectations, Frost said desktop browsers will start to support AV1 later this year, and see it start to show up on devices such as gaming consoles towards the end of 2018 or into 2019.

In a presentation about the release, AOMedia noted that AV1 would be “coming to a screen near you,” with examples that included Twitch, Facebook, Android, Amazon Prime Video, Chromebook, Chrome, Google Play, Netflix, Windows, Xbox One, Daydream (Google’s VR platform), Skype, and YouTube.

“It will be 2020 when we see it available on all new silicon that comes out,” Frost predicted.

Michelle Abraham, senior analyst, media and communications at S&P Global Market Intelligence, said the buy-in among a wide number of companies and players gives AV1 a “good opportunity” to make waves in the video codec market.

She sees its greatest opportunity coming from video streaming and OTT, but doesn’t expect to see it widely used by broadcasters, who typically go with technologies that are tied to international standards bodies.

There’s also a question about whether the latency requirements of broadcasters will be suited to AV1, at least in its initial release.

AV1 will be getting some play at next month’s NAB show in Las Vegas, including demos on April 10 from 2 p.m. to 3 p.m. at the South Upper Hall Destination NXT Stage.

A panel is also slated for April 11 starting at 3:20 p.m. at the North Hall (N257) that will feature Microsoft’s Gabe Frost, Google’s Matt Frost, Intel’s Zach Hamm, Bitmovin’s Stefan Lederer, Microsoft’s David Rudin, and Netflix’s Mark Watson.

This article originally appeared in TV Technology sister publication, Multichannel News

LOS ANGELES—NEWS FLASH: There is a huge shift in viewing of media (TV and movies) from traditional distribution outlets (cable, satellite, broadcast) to the internet, including both fixed and wireless as well as mobile devices…OK, we all know this, just wanted to get your attention.

While the established modes of media distribution are well understood and engineered, governed by open standards (ATSC A/53, SCTE 23 (Docsis), DVB-T, DVB-S, MPEG) the internet is an amalgam of “open standards” (IETF) and quasi proprietary approaches (Apple HLS), as well as a variety of software protocols and applications (web browsers, media players, etc.). Further there is a loose form of registration (ICANN) and operational principles (peering) that govern traffic and routing on the internet. 

This looseness, with a minimal of standardization, has encouraged the development of new feature-rich applications that empower people to consume media content anywhere and anytime they have access to the internet. On the minus side, the “core internet,” which has evolved over time, is not designed to expand at the rate of consumption of video. Unlike other forms of internet traffic, media streaming/downloading requires certain performance criteria:

1. Low latency

2. High end to end bandwidth

3. Low packet loss

4. High storage capacity

ONE TO ONE

One of the key differences between the internet and traditional video distribution is that the internet is based on datagram transmission protocols, not switched circuit technology nor linear channel distribution (cable, OTA, satellite). In datagram transmission protocols, the video content is put into sequential packets of limited size and then each packet is launched over an IP network. Each request for content requires unique packets be sent to that user, thus 1 million viewers means 1 million distinct video streams. Linear channels can support multiple receive points with one common video feed.

Along the way, these packets are buffered and routed to their ultimate destination. Because there is no centralized routing or path assignments, the route of any two packets of the same video stream may or may not traverse the same set of routers and links thus experiencing different transit delays. The IP packet video receiver has to buffer the incoming packets, perhaps re-ordering them, prior to processing (decoding). This buffering introduces additional delay. In contrast, in a linear channel or circuit-switched transmission, the routing of video data is fixed thus the received video packet order is fixed, minimizing buffering and delay. There may be an initial delay in setting up the circuit or tuning to the desired channel, but once the streaming has begun, a minimum of delay will occur from receiving the compressed video data and decoding back to base band image data.

THREE METHODS OF SEGMENTATION

To overcome some of the challenges of data packet transmission, a technique known as segmentation has evolved. There are three competing approaches: Microsoft Silverlight, Apple HLS and Adobe HDS. Basically, the media is encoded in bundles of discrete segments of content length (from milliseconds to several seconds usually) and are then sent as a “package.” Once the first package is received and verified, decoding starts while the next package is being received. As long as the transmission delays are less than the segment content length, the receive buffer is kept from emptying, which prevents stalling the decoder output. 

However this only solves the delay/re-buffer problem. If the end-to-end IP bandwidth is not stable and drops below the criteria of delivering packets faster than the decoder is pulling them out, then buffer underflow will occur causing freeze frames or black frame output. An advantage of the internet is the inherent bi-directional connection; the receiver can report back to the video server statistics on the recovery of packets. If the packet receive rate is too slow for the current encoding bitrate, the server can “switch” to a lower bitrate. For the segmented protocols, this usually means swapping bit rates at segment boundaries. Likewise, if there is more than sufficient bitrate, the server can raise the bitrate and improve the overall video quality. For circuit-switched or linear channel-based delivery, fixed bandwidth is guaranteed along with a fixed transmission path.

From a practical point of view, given that encoding is processor-intensive and each bitrate rendition require more storage at the server, video servers have a fixed number of encoded bit rates available, typically four. The server selects the appropriate bitrate segment for the viewer encoded in the bitrate below the reported receive bitrate. A further refinement of this approach, called CAE (Content Aware Encoding) optimizes the bitrate per segment, with multiple tiers of quality performance. For a given quality, the bitrate will fluctuate between segments; if the available bitrate is insufficient, the server will select a tier (or ladder) of less quality/lower bitrate.

Other techniques to manage video delivery over the internet include transcoding at the edge (where the origin video is re-encoded to fit down local IP connections), multicast (similar to a linear channel approach), shared caching (where receivers share data streams) and peering (similar to Bit Torrent). 

HANDLING THE CROWDS

The reality of internet delivery of media content is a complex topic. There are physical constraints, such as bandwidth, storage, processing, as well as electrical power (it is estimated that in the U.K., 16 percent of all power generated is used by the internet data centers). While there are traffic flow models for the internet, media streaming does not fit well into these models, so we have to use empirical measures of how well the internet is handling video. We know many “peak event” viewings of live content (season premieres, live sporting events, etc.) cause either slowdowns or disruptions to the video feeds. The causes are manifold—maybe over-subscription of the origin server or if the client side connectivity degrades or the internet core routing is over-taxed or if the ISP gateway is overloaded. CDNs in part, mitigate these peak flows and attempt to route around bottlenecks. But the root cause is larger audience sizes coupled with increasing bandwidth requirements of video content (HD, 4K, high frame rate). Since each viewer receives a unique media stream, as the viewership grows, the total internet bandwidth grows. 

The current estimate is that around 70 percent of all internet traffic is due to media consumption. This means that everything else (email, financial transactions, web browsing, etc.) is 30 percent. However, it is interesting to note that the 70 percent represents many duplicative feeds—if 1 million people are interested in viewing the World Champion Darts competition finals, then that represents 1 million streams at some bitrate (1Mb/s to 4 Mb/s typically). But at the same time, there can be many other instances of stream viewing with an audience of a single viewer to over 1 billion viewers. While CDNs can scale and create multiple delivery pipes—and multiple origin servers can be made available, thus spreading the load over a broader set of servers and data connections—there is ultimately a finite resource of bandwidth and processing nodes. 

THE BOTTLENECK

Much like the traffic on the 405 in Los Angeles, there are finite resources (lanes) that can carry cars. We can carpool or reduce the size of the lanes (compression) but this solution has a linear effect on the problem while the growth of internet video traffic is exponential.

Another way to look at this is Compound Annual Growth Rate (CAGR). Video streaming is growing at a CAGR of 30-35 percent; video compression over the last 25 years has improved around 7 percent CAGR (halving the video bitrate every 10 years). Unless video compression has a major breakthrough (15:1 improvement in compression efficiency) compression alone cannot solve the internet video bottleneck. (Note that 15:1 compression improvement does not take account of the increase in bitrate due to increasing video format size or frame rate.) More and more content delivered over the internet is HD and above (4K, eventually 8K), which needs more bandwidth than the current mix of internet video streams.

The solution has to be found in alternate video protocols for distribution of video over IP. The 1:1 relationship between the viewer and a unique video stream is unsupportable with the growth of consumption of video over the internet. The challenge is the desire for personalized delivery, one that can be controlled by the viewer using many different devices (mobile, fixed, PC), connection types (wired and wireless) and viewing conditions. Some proposed solutions are variants of multicast protocols, but multicast means “appointment TV” viewing or having to record each multicast feed locally in an appliance. However this does not support “impulse viewing” if that multicast feed was not previously subscribed to by the viewer. Perhaps a hybrid approach where there is a “linear feed” multicast to millions and 1:1 individual streams for those that either come late to the party or want a unique viewing experience (such as a “cut down” version of a live event, minus all the game breaks). Maybe.

Well let’s hope that 2018 will be the year that someone figures out a solution to the bottleneck.