Delivering high streaming quality can be more challenging than ever given the wide range of devices and operating systems that video service providers are expected to flawlessly support. As providers look to streamline the quality-assurance process (QA) and deliver outstanding streaming quality, AI-powered video testing and monitoring is transforming workflows. With AI video testing, video service providers can efficiently evaluate video compression, test real-time live content and assess video quality without needing reference streams. AI-driven video testing is supplying providers with valuable insights to improve streaming quality, build brand awareness and drive viewer retention.

The Challenges of Video Testing and Monitoring
The video streaming landscape is rapidly evolving, posing complex challenges for video service software development. In the past, software development followed a more linear approach, with a new release every few months in a controlled update path. Today, agile development has dramatically increased the frequency of updates, requiring weekly—and sometimes even daily —new releases across various platforms and systems. Unlike traditional cable or satellite systems, where stability in encryption and broadcasting minimized the need for frequent updates, streaming video requires constant adaptation.

Modern streaming services are expected to perform identically well on a multitude of devices and operating systems backed by various cloud environments, all of which are constantly evolving. Even the simplest of video app features relies on seamless integration between multiple layers: the application running on the device, which runs on a proprietary operating system, often with multiple cloud-to-cloud service integrations. Each of those components has its own lifecycle and can be updated at any time.

To look at just one set of devices, smart TVs in 2024 run on more than 12 different operating systems, with LG even shipping new updates only for its most recent models and keeping older models on a different software version. Hisense provides five different operating systems for its TVs in addition to the Vidaa operating system it developed in-house. Providers need to find a way to test how their service works on all of these devices and operating systems to deliver optimal quality.

The complexity of testing increases as service providers launch partnerships. Providers like Verizon now offer packaged subscriptions to third-party streaming services like Peacock, which means they must ensure seamless performance not only for their own platform but also across partner applications. A third-party app failure could harm the provider’s brand, pushing users away. As partnerships look to become an even larger part of the ever-shifting market, QA teams face immense pressure to maintain functionality and reliability across a wide range of devices and services.

How AI is Transforming Testing and Monitoring
AI plays a crucial role in transforming the QA testing process by enabling scalable, efficient testing that meets the demands of fast-paced software development. Through automation, AI, and machine learning, companies can test complex systems at a level that would be impossible with manual testing, saving valuable resources and time in an industry pushing for profitability.

An example of AI’s usefulness is in enhancing how video quality is measured and optimized during the encoding process. Netflix’s open-source VMAF model, for instance, allows compression algorithms to evaluate video degradation by comparing the original and compressed versions of a video stream, identifying how much quality was lost. However, because VMAF is computationally heavy, providers often opt for lighter, customized versions, such as pVMAF, which is computed on multiple encoding profiles, to help select the best bit rate quality balance.

This approach relies on the availability of reference streams, which are usually unavailable for live sports or user-generated content. Viewers at home also don’t have a reference stream to assess video quality—they just evaluate the performance themselves. AI algorithms can be programmed to reliably perform this non-reference-based quality assessment on real devices, allowing video service providers to maintain quality control at scale, testing without sacrificing speed or accuracy.

The Role of AI in Testing Live Streaming Events
Real-time AI testing can help providers prevent disruptions during high-traffic live streaming events. The field of research includes anomaly detection and issue prevention, especially on the field of multi-CDN delivery. Providers may choose to disable computationally expensive features, such as targeted ads or additional security layers, to preserve scaling capabilities.

Results From AI-Enhanced Video Testing and Monitoring
AI is also being used to enhance the quality and reliability of targeted ads by reducing failure rates in Dynamic Ad Insertion, which directly impacts revenue. By analyzing and monitoring slate occurrences—instances where an ad fails to load properly—AI helps identify patterns, working collaboratively across development, ad generation and ad provider teams to improve ad placement success.

AI insights have also guided providers toward a balance between visual quality and network efficiency. Delivering “good enough” video quality on mobile networks—consistent with user expectations and competitive standards—enables service providers to avoid excessive data consumption that could cause disruptions or resolution drops. For instance, streaming a video at a 4K resolution consumes seven times the amount of bandwidth as the industry standard of 1080p, when it does not necessarily provide a viewing experience that is seven times better. Optimizing this balance ensures a smoother viewing experience, improving engagement without sacrificing reliability on mobile networks.

Relying on AI-Powered Video Testing and Monitoring
AI is an essential tool for meeting the complex demands of video streaming, allowing QA teams to scale testing and monitor performance in real time. Since streaming platforms and partnerships evolve frequently, traditional testing methods can’t keep pace. AI-driven testing adapts to these rapid changes, simulating user experiences across various devices without requiring deep integration into each app or operating system. Rather than trying to develop new testing technology from scratch, relying on powerful AI-driven testing and monitoring technology allows talented teams to put their time and effort back into their video service. This agile, user-centric AI-powered technology prepares providers to tackle emerging trends and anticipate future challenges, enabling them to focus on enhancing service quality and user satisfaction.

LOUISVILLE, Colo.—The CableLabs subsidiary Kyrio has launched its first Testing as a Service offering, FlexTest, which provides device manufacturers with automated, remote, and flexible test capabilities for the full product development lifecycle. 

The service is designed to help manufacturers speed up the process of certifying devices in an rapidly changing telecommunications, broadband and media landscape. 

Since 2001, Kyrio has certified more than 1500 devices certified, resulting in 2.3 billion units shipped since 2001, 

With FlexTest, customers ship a device directly to Kyrio. Devices are then checked in and connected to a test bench. Through our web portal, customers can easily select and schedule specific tests, and review results in real-time. Additionally, these devices can remain with Kyrio for any future testing, such as software updates. 

The service gives companies control over testing timelines; provides full visibility of the testing process and real-time access to status and detailed results. Users can also pause testing to troubleshoot your device, rather than waiting until testing is complete. 

Kyrio says that this revolutionizes current certification models by reducing friction and allowing customizable and flexible testing at all stages of a product’s lifecycle, thus ensuring validation of hardware and software before and after launch.

For more information visit here. 

My first Consumer Electronics Show was 2006. That year the conversion to digital TV (ATSC 1) was underway and manufacturers of all sizes were showing TV sets of all sizes, as well as converter boxes people could use to receive the digital broadcasts on their existing analog TVs. Fourteen years later, TV sets have been replaced by large displays with streamed programming.

In the limited time I had to walk through the exhibits at this year’s show, I saw media products had taken a back seat to artificial intelligence, autonomous vehicles and sensors, smart home/city, gaming, drones and health products. Hyundai had a big air taxi on display (see Fig. 1), joining the Bell air taxi that attracted interest last year. I didn’t see any mention of over-the-air TV (outside the NextGen TV exhibit), except for one sign touting Samsung’s technology advances, with ATSC 3.0 listed under 2020.

However, the major TV manufacturers are supporting NextGen TV. LG Electronics, Samsung and Sony announced they would have ATSC 3.0 tuners in several of their 8K displays. Because there is limited 8K content, manufacturers are using AI to up convert 2K and 4K content for viewing on the 8K screens. This is important for broadcasters, since data bandwidth for ATSC 3.0 will be precious until ATSC 1 is shut off, making it unlikely there will be any 8K broadcasts and a limited number of 4K broadcasts.

I’m concerned that when people start taking these expensive 8K TV sets home later this year and decide to try out that new tuner, they may be disappointed in the quality of the over-the-air video compared to 4K and the limited 8K streaming content. I expect the quality will improve, just as it did in the years after the rollout of HDTV 20 years ago.

CES attendees could see NextGen TV in action and the plans for the rollout later this year at the NextGen TV exhibit in the Central Hall. The good news is there will be options for receiving NextGen TV without purchasing a 75-inch 8K display.

TESTING NEXTGEN TV

Gaian Solutions, an India-based technology development company, was showing an inexpensive set-top box running GAIATV3.0 middleware on Linux that, with the addition of an ATSC 3.0 USB tuner, was able to receive ATSC 3.0 broadcasts, run applications and display them on a connected display (Fig. 2). The good news is that the STB should be available in quantity for under $50. The bad news is that at this time, the only ATSC 3.0 USB tuner I’ve found will cost much more than that—close to $1,000 individually.

The good news for stations wanting to test many NextGen TV applications is the Gaian Solutions STB and middleware will work over ATSC 1, allowing use of an ATSC 1 USB tuner costing well under $100. A station using the GAIATV3.0 software and STB can roll out demonstrations of guide services, targeted advertising and such using their existing ATSC 1 facility.

Gaian Solutions plans to sell the STBs and receivers in bulk to broadcasters or distributors (with custom branding) and may also have a direct purchase option.

AIRWAVZ REDZONE TVXPLORER

As far as I know, the RedZone Receiver from Airwavz is the only ATSC 3.0 USB tuner available now that anyone can purchase. I met Bonnie Beeman and John Casey from Airwavz at CES to get an update on the tuner and evaluate their latest TvXplorer software.

The RedZone Receiver I have uses an LG demodulator chip and Silicon Labs tuner. The current RedZone Receiver uses a Sony tuner and demodulator, which Casey said reduces power consumption and provides additional data for the TvXplorer application.

The TvXplorer application runs on Windows and shows ATSC 3.0 signal and payload characteristics in expandable tables with the ability to view the xml code directly if needed. During the transition from analog to ATSC 1, Rod Hewitt’s TSReader program gave broadcasters and TV enthusiasts a way to dig into the ATSC signal, see the PSIP data and identify problems with ATSC streams.

TvXplorer provides similar features for ATSC 3.0 with the addition of physical layer data identifying modulation, coding, data rates and SNR. It also has a zero–100 quality scale calculated by comparing the received SNR with the required SNR for the signal’s required SNR for its modulation constellation and coding. This should be useful for field measurements of stations with PLPs (Physical Layer Pipes) with different modulation and/or coding.

I had no problem receiving the Channel 33 ATSC 3.0 signal transmitted at 35 kW ERP from Black Mountain. I used the Walmart Onn indoor antenna I described in my last column and my older RedZone Receiver with the latest TvXplorer software (Fig. 3 shows the setup). The software was running in a Virtualbox Windows 10 virtual machine on my portable laptop.

The signal analysis worked fine, but I ran into CPU limits when attempting to stream the ATSC 3.0 video to VLC or TvXplorer’s built-in viewer. I did not have a chance to try streaming to a player in Linux (outside the VM), which might have worked better. The video streaming worked fine on John Casey’s Surface Windows laptop at the CTA lounge. The ATSC 3.0 signal was very robust.

In testing at the CTA lounge, it was hard to break the reception—collapsing the whip antenna to minimum height didn’t kill it; even unscrewing the whip antenna from the mount (only a stub remaining) didn’t completely kill reception, although there were errors. Figs. 4 and 5 show the setup and some data from my TvXplorer hotel room testing.

The Redzone Receiver and TvXplorer bundle isn’t cheap. The website shows the current price as $1,995. While this is more expensive than a high-end 4K large screen TV, the price is reasonable when compared to alternatives, which often include annual support fees.

Airwavz promises lifetime support and updates at no additional charge. Stations launching their own ATSC 3.0 services will probably want the more expensive options from companies like Avateq and Triveni, but for engineers looking for a tool they can put on their own laptop, take to transmitter sites and use to monitor ATSC 3.0 broadcasts from other stations in the market, the bundle is attractive. TvXplorer is still evolving and future versions may include support for checking and decoding the ATSC STL-IP protocols.

OTHER OPTIONS

I have heard rumors of lower-cost ATSC 3.0 USB tuners coming from Korea and China and also got a glimpse of a ONE Media dongle at last year’s NAB Show. I expect we’ll see other options at the 2020 NAB Show.

One of the companies I’ll be following is BitRouter, which had an ATSC 3.0 STB demonstration in a suite at the Westgate, but unfortunately I was unable to connect with them before they took down the demonstration. Details on the BitRouter ATSC3pro are available at https://bitrouter.com/products/atsc3pro/, as well as a timeline for implementation of various ATSC 3.0 features.

The company said pricing would be announced at the NAB Show. I was told the ATSC3pro uses the Sony chipset and the website says it will be a USB tuner, like the GAIATV3.0. Like Gaian Solutions, BitRouter also offers ATSC 3.0 middleware (visit https://bitrouter.com/products/atsc3pak/ for information on its ATSC3pak).

We’ll have to wait until late this year to see how the NextGen TV rollout proceeds. How many TVs with ATSC 3.0 capability will be in viewers’ homes by the end of the year? Given the initial cost of those 8K sets I expect the number to be small, but the people that buy them will likely be enthusiasts who will want to hook up an antenna and try out the ATSC 3.0 tuner.

Will they find a signal? For those that aren’t in the market for a new TV, the boxes and software from companies like Gaian Solution, BitRouter and Wipro will provide viewers a lower-cost ATSC 3.0 solution, provided the price of ATSC 3.0 USB tuners drops to the point where they are less expensive than a big screen TV.

Is your station planning to launch ATSC 3.0 broadcasts? Are you ready to buy an ATSC 3.0 receiver? Which do you think will arrive first in your town—the Hyundai air taxi or NextGen TV? Contact me atdlung@transmitter.com.

For a comprehensive source of TV Technology’s NextGen TV coverage, see our NextGen TV silo.

MUNICH—A new module in Rohde & Schwarz’ VTx video tester line allows for HDMI 2.0a and HDR testing. This new module offers generator and analyzer characteristics for data rates of up to 18 Gbit/s, as well as functions included in the new HDMI standard, like scrambling. It is backwards compatible with HDMI 1.4, supports HDCP 1.4 and HDCP 2.2, and covers real-time testing and compliance testing in line with HDMI CTS 1.4/2.0.

VT-B2363 HDMI RX/TX 600 MHz

The generator function of the module can be used to edit the dynamic range and mastering infoframe for HDR. It displays the extended display identification data (E-EDID) of the connected sink, including the HDR metadata block. In addition, it permits the playback of customer-specific, uncompressed moving picture sequences. The analyzer function also displays E-DID, dynamic range and mastering infoframe. HDR-CTS tests are available for both the generator and analyzer.

In addition to protocol tests, users can perform electrical tests on HDMI sources for debugging and precompliance purposes, as well as analyzing video and audio content.

The R&S VT-B2363 HDMI RX/TX 600 MHz module for R&S VTS, R&S VTE and R&S VTC video testers is now available.