Back in the day, when you produced and/or distributed content mostly through one broadcast or cable channel, making certain everything looked and sounded the way it should was easy. You simply checked it at the last point where you touched it and then verified it via a return feed. Oh, that it was still that easy.
Now, thanks to content distributed through multiple distribution pipes and consumer-
side reception devices, you have to check the same piece of content multiple times. Think of it as creating a single drink but having to deal with a wide range of pipes, pressure and different faucets. Oh, and on top of that, different sizes and shapes of glasses. If you are not careful, this economic necessity becomes a QoS nightmare.
The options used to be simple: Black and white or color; mono or stereo. Monitoring, particularly at the local level? Easy peasy: a decent consumer TV set and an antenna or cable feed.
Know Where to Look
How have you and those in your organization or at client locations adjusted to the ways that even a single program or piece of content is viewed? How are you and your team assessing and then assuring end user or viewer image and sound quality?
Here’s a bit of guidance from the consumer side of things: No matter who is at the root cause of an issue, the one who touched it last is always blamed by the end user. They don’t care about who or what is upstream from you; if it breaks, it is your fault and you look bad. In the age of social media, you can’t afford that, so here are some suggestions on how to avoid it.
First, make certain you are properly monitoring any error conditions in the full streaming workflow, no matter whose side it is on. Encoding, packaging, CDN delivery, DRM, caption timing, audio and anything related to ad insertion, such as SCTE-35 Ad Break Market Integrity. That last one has to be “right on the money”—if not, there may not be any money! You also need to ensure your own internal content management system matches the needs of the various distribution feeds and services. A metadata mapping mismatch may cause your content to be rejected.
Even when you’re comfortable with how content left your facility, you must check the quality against the matrix of the streaming services/distribution channels and the multiple consumer devices.
To do that, you need to consider two things: First, create a “streaming QC station.” The second and equally important part is to create checklists and score reports to validate the results.
Remember that the same feed may not look and sound the same on different services or on different devices. Beyond that, each generation of the same device brand and model may impact how a feed looks and sounds. You are no longer monitoring a single off-air or return feed.
Devices Matter
Start with one each of the Roku, Fire TV, Apple TV 4K and Google TV 4K external streamers with the latest OS updates. Then, look on sites like eBay or ask around with colleagues for older models that no longer get updates. That covers you from missing errors that might not be revealed on current devices or OS drops.
For monitoring, have two midpriced connected TVs (CTVs). Why two? Because Roku, Fire TV and Google/Android TV can be monitored with external streaming dongles, but the captive TV manufacturer OS systems (LG’s WebOS, Samsung’s Tizen and Vizio’s OS/SmartCast) can’t. Depending on which HDR formats your consumer-facing feeds use, make sure you can check the current HDR formats beyond the now almost standard HDR-10, including Dolby Vision, HDR-10+, HLG and Advanced HDR.
When it comes to switching between the two sets, instead of using an HDMI switcher, get a midline AVR with at least six HDMI inputs and two HDMI outputs. That will also let you monitor multichannel audio presentations. If you need more HDMI inputs, simply connect some of the streaming devices directly to the TV sets.
For control, the AVR remote should be able to cycle through everything. However, because you may need to have a bucket of remotes for the TVs and the streamers whose remote codes might not be in the AVR remote, consider a programmable universal remote.
For the more adventurous, experiment with a simple home-device control system or talk to a local CEDIA member residential design/installation firm, as this type of system is what they do for a living.
Test Systematically
To justify the time and expense of the QC station, it is essential you develop a repeatable test sequence to identify when something is wrong and then locate where the fault originates. That will help you correct the root cause for things you can correct.
You can’t be expected to provide tech support for your viewers, but if something is happening outside your control, knowing what is going on allows you to post a message on social media feeds (or, if it is more significant, consider an on-air crawl message). Remember, if you don’t catch errors when the cause may be the viewer’s slow bandwidth or outdated streamers and CTVs, you could become the scourge of the internet.
Capture your daily QC check across all services in a database matrix against how each stream lands from all services and on each device. Remember, the same stream viewed on a Roku might appear differently than on a CTV app or through a different streamer. Keep a record of the download speed and occasionally check it using a smartphone, either directly or by using the phone as a hotspot to the QC station rather than from your facility’s usual broadband access. There are a lot of variables, and any one of them can be the root cause for a single outage or reception issue, even when other streams are perfect.
As you would for any QC/QoS testing, check and record the image quality and color gamut. Is there buffering or occasional freezing? Is there excessive latency from one stream and device to another? Is the sound in sync? Be sure to check that for a few minutes on each view to make sure you don’t have issues with audio/video sync drift when variable frame rate (VFR) encoding is used. Is the audio, particularly multichannel audio, correctly passed through from an app originating in a CTV through to the AVR? Are the captions working properly?
Finally, chart every place your content appears, including channel aggregators. Think of a feed coming not from a station, but from the feeds of local newscasts that are part of a network’s news app. Those, in turn, may be picked up by FAST aggregators like Pluto TV or Xumo.
This may seem a big task, and to some extent it is, or at least can be. However, if you construct your QC station properly, set a logical-but-comprehensive test schema and develop the database for metrics, you don’t need to do everything every day. Create a manageable rotation so that over the course of a week, you check everything at least once with the caveat that during major news, sports or milestone programs, you have someone checking things in real time.
Always remember that no matter what the root cause of a system or technical issue is, and regardless of how far upstream it was from you, the consumer blames the last one to touch it. In the case of streaming content, if it has your name on it, that’s you. Don’t let that “last one to touch it” syndrome hit you in the face.
