Last summer I started splitting residences between North Georgia and North Alabama and decided that I would “cut the cord” in Decatur, Ala., because my internet connection was at least a 1,000% improvement over my DSL-barely usable AT&T internet in Gainesville, Ga. As all of you know DSL is over copper phone lines and after 8,000 feet of poorly maintained copper wire, the data stream in Georgia was slow.
I set up shop in Decatur, ordered my coax internet service and bought my first streaming device. I checked my internet speed, plugged my new streaming device into my Sony 4K television and then tuned into a 4K streaming channel. To my surprise the picture and sound was out of sync.
As a sound guy I am particularly annoyed by lip sync, which is not hard to find anywhere you look these days. We experience sound lagging behind visuals in the real world because the speed of light is significantly faster than the speed of sound. Many knowledgeable engineers would say that lip sync issues started with the digital age, but I first experienced a lip sync issue in the analog world.
TRAVELING TWO PATHS
Often audio sync issues begin when audio and video are traveling down two different paths. I first encountered out-of-sync audio and pictures in the early days of wireless cameras in the NASCAR pits. The video went one way; the audio went another way and after a couple of frame syncs on the cameras there was clearly a sync issue.
In the good old days of analog, audio and video issues were obvious and usually easily fixed because the audio was often ahead of the picture and quality-sounding analog audio delays were relatively cheap, plentiful, sounded good and were in the OB van—plus CRT screens were an excellent reference for lip sync.
There is no doubt that digital audio solved and caused many audio problems. Video is compressed to reduce the data per frame, which maintains a unique sequence of pictures. Audio is a continuous signal and is compressed separately from the video. The next time I began to see sync issues were with mismatched sampling rates. In every single piece of digital equipment and editing software there are menus for sampling rates and operation for sampling rate conversions. If there is a mismatch, expect a sync issue.
There are a host of reasons and places where the audio and the video can get out of sync, but ultimately it becomes the problem of the broadcaster or steaming service. As broadcast professionals we must keep an eye on sync because in the broadcasting world there is digital processing that happens all along the workflow, from remote operations to transmission, which can result in a cumulative delay effect. We can only make sure that the audio and video is in sync when it goes to the transmitter. But let’s face it; so much video is consumed through an internet connection and I can safely say that will not change.
Data compression and corrupt data-related problems are prevalent on the internet and this significantly affects the quality of our sacred broadcast product—content. Digital over-the-air television seems to have fewer audio problems, but I am guessing that the few broadcast engineers that are left out there are taking care of large network operations plants and antennas. I have often thought that there has been a huge influx of people with a computer mindset where problem solving has devolved to a hard shutdown and reboot.
STREAMING MEANS BUFFERING
Cutting the cord has been a learning experience where I quickly found out that streaming means buffering. With my DirectTV in Georgia I did not encounter buffering issues, which either stops the stream or decreases the quality of the audio and video to compensate for a reduced data stream. In Alabama I have measured the data stream and my internet averages between 80 and 100 Mbps, which adequately covers 4K video for most of my viewing pleasures—but I can still have buffering delays. And buffering can complicate sync problems. It’s a processing thing.
As I have discovered, I have all the specs, but still encounter buffering. On the consumer side there can be problems with streaming services, internet connection, browsers or audio/video device drivers and interconnect cables; or all of the above can cause audio and video sync problems. Chasing down these problems can be incredibly daunting for the average consumer and taking the chance that you would call customer service and get someone on the line to tell them you have a glitch, the pat answer is unplug, wait a minute and reboot.
Another issue that is particularly annoying is the occasional repeat, missing word or upcut word that occurs when watching TV. I reached out to a friend and TV Tech contributor Jim DeFilippis, who said that some of the bumps and jumps are due to buffers in the transmission path.
If the transmission is error-free and there is no lost or damaged data, everything is fine but often the data is corrupted and lost. If there is a video loss the decoder jumps a frame or repeats a frame of video, but the audio can’t jump ahead so sometimes audio gets repeated or words get cut out or upcut.
If the audio data is then damaged over time, the audio and video drift apart and, ideally, when the A/V delay reaches objectionable tolerances the decoder should reset the buffer and start over. If for some reason the buffer does not reset then lip sync will persist until a reset.
The ATSC 3.0 specifies that the audio and video are embedded together and this is where I think the old saying fits: “It is fine leaving here” applies. As professional broadcast engineers we already know that there are a lot of things that can go wrong with audio sync.
Audio delay is everywhere and audio sync must start from the beginning to the end. Broadcast engineers understand tolerances of acceptance but many of them have retired. Perhaps they should come back and teach a couple of things to the computer jocks.
Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production. He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com.
