COOKHAM DEAN, U.K.—Hitomi Broadcast has announced that it supplied a full set of its synchronization tools to MMC Studios in Cologne. The Hitomi kit is a critical part of the delivery of some of Germany’s most popular live television programs, the companies said. 

MMC Studios is the leading television and film studio provider in Germany, Hitomi reported. Its studios are responsible for some of Europe’s most popular programs, including live shows like “Let’s Dance” (the German version of Dancing with the Stars/Strictly Come Dancing), “The Masked Singer” and “The X Factor”. The studios are fully equipped for 4k HDR Ultra HD production. 

“On a show like `Let’s Dance’, we produce in HDR Ultra HD, and deliver that and a SDR HD version to CBC, RTL’s broadcast center,” said Andreas Albert, head of technical operations at MMC Studios. “The show can have as many as 20 or 30 HD/4k and color space conversions in production, as well as the final output, and with different latencies in different converters, we have to work hard to keep everything in synchronization.” 

The solution is to use Hitomi MatchBox to align every stage of the process. MatchBox Glass, the iPhone app version of the signal generator, is used on every camera into the switcher. A MatchBox Generator provides test signals for video processing chains, and the MatchBox Analyser provides absolute precision in measurement of lip sync, phase coherence and channel identification. Both the generator and analyser are 4k ready with 12G-SDI connectivity, the companies reported. 

The unique Hitomi test sequence tests every element of the circuit, and the Analyser calculates the difference between what it knows should be there and what it actually receives. For MMC, this means that every delay in every part of the production chain is precisely calibrated and can be calculated. When the signals leave the building on a live broadcast, they are precisely in sync, Hitomi explained.  

“Our reputation depends on us delivering to the very highest standards, especially now that Ultra HD is growing in importance,” MMC’s Albert said. “Lip sync errors are very obvious and irritating for viewers, and we absolutely rely on Hitomi as the industry standard for addressing this issue. Hitomi allows us to send test signals to our customers and ensure that they can deal with any potential lip sync issues before they become a problem. It's an indispensable tool for us, and we simply couldn't operate at the level of quality that we do without it." 

The MatchBox equipment was supplied to MMC by VIDI as part of its network management system. Sales director of VIDI Bernd Meinl said “MMC is a really busy operation, with studios constantly in use. They need the equipment to be configured and set up really quickly and reliably, and the Hitomi system provides the vital alignment quickly and absolutely reliably.” 

Last summer I started splitting residences between North Georgia and North Alabama and decided that I would “cut the cord” in Decatur, Ala., because my internet connection was at least a 1,000% improvement over my DSL-barely usable AT&T internet in Gainesville, Ga. As all of you know DSL is over copper phone lines and after 8,000 feet of poorly maintained copper wire, the data stream in Georgia was slow.

I set up shop in Decatur, ordered my coax internet service and bought my first streaming device. I checked my internet speed, plugged my new streaming device into my Sony 4K television and then tuned into a 4K streaming channel. To my surprise the picture and sound was out of sync.

As a sound guy I am particularly annoyed by lip sync, which is not hard to find anywhere you look these days. We experience sound lagging behind visuals in the real world because the speed of light is significantly faster than the speed of sound. Many knowledgeable engineers would say that lip sync issues started with the digital age, but I first experienced a lip sync issue in the analog world.

TRAVELING TWO PATHS

Often audio sync issues begin when audio and video are traveling down two different paths. I first encountered out-of-sync audio and pictures in the early days of wireless cameras in the NASCAR pits. The video went one way; the audio went another way and after a couple of frame syncs on the cameras there was clearly a sync issue.

In the good old days of analog, audio and video issues were obvious and usually easily fixed because the audio was often ahead of the picture and quality-sounding analog audio delays were relatively cheap, plentiful, sounded good and were in the OB van—plus CRT screens were an excellent reference for lip sync.

There is no doubt that digital audio solved and caused many audio problems. Video is compressed to reduce the data per frame, which maintains a unique sequence of pictures. Audio is a continuous signal and is compressed separately from the video. The next time I began to see sync issues were with mismatched sampling rates. In every single piece of digital equipment and editing software there are menus for sampling rates and operation for sampling rate conversions. If there is a mismatch, expect a sync issue.

There are a host of reasons and places where the audio and the video can get out of sync, but ultimately it becomes the problem of the broadcaster or steaming service. As broadcast professionals we must keep an eye on sync because in the broadcasting world there is digital processing that happens all along the workflow, from remote operations to transmission, which can result in a cumulative delay effect. We can only make sure that the audio and video is in sync when it goes to the transmitter. But let’s face it; so much video is consumed through an internet connection and I can safely say that will not change.

Data compression and corrupt data-related problems are prevalent on the internet and this significantly affects the quality of our sacred broadcast product—content. Digital over-the-air television seems to have fewer audio problems, but I am guessing that the few broadcast engineers that are left out there are taking care of large network operations plants and antennas. I have often thought that there has been a huge influx of people with a computer mindset where problem solving has devolved to a hard shutdown and reboot.

STREAMING MEANS BUFFERING

Cutting the cord has been a learning experience where I quickly found out that streaming means buffering. With my DirectTV in Georgia I did not encounter buffering issues, which either stops the stream or decreases the quality of the audio and video to compensate for a reduced data stream. In Alabama I have measured the data stream and my internet averages between 80 and 100 Mbps, which adequately covers 4K video for most of my viewing pleasures—but I can still have buffering delays. And buffering can complicate sync problems. It’s a processing thing.

As I have discovered, I have all the specs, but still encounter buffering. On the consumer side there can be problems with streaming services, internet connection, browsers or audio/video device drivers and interconnect cables; or all of the above can cause audio and video sync problems. Chasing down these problems can be incredibly daunting for the average consumer and taking the chance that you would call customer service and get someone on the line to tell them you have a glitch, the pat answer is unplug, wait a minute and reboot.

Another issue that is particularly annoying is the occasional repeat, missing word or upcut word that occurs when watching TV. I reached out to a friend and TV Tech contributor Jim DeFilippis, who said that some of the bumps and jumps are due to buffers in the transmission path.

If the transmission is error-free and there is no lost or damaged data, everything is fine but often the data is corrupted and lost. If there is a video loss the decoder jumps a frame or repeats a frame of video, but the audio can’t jump ahead so sometimes audio gets repeated or words get cut out or upcut.

If the audio data is then damaged over time, the audio and video drift apart and, ideally, when the A/V delay reaches objectionable tolerances the decoder should reset the buffer and start over. If for some reason the buffer does not reset then lip sync will persist until a reset.

The ATSC 3.0 specifies that the audio and video are embedded together and this is where I think the old saying fits: “It is fine leaving here” applies. As professional broadcast engineers we already know that there are a lot of things that can go wrong with audio sync.

Audio delay is everywhere and audio sync must start from the beginning to the end. Broadcast engineers understand tolerances of acceptance but many of them have retired. Perhaps they should come back and teach a couple of things to the computer jocks.

Dennis Baxter has spent over 35 years in live broadcasting contributing to hundreds of live events including sound design for nine Olympic Games. He has earned multiple Emmy Awards and is the author of “A Practical Guide to Television Sound Engineering,” published in both English and Chinese. He is currently working on a book about immersive sound practices and production. He can be reached at dbaxter@dennisbaxtersound.com or at www.dennisbaxtersound.com.

Hitomi Broadcast, manufacturer of MatchBox, the industry’s leading audio video alignment technologies, today announced that Viacom International Media Networks Media and Technology Services (MTS) used MatchBox for the 2019 MTV Europe Music Awards in Seville, Spain, on 3 November.

Viacom International MTS deployed MatchBox’s sync toolbox for the red carpet, main award show, backstage, and after-show party feeds, televised on 100+ channels worldwide from the FIBES Conference and Exhibition Centre in Seville. Hosted by American singer, songwriter, and actress, Becky G, the show included a special headline performance, filmed by Viacom International the previous night at the MTV World Stage at Plaza de España in Seville, by ‘Best Rock’ nominee, Green Day.

Viacom International MTS Global Lead Systems Engineer for Studio and Production Technology, Matt Okotie, said, “Using MatchBox, I can sync signals in under a millisecond to ensure that before we go to air, lip sync is correct. This is crucial for any production, but particularly for a live, globally distributed event, so you have to be absolutely sure you get it right. That’s why I leave it to MatchBox. Basically, if MatchBox reads ‘zero’, you know that lip sync is perfect.”

Viacom International MTS used MatchBox to sync its 4K/UHD feed from Seville to London, and then on for a global sync to all the other MatchBox readers around the world.

According to Okotie, “We send the synced 4K/UHD feed to HQ in London, and then down convert that feed to HD and send both UHD & HD for onward distribution. Our MatchBox readers at Viacom and other global broadcast locations pick up both signals. That process is straightforward because MatchBox is backwards compatible with all of our existing technology; and as far as stability, it’s rock solid.”

MatchBox is a favourite of broadcasters worldwide for ensuring the timing of live news, sports, and events. It is a state-of-the-art toolkit that streamlines live broadcast synchronisation and, in addition to intuitive identing; rapid lip sync measurement; audio coherence; and video timing, MatchBox is also a 4K test pattern generator.

Hitomi Broadcast Director Russell Johnson said, “High profile events have emerged as a specialty for MatchBox. Getting lip sync wrong, perhaps most noticeable in live music, can be disastrous on multiple levels, but by deploying MatchBox sync generators and readers, the band can play on without consequence.”

###

About Hitomi
Hitomi is a professional television broadcast equipment manufacturer located in the UK. It's flagship product 'MatchBox'​ solves lip-sync, coherence, line identification, audio level monitoring problems. Hitomi's engineering team has many decades of experience in professional broadcast manufacture, developing world-leading products to exacting standards. For more information, visit www.hitomi-broadcast.tv

Company contact:
Anna Cox
enquiries@hitomi-broadcast.tv
+44 (0)1753 208803

Media Contact:
Jennie Marwick-Evans
Manor Marketing
jennie@manormarketing.tv
+44 (0) 7748 636171

About Viacom
Viacom (NASDAQ: VIAB, VIA) creates entertainment experiences that drive conversation and culture around the world. Through television, film, digital media, live events, merchandise and solutions, its brands connect with diverse, young and young at heart audiences in more than 180 countries.
For more information on Viacom and its businesses, visit http://www.viacom.com. Keep up with Viacom news by following it on Twitter (twitter.com/Viacom), Facebook (facebook.com/Viacom) and LinkedIn (linkedin.com/company/Viacom).

Hitomi Broadcast, manufacturer of MatchBox, the industry’s leading audio video alignment toolbox, is partnering with Broadcast Wireless Systems (BWS), manufacturer of the ultra-low latency BWS NanoPRO TX camera-back transmitter, to showcase remote production capabilities featuring Hitomi’s new ‘Matchbox Glass®’ iOS app at IBC 2019.

Using a Sony PXW 500 camera on loan from Presteigne Broadcast Hire and two of Hitomi’s own, fixed cameras, Hitomi will deploy BWS’s NanoPRO TX and associated RF receiver to demonstrate MatchBox Glass’s real-world ability to instantly measure latency and provide fast, accurate lip-sync and alignment of multiple cameras.

Hitomi Broadcast Director Russell Johnson said, “This is not just a demonstration of how to save time and money with MatchBox Glass, we’re highlighting how the latest remote production techniques and technologies can provide numerous advantages for reporters in the field.

“As broadcasters need simple and reliable RF transmitters to move content, they similarly need a quick and accurate way in the field to ensure sync, including for multicamera productions. So long as they have a MatchBox Glass-equipped iPhone or iPad, production teams can now easily measure and correct for latency differences between cameras on site, which is especially useful for remote productions.”

BWS Director Stuart Brown added, “We have an excellent and ongoing working relationship with Hitomi and are delighted to have this additional opportunity to demonstrate the various benefits of what BWS NanoPRO TX offers to the production community.

“Many aspects of technology are rapidly converging to enable remote productions to get faster, more economic and, ultimately, more exciting for consumers. What we are demonstrating with Hitomi are extremely versatile systems that can be readily deployed by broadcasters worldwide to ensure quality transmission of live news, sports, and events, guaranteed by fast, high-quality synchronisation, all of which are essential for modern productions.”

MatchBox Glass and its companion, MatchBox Analyser, are being introduced along with the remote production demonstration at IBC 2019 on Stand 2.C11 in Amsterdam from 13-17 September.

###

About Hitomi
Hitomi is a professional television broadcast equipment manufacturer located in the UK. It's flagship product 'MatchBox'​ solves lip-sync, coherence, line identification, audio level monitoring problems. Hitomi's engineering team has many decades of experience in professional broadcast manufacture, developing world-leading products to exacting standards. For more information, visit www.hitomi-broadcast.tv

Media Contact:
Jennie Marwick-Evans
Manor Marketing
jennie@manormarketing.tv
Phone: +44 (0) 7748 636171

About Broadcast Wireless Systems
BWS has decades of experience in the wireless broadcast area. As well as operational experience the team are product designers with extensive in-house mechanical, software, hardware and FPGA experience.

Company Contact:
Stuart Brown
stuart.brown@broadcastwirelesssystems.com
Phone: +44 (0)1376 390647

Stand 2.C11, IBC 2019, RAI, Amsterdam: Hitomi Broadcast, manufacturer of MatchBox, the industry’s leading audio video alignment toolbox, will launch ‘MatchBox Glass’, a new and free iOS App for near instant lip-sync alignment of mics and cameras at IBC this year.

Hitomi Broadcast Director Russell Johnson said, “Synchronisation is often one of the last items on a field production checklist and can happen right up to going live or on-air. Often, it can be discovered too late that there are sync issues, which can be costly and, at the very least, embarrassing to resolve at the last second.

“MatchBox Glass enables quick and reliable camera line-up into truck, remote production facility or in newsgathering applications. By simply holding up an iPhone or iPad, running the Glass App in shot, allows our analyser to instantly verify camera and mic timing alignment. Glass reduces a time consuming and error prone process to a matter of seconds.”

MatchBox is already used by broadcasters worldwide to ensure the timing of live sports, news and events, but until now has not been able to measure the paths through cameras and mics themselves. By putting a perfectly timed source in view, the analyser can measure exactly what delays have occurred to the pictures and sound. Not only that, you can point two cameras at the same iPhone and measure the latency difference between them. This is especially useful where significant delays through wireless links are involved. MatchBox Glass offers greater convenience and flexibility by being a free, downloadable App for anyone using an iOS-based device. Smarter and more accurate than a clapper board and removing the subjective nature of a tap test.

The additional option for its companion MatchBox Analyser is also being introduced at IBC. The MatchBox Analyser tool is typically situated in a master control room, studio or OB vehicle. It looks and listens for the specific Hitomi video and audio signatures produced by the hardware MatchBox Generator or MatchBox Glass App. The measurement performed by the analyser is very fast - just a few seconds - and highly effective, vastly speeding up the process and ultimately helping to eliminate the headaches and frustrations of last-minute lip-sync issues.

Johnson added, “We made the MatchBox Glass free because we didn’t want to penalise independent operators who can benefit from its accuracy and power. There is a licence requirement, but it’s paid for at the receiving end by those who want to acquire the synced content.”

Also being highlighted at IBC in addition to its multiple sync and analysis capabilities, MatchBox is also a 4K test pattern generator, which is critical for many contemporary broadcast operations. Hitomi’s extensive experience and knowledge of swiftly establishing and accurately timing a reliable link to base, for all resolutions and delivery platforms, is evident throughout the design of all Hitomi products which are uniquely engineered to make the synchronisation and alignment process as quick and painless as possible.

###

About Hitomi

Hitomi is a professional television broadcast equipment manufacturer located in the UK. Its flagship product 'MatchBox'​ solves lip-sync, coherence, line identification, and audio level monitoring problems. Hitomi's engineering team has many decades of experience in professional broadcast manufacture, developing world-leading products to exacting standards. For more information, visit www.hitomi-broadcast.tv

Company Contact:
Anna Cox
enquiries@hitomi-broadcast.tv
+44 (0)1753 208803

Media Contact:
Jennie Marwick-Evans
Manor Marketing
jennie@manormarketing.tv
+44 (0) 7748 636171

Example of a lip-sync test signal.
LOS ANGELES—So we’ve all been there, watching a TV show and the actor’s lips are moving but the words are just not coming out at the right time. Sometimes a channel change or re-boot can fix this, but sometimes not. Frustrated, you give up on that program and ask why isn’t anyone doing anything about lip-sync problems!

To begin, let’s go back in history. Before TV, there were the movies, which originally didn’t have ‘sound,’ but did have music accompaniment. When sound was introduced, the production of sound was separate from picture. To insure that the sound matched picture, the ‘clapper’ slate was developed to provide a ‘sync’ point. Still in practice today, the ‘clap’ provides a method to insure the sound and picture are together in time.

However, when TV broadcasting started, there was no post production, and thus all programs were transmitted live, with the audio and video captured in real time and transmitted to the home without any significant path delays (some delay due to modulation and perhaps a part of a video line, but nothing significant).

Videotape recording with audio became available in the late ‘50s. One of the many attributes of videotape recording is the ability to alter the video to audio timing. However, the videotape machine design goal was to keep audio and video in time, so analog videotape machines were designed for zero offset between audio and video.

Later, digital technologies were adopted by television. The first significant digital device that affected the A/V timing was the digital frame sync. Designed to align an external video signal to a television plant, there was no thought about delaying audio. At first, the video delay was one to two frames, which means the audio led the video by about 33 to 66 mS. The ITU (BT 1359) has defined acceptable range of audio-to-video offset of +50mS to -120mS (+ means audio leads video, - audio lags video). Thus, a two-frame (66mS) delay of video is detectable.

DELAYING AUDIO
The first solution was to add fixed audio delays to match the video delay (usually two-and-a-half frames of delay) so the audio would always lag video, which is more tolerable to the viewer. This situation was satisfactory until additional digital devices came into use in television production, such as Digital Video Effects (DVE). Some DVEs have four or more frames of video delay, which definitely can cause objectionable A/V sync problems, but more troubling is when the effect was inserted or removed, causing the delay to jump in and out.

This was very evident with live interviews, where the interviewee would be taken either full frame (no effect) or in a picture-in-picture box (effect). Some productions would just add fixed audio delay but have to provide un-delayed audio for the return IFB channel to the interviewee, or sometimes the audio mixer would switch between delayed and un-delayed audio depending on the state of the effect.

But these complications were ‘behind the scenes’ and did not effect the overall distribution of the program and delivery to the consumer. Consumer analog TVs did not have frame-based digital processing and thus no significant video delay.

In professional production, A/V sync was kept in line more or less. Part of the reason for this success was that the delays were generally static and manufacturers developed frame syncs with tracking audio delays that use audio re-sampling techniques to insure zero A/V delay.

MPEG
Then MPEG happened! Digital compression works to reduce the data per frame required to represent a unique picture or sequence of pictures. This works fine for video because video is a discrete time signal, i.e., there are defined gaps in time between each successive video frame. These gaps provide the decoder with the ability to de-compress the original frame and then output the full picture sequence.

But audio is a continuous time signal, there are no regular silent gaps and thus no way to catch up with the video. Audio is compressed separately from the video and the MPEG encoder has to insure the video data packets and the audio data packets are time aligned both inside the encoder and after the multiplexer (mux). The time alignment is insured by the inclusion of timestamps called Program Time Stamps (PTS) as well as Decoded Time Stamps (DTS) within the bitstream. PTS and DTS are derived from a common System Time Clock (STC) which is a 90 kHz counter related to the video sampling clock (27 MHz).

Great! What could go wrong? Well if the transmission is essentially error and jitter free, there are no problems. This is why DVD and Blu-Ray don’t generally have a lip-sync issues except if the disc is damaged. But in the real world of broadcasting, transmissions are not error free, and frequently, some data is damaged or lost. The MPEG receiver uses transmission buffers that absorb these errors, recover the STC and lock to the local sample clock.

However, the receiver has to compensate for loss in the final decoding. In the case of video loss, either the decoder jumps ahead a frame or repeats a frame of video. The audio can’t jump forward or repeat audio samples to adjust to these timing changes, without causing objectionable audio distortions or silence. If the audio data is damaged, the decoder typically will mute and the video timing can be adjusted by dropping a frame. Over time, the video frame times and the audio frame times drift, causing some mismatch between the audio timing and video timing, typically within one video frame time (33 mS).

Ideally, when the A/V delay reaches detectable limits, the decoder should reset the buffers and start, over but most decoders don’t do this as it maybe considered more objectionable to interrupt the signal flow than correcting lip sync. That’s why picking up the remote and changing the channel and back will many times fix the A/V sync problem. The only option left for the consumer device (TV, smartphone, set-top box, etc.) is to add or drop a video frame.

IT’S EVERYWHERE
The AV problem related to compression is everywhere, even on the internet. While internet video uses a different timing synchronization—Real-time Transport Protocol, or RTP, timestamps—it does have the same problem with corrupted data and variation in packet timing delivery in the form of jitter.

The final lip-sync challenge that digital has introduced is the development of flat-panel displays. These displays have intrinsic delays due to the buffering of the video data into frame memory prior to the pixel readout. While this is typically just one or two frames of delay, the accumulated A/V delay can be perceivable and perhaps objectionable.

So what has the industry been doing to fix these digital compression issues? On the professional side, many techniques have been developed and put into practice. For off-line testing, test patterns have been developed that help line up audio and video timing (e.g. VALID). But what about in-service A/V timing measurement and correction?

In the mid ’90s, Tektronix developed a technique that inserted a watermark into the video image that represented a digital sample of the audio envelope. This technique worked fine with uncompressed or lightly compressed television signals, but was not reliable when heavy video compression was employed.

SMPTE published a standard last year (ST 2064-2015) for lip-sync measurement using video and audio signatures. Signatures are metadata elements derived from the image pixel data or audio envelope via an algorithm that insures that the signature is unique to that image or audio envelope. The origin signatures are transported with the audio and video signals and at the receive end and are compared with new signatures calculated from the received audio and video. The received signatures are then compared to the origin signatures to determine the video and audio delays. The receive device can then make delay adjustments to achieve zero A/V delay.

This SMPTE method is available for professional equipment but has not been adopted for use by the consumer products. The Consumer Technology Association (aka CEA) has a recommended practice (CEA CEB20) for consumer television products that address A/V synchronization problems and offer up best practice solutions. But in the end, the consumer product is dependent on the limitations of MPEG (and RTP) timing and synchronization techniques.

Most flat-panel displays today compensate for the video delay by delaying the audio decoding internally so that the built-in speaker audio is in time with the picture. The HDMI 1.3a specification includes the ability to allow consumer displays to declare their audio/video delay and thus compensate for display delays when using an external AV receiver.

So the good news is that the industry is engaged in fixing the problem. The bad news is that no one solution has been found that can solve every aspect of the lip-sync problem.

Jim DeFilippis is CEO of TMS Consulting, Inc., in Los Angeles. He can be reached at JimD@TechnologyMadeSimple.pro. See more at his author archive.