Image restoration

Restoration is a hot topic for content owners, broadcasters and producers alike these days. Viewers are more discriminating than ever now that HD programming is becoming the norm, and high-quality, large-screen HD displays are showing up in every living room. In addition to live and current programming, demand is high for classic films, features and television series to be remastered for distribution on DVD and Blu-ray, or for transmission on satellite and cable. Much of this material — especially older titles — requires comprehensive restoration in order to meet consumers' high-quality expectations.

But restoration is not just for old reruns of “I Love Lucy.” Even with today's sophisticated production equipment, errors such as noise, flicker and unstable sequences can still creep into new content. Often these errors are not noticed until much later, when it is impossible to reshoot the scene. Likewise, state-of-the-art restoration technology is becoming a boon to independent producers who want to incorporate archival footage into documentaries, historical and nature series, travel shows, and biographies. With the right restoration technology, material that might otherwise be unusable can be included without any sudden quality drop-off that could adversely affect the overall impact of the program.

Advanced restoration techniques

High-quality film and video restoration has traditionally been a laborious and time-consuming process requiring frame-by-frame editing, but the technology has come a long way in just a few short years. More powerful processors and improved efficiency algorithms have made available new restoration systems that can restore HD material within just two or three times its running length, delivering a clean product that often surpasses the original version. The state of the art in restoration systems includes tools for real-time removal of dirt, dust, grain, noise, scratches, instability and flicker with a result that requires less bandwidth for digital transmission. Consider the problem of physical dirt and dust that can mar film even if it has been carefully stored. When the film is transferred to video, dirt and dust show up as white, black or colored speckles, spots, or lines of varying sizes and shapes. Motion-compensated processing is a fast and highly effective method for removing dirt and dust because it can automatically rebuild a frame from the previous and subsequent frames.

Another irritating defect is visible film grain that results from the physical composition of the film itself. With video, the thermal properties of cameras can produce noise that has a similar appearance to film grain. Grain and noise are often more noticeable and distracting when content is shot in low lighting conditions, such as the example in Figure 1. When the content requires downstream compression for transmission or distribution on Blu-ray, high levels of noise and grain severely affect the compression codec because it must waste valuable bandwidth coding these unwanted picture artifacts.

Noise and grain reduction tools can significantly improve content without sacrificing picture quality, as shown in Figure 2. The most powerful technologies will adapt between temporal recursive filters and spatial filters to obtain the best possible results in both moving and stationary areas in real time. In addition, such tools allow operators to adjust the level of noise and grain reduction on the fly so that they can rapidly obtain the most pleasing subjective effect — for example, leaving some visible grain if this is the artistic effect desired by the original director.

Vertical scratches are another common type of picture defect, typically introduced to content during the film scanning process. Although scratches are often associated with archival content, even modern productions can suffer from errors that lead to scratches if there are problems in the scanning process. However, the latest generation of hardware restoration systems include powerful algorithms that can automatically detect and repair unwanted scratches in real-time, and can distinguish the scratches from areas of fine picture detail. The user can choose how sensitive the scratch detection should be and what degree of repair should be applied.

Improved compression efficiency

In addition to a less-than-optimal experience for viewers, noise and grain in video content add an underlying movement that reduces the ability of a compression codec to achieve efficient compression for transmission or Blu-ray/DVD production. If the noise and grain can be removed before compression, the content can be compressed into a lower bandwidth, allowing more channels to be transmitted per transponder and thus lowering the costs of transmission.

The most advanced restoration systems use a sophisticated adaption technique to select between motion-compensated temporal processing and a complex spatial filter. Automatic threshold calculation distinguishes between noise or grain and motion to ensure removal of noise and grain without reducing any of the wanted detail in the image. In experiments with a high-quality, real-time noise reduction process for one product, the noise and grain filter was found to reduce the compressed data rate down to one-fifth of the bandwidth for the same transmission quality.

Thus, broadcasters can either use a lower data rate for the same quality of transmission, or they can achieve a better-quality picture at a fixed data rate. All these benefits lead to more satisfied viewers and lower costs because transmission bandwidth — especially over satellite transponders — is an important cost element in broadcast services.

Better compression efficiency also enables Blu-ray publishers to create higher quality results, which will be guaranteed to look good on large screen displays. In addition, the improved efficiency of storage means that more content can be made available on the discs, leading to more attractive products for consumers.

A case study: “The World at War”

One important role for state-of-the-art restoration is to enable historic footage to be incorporated into new productions without affecting their overall quality. A case in point is a stunning new HD Blu-ray edition of the documentary “The World at War,” recently completed by London-based post-production company Dubbs and its restoration and digital media division, Eyeframe. Originally broadcast in 1973 by ITV, the 26-episode “The World at War” series focuses on the events immediately before, during and after World War II, with rare color film footage and interviews with major figures of the Allied and Axis campaigns.

Eyeframe and Dubbs oversaw the complete restoration process for the new version, including telecine, regrading and completing a full set of HD broadcast masters. The project presented numerous challenges — not the least of which was working with 37-year-old material and original source footage that was almost 70 years old. In addition, the Blu-ray edition would include several additional features with never-before-seen footage, which meant working through many hours of very old and fragile film.

Continue on next page

The restoration team relied on an automated restoration solution that could maintain the series' original production values while delivering new experiences to viewers. With tools for stabilization, deflicker, dust and dirt removal, and tramline filters, the system operated in real time, allowing the restoration team to continuously monitor the output to ensure the highest quality and guard against adding artifacts to the program material. At the same time, the team's skilled operators had plenty of control and fine-tuning abilities that allowed them to deliver an effective restoration that still maintains the look of the period. The system completed the job in only two to three times the program length, enabling the team to avoid time-consuming frame-by-frame “painting out” work and helping them meet a very tight delivery schedule.

A win-win

Today's state-of-the-art technology has brought the important and highly specialized field of restoration into the digital age, enabling rapid and cost-effective delivery of both old and new content that measures up to the high-quality standards of HDTV and Blu-ray. Everyone in the content delivery chain reaps the benefits. Content owners are able to reuse assets that might not have been considered candidates for HD remastering before because of their questionable quality. Thus, they can monetize and generate new revenues from assets that are currently languishing in costly storage.

At the same time, specialized restoration service providers can rely on real-time hardware-based systems that automate a wide range of repairs, allowing specialist operators to focus on more difficult restoration tasks. In simple terms, this means restoration is more affordable for providers and clients alike.

But perhaps the biggest winners are consumers, who can access their favorite programs in dazzling HD without the distraction of artifacts arising from older production methods, poor storage or poor initial post production.

Paola Hobson is product manager for conversion and restoration at Snell.