Today, the “tapeless” dream has already been realized. Tape is still used for acquisition and archiving but has been long ago forgotten in the post-production and playout departments. Although file-based technology is progressing, compatibility issues still remain. While using fast, nonlinear applications to broadcast media is a true time-saver, dealing with various file formats and compression types can be a real headache.
In the past, we had to deal with a number of mutually incompatible tape formats, such as BetaCam, DVCAM, DVCPRO, D1, D9, etc. Now we are drowning in a mixture of file formats and compression types, such as MJPEG, MPEG-1,- 2, -4, Windows Media, QuickTime, DV, JPEG2000, etc., as well HD and SD and the requirement to work with other non-TV image formats including those of mobile phones. In many cases, the speed and flexibility benefits are compromised by the hassle and quality degradation involved when converting from one format to another.
It seems the file-based approach is not a panacea for content workflow. While modern networks and storage backbones can transfer content blazingly fast, dozens of times faster than the real-time content duration itself, content conversion is still a tedious, semiautomated task. New file formats or compression types introduce even more challenges to content conversion engines. Not to mention that every manufacturer tends to create and fight for its “own” media implementation of the “same” format, sometimes within the standard specs, sometimes not — and this is when a nice standard emerges. A good example would be AVC-Intra, which is essentially AVC (H.264 compression) that doesn't take advantage of the interframe compression algorithm; therefore, every frame is independent from the others.
Why so many file formats and compression types?
This question may sound familiar to many because it reflects the not-so-old “tape format war” discussions. And the answer is still the same: No single existing format is perfect for all applications. As a rule of thumb, the larger the tape is, the better quality it provides (for its time). However, there are always a bunch of lightweight, portable formats that are not up to the large tape quality but are acceptable for newsgathering and other outdoor activities. It is the same situation today — MPEG-2 is perfect for transmission and storage purposes, but it is not so good for editing, while DV edits easily, but has a relatively huge file size. Economy versus flexibility, reliability versus mobility, quality versus speed — these are all trade-offs when dealing with formats to suit particular parts of the scene-to-screen chain.
Role of content conversion
Simply put, content conversion is necessary because of the compromises and limitations of the various individual file formats and compression systems. Many of the file-based products on the market (NLEs, playout servers, etc.) are designed to work with one or two media formats. This simplifies product development and significantly shortens the time to market. For example, a server that supports only DV can be launched in less than half a year; a server that supports only MPEG-2 can start shipping in about one year. Imagine how long it would take to launch a product that supports every existing compression system or format today; it is just not feasible.
Some manufacturers go in the opposite direction by using their own proprietary formats so they can lock customers to their product line only. Previously, content had to undergo at least three or four conversion stages in a file workflow from ingest to transmission and archiving. Today, many systems can consistently support the major compression types and file formats, so such conversion is not mandatory. (See Table 1, bottom of page 2.)
Content conversion is a complex process to consider. Do we need to convert from one compression type to another, e.g. MPEG-2 to DV? Do we need to convert from one resolution to another, e.g. SD to HD? Do we just need to move content from one file container to another, e.g. AVI to MXF? Or do we need to do all at the same time? The conversion can be lossless or lossy, quick or slow, simple or multistep, depending on exactly which processes are required.
Despite all efforts to avoid compression conversion, we usually end up with at least one conversion: the transmission encoding. Whatever the choice for the in-house file format, the transmission format is always different, or at least the bit rate is. Even if we choose to stick to MPEG-2 (studio quality) for good in-house storage ratio, for the MPEG-2 satellite transmission, we need to provide an even lower bit rate, somewhere in the 2.5Mb/s-4Mb/s range. And although the compression is essentially the same, recompression needs to be done. (See Figure 1.)
Containers and codecs
“Codec” stands for “Coder and DECoder,” and usually means “Compressor and DECompressor.” It is the engine that transforms the baseband (uncompressed) video into a compressed stream and vice versa. The container is the file wrapper that is used to hold the compressed stream in one entity (the file) throughout the file system of the storage equipment. File wrappers are related to codecs and to the computer operating system as well. However, these relationships are not very well documented or standardized. As a general rule of thumb, Apple computers use QuickTime containers for storing video, regardless of the codec involved, while Windows computers use a variety of containers depending on codec and implementation, including QuickTime files (if the QuickTime add-on is installed).
The most popular container type on Windows used to be AVI, also known as Video for Windows. It can hold virtually any type of compressed stream. This means that you can find MPEG-2AVI, DV AVI, MPEG-4 AVI, MJPEG AVI, etc. Even though these share the same file extension, their content is not compatible in terms of a single codec. Today, Microsoft is promoting its Windows Media Video, which is stored in WMV or ASF file formats. Additionally, all MPEG formats have their own file types, e.g. MPG, M2P, M2T, MP2, MP4, TS, etc. Note that file extensions are not a warranty of compatibility. For instance, the same file extension, MPG, can be used both for MPEG-1 and MPEG-2 streams. Only an analysis tool can identify the content codec inside.
The trouble with most converters is that they do not care about seemingly nonessential things such as metadata, closed-captioning data, teletext, etc. The reason is that these nonessentials are stored in different ways for each file format or in a different place in the stream. So when choosing a conversion solution and wondering why some cost much more than others, check the small print. It is most likely that the cheap solutions will discard nonessentials. Additionally, during the HD transition period, special attention needs to be paid to aspect ratio information (WSS, AFD) so it's correctly applied during conversion. Advanced converters would offer additional benefits such as audio transformation, normalization, multiple tracks, etc.
An effort has been made toward unifying file containers — at least in broadcast equipment. The MXF format is gaining momentum with all broadcast manufacturers. However, it will be awhile until equipment from one brand properly talks to another through the MXF format. MXF is an excellent move, but it enables so many operating profiles that few manufacturers can entirely support it. The “MXF-compliant” label does not necessarily mean that this equipment can handle any MXF content. This compliance can only refer to a specific codec within an MXF container, or to a specific profile of the format. Therefore, don't assume MXF files can be used with all MXF-compliant equipment.
Select the best converter
Despite the progress with MXF, it seems certain that the number of file types, codecs and wrappers will continue to increase, so format wars are set to continue. The best approach when selecting file converters is to always verify manufacturers' claims. Try to avoid the universal type of converters; they are usually mediocre in most aspects. They will either be too slow or too simplified. Here is a short list of steps that can be taken to ensure against misunderstandings:
- Get a trial unit or license.
- Try it with your own files.
- Benchmark its performance against competing converters with the same content.
- Benchmark its quality preservation against competing converters at the same target bit rate and other conversion settings (e.g. quantization factors, GOP size, etc.) using a PSNR comparison tool, or visually.
- Make sure it behaves properly with long files (larger than 2GB).
- Check the lip sync at the beginning and the end of the converted file.
- Verify the resulting converted file with the equipment that is destined to use it afterward. Look for drifting lip sync and smooth playback.
- Look for the small things such as additional audio tracks, metadata, closed-captioning, etc.
- Optionally, send a small sample file to an analysis facility that can give a verdict on whether the file complies with the relevant standard, e.g. if an MPEG-2 file complies with ISO-13818. These facilities have expensive tools that the average broadcaster cannot afford to purchase.
Stoyan Marinov is chief technology officer at PlayBox Technology.
Production Post production Playout Broadcast Internet/mobile H.263
MPEG-4
H.264 WMV 2Mb/s-5Mb/s WMV 2Mb/s-5Mb/s WMV < 1Mb/s
Flash < 1Mb/s SD Beta SP
DV
DV50
IMX
P2 DV
AVC-Intra
MPEG-2 I-frame 50Mb/s
MJPEG
Baseband 270Mb/s MPEG-2 IBP 8Mb/s-15Mb/s
DV MPEG-2 2.5Mb/s-5Mb/s
H.264 1Mb/s-3Mb/s HD DV HD
MPEG-2
HD DV HD
MPEG-2 HD 8Mb/s-300Mb/s
Baseband 1.5Gb/s MPEG-2 HD 20Mb/s-80Mb/s MPEG-2 15Mb/s-20Mb/s
H.265 8Mb/s-16Mb/s
Table 1. Compression types and file formats operating at particular data rates are chosen as a “best fit” for each application across SD, HD and Internet/mobile applications, making file conversions essential to run the scene-to-screen workflow.
