Mixing Formats in the Edit


ORLANDO, FLA.
The sheer mix and volume of formats to deal with today can be mind-boggling. Videotape player/recorders—formerly a common denominator—are a vanishing breed. Post facilities still own and use VTRs, but operations at the local market level, especially in broadcast, are becoming increasingly tapeless. Clearly, once the current crop of installed VTRs become a maintenance head-ache or is no longer an important cog in the operation, they won't be replaced with another shiny new mechanical videotape transport from Sony or Panasonic.

DEFINING 'FORMAT'

It all starts with the camera, so the driving influence is the move to tapeless acquisition—P2, XDCAM-HD, GFcam, AVC-HD, etc. On the bright side, it means that the integration of another format will cost no more than the purchase of an inexpensive reader, rather than a new VTR to support that format. Unfortunately this will also mean a proliferation of new formats for the editor to deal with.

The term "format" should be clarified with tapeless media, like P2. First, there is the codec used for the actual audio and video content (essence). That essence is defined by the compression method (like DVCPRO HD or AVC-Intra), frame size (SD or HD), pixel aspect ratio and frame rate. The essence is encapsulated into a file wrapper (MXF), which holds the essence and metadata (information about the essence). Lastly, in the P2 example, the files are written to a physical transport medium (the P2 card itself), using a specific folder and file hierarchy. Maintaining this folder structure is critical in order that an NLE can natively recognize the media, once it's copied from the card to a hard drive.

Nonlinear editing systems have been built around a specific media structure. Avid Media Composer uses OMF and MXF; Apple Final Cut Pro is based on QuickTime. In theory, each can ingest a wide range of tapeless file formats, but the truth is that they only work well with a much narrower range of optimized media. For instance, DVCPRO HD is handled well by most NLEs, but H.264 is not. You can toss a mix of formats onto a common timeline, but the system is internally operating with specific settings (codec, frame size and frame rate) for that timeline.

These settings are established when you first create a new project or a new sequence, depending on the application. Any media on the timeline that deviates from these settings must either be scaled and decompressed on-the-fly by the real-time effects engine of the application—or must be rendered—in order to see full-quality playback. Most systems are optimized for NTSC, PAL, 720p and 1080i frame sizes. Even Final Cut Pro—touted as resolution independent—works best at these sizes and effectively tops out at 2K film sizes. All the desktop NLEs freely allow you to mix SD and HD content on a timeline, but the rub has been a mix of differing frame rates. FCP could do it, but Media Composer wouldn't. That barrier disappeared with Avid's introduction of the Mix & Match feature in the Media Composer 4.0 software. Now, if you edit a native 23.98p clip into a 29.97i timeline, all of the leading editing applications will add a pulldown cadence to the 23.98p clip for proper 29.97i playback.

AVOIDING CONVERSION

When editing a project that has a mix of SD and HD sources and formats, it is best to select a timeline or project setting that matches the predominant format. For instance, if 75 percent of your media was shot using a Panasonic VariCam at 720p/59.94, then you'd want to use a matching timeline preset, so that the 720p footage wouldn't require any rendering, except for effects. In this example, if the other 25 percent was NTSC legacy footage from Betacam SP, you'd need to have a system equipped with a capture card capable of ingesting analog footage. The Beta SP footage could be upconverted to HD during the capture using the hardware conversion power of a capture card. Alternately, it could be captured as standard definition video, edited onto the timeline and then scaled to fill the HD frame. Betacam SP clips captured as standard definition video would ultimately be rendered to match the 720p/59.94 settings of the timeline.

OPTIMIZED FOR BEST PERFORMANCE

Avid Media Composer 4.1 introduced a "Mix & Match" feature for editing multiple formats in the same edit. Until recently, Avid systems transcoded incoming media into an Avid codec wrapped as an MXF file. This creates media files that are optimized for the best performance. Final Cut would let you drag and drop any QuickTime file into the FCP browser without a transcode, but non-Quick-Time files had to be converted or rewrapped as Quick-Time .MOV files. These frontrunners were up-staged by applications like Grass Valley Edius and Sony Vegas Pro, which have been able to accept a much wider range of media types in their original form. The trend now is to handle native camera codecs without any conversion. Apple added the Log and Transfer module to Final Cut and Avid added its Avid Media Architecture (AMA). Both are plug-in architectures designed for native camera media and form a foundation for the use of these files inside each NLE.

Final Cut's Log and Transfer is recommended for importing P2, Red, XDCAM and other media, but it still doesn't provide direct editing support. Even natively supported codecs, like REDCODE and AVC-Intra must first be wrapped as QuickTime files. When clips are ingested via Log and Transfer, the files are copied to a target media drive and in the process rewrapped as QuickTime .MOV file containers.

If you want true native support—meaning the ability to mount the hard drive or card containing your raw media and start editing at full resolution—then the Avid Media Composer family, Grass Valley Edius and Adobe Premiere Pro provide the broadcaster with the strongest desktop solutions. All three recognize the file structure of certain camera formats (like P2), natively read the camera codec and let you use the media as an edit source without the need to transcode or copy the file first.

These APIs are evolving and are dependent on proper media drivers written by the camera manufacturers. Not all applications handle every format equally well, so select a system that's appropriate for you. For example, P2 using the DVCPRO HD or AVC-Intra codec is becoming widely supported, but Panasonic's AVCCAM has less support. Sony hit snags with XDCAM-EX support for Final Cut Pro when Apple upgraded the Mac OS to 10.6 ("Snow Leopard"). Fortunately these issues are short-lived. In the future it will be easier than ever to mix taped and tapeless camera media of nearly any format with little negative impact.