Understanding and implementing file-based architectures for news

Today, news directors and engineers are increasingly focused on building, developing and installing file-based workflows. The advantages are video and audio elements that are handled as computer files, which can be easily stored, manipulated and distributed in a variety of formats. Tasks like editing can begin as soon as content is ingested. As broadcasters transition to HD, these benefits become even more important.

The benefits

Broadcasters can benefit from file-based workflows in many ways, including:

  • Random accessSignificant time is lost by moving around tapes. In a tapeless system, editors move instantly to any frame or content and begin work.
  • Instant accessJournalists can start working on footage upon ingest. With tapeless systems, frames are available for use throughout the system within a few seconds, even if the feed is still recording.
  • Multiple accessIn a tape-based newsroom, only one person can work on the content at a time. In a properly configured tapeless environment, everyone can access content simultaneously.
  • Easy revision and repurposingMaking changes is easier. A promo can be quickly made by clipping the in and out points and adding a voice-over on an upcoming story. Operators never have to start from scratch. Jobs can be reopened, and changes can be made and then pushed to air almost instantly. These benefits are especially important when creating material suitable for alternate distribution channels like the Web or mobile applications.
  • Management oversightIt's impossible for a producer or news director to visit every edit room and preview each story before it goes to air. A tapeless system can bring the stories to the desktops of essential decision-makers.

Planning your system

There are many different ways to build a file-based infrastructure, each with its own unique performance and content availability capabilities. Although these IT-centric system designs allow users to pick and choose best-in-class applications and systems, it is important that disparate systems be compatible in order to create the desired seamless workflow.

The goal should be complete and unlimited interoperability. The solution requires a well-matched sampling strategy and a data rate based on a house compression format and a common file wrapper. This wrapper needs to be identifiable by all of the facility's systems and devices. Also, don't forget the importance of having standardized physical connections, like-minded transfer protocols that are supported by all applications and an integrated metadata dictionary.

Storage architectures

Start by considering storage needs. File-based architectures must have an adequate amount of storage. The storage platform must accommodate multiple users who carry out a task or series of tasks on a daily basis. The content should be simultaneously available to editors, producers, journalists and others involved with the production process.

Storage area network (SAN) and network attached storage (NAS) are two common types of storage platforms used in broadcast news facilities. When properly implemented, both can support a seamless two-way SD/HD signal flow and multiformat/multichannel outputs. Both enable multiple users to access media files simultaneously, thereby taking full advantage of a tapeless environment.

In newsroom applications, the key difference between a SAN and a NAS is how the storage system appears to the local workstation. A NAS appears as a networked drive with reads and writes managed on a file basis. This is similar to sharing files across a home network. A SAN, whether connected to the client by Fibre Channel (FC) or iSCSI over Ethernet, behaves like a local drive. Reads and writes take place on a block (as opposed to a file) basis. This allows software developers to implement deterministic performance for nonlinear editors. Deterministic performance is not a given. Nonlinear editing software must contain a specific code to take advantage of the SAN's read/write characteristics. A SAN, along with properly developed and configured NLE software, can ensure that editing applications perform optimally all the time.

A SAN production system uses a ring of storage devices linked together via FC. This allows users to share the storage arrays as if they were one device. If an editor needs capacity from server A, but server A is busy, the SAN system will automatically redirect the user's talk to server B or C.

Using high-speed FC technology, a SAN system can service dozens or even hundreds of audio and video channels through a shared, RAID-protected storage network. And, it can be configured to offer both deterministic real-time capabilities as well as open Common Internet File System (CIFS) connectivity at the same time.

A NAS architecture also uses multiple networked media servers, RAID storage, Ethernet connectivity and off-the-shelf components. It provides deterministic total server and client channel bandwidth, but may not guarantee that any editor or server will get bandwidth at a given time.

Like SANs, NAS can accommodate multiple video formats (including DV at 50Mb/s), scales to 14.6TB and provides open support for any CIFS-based program.

House compression format

All digital news field acquisition in SD or HD involves compressing the audio and video data. HD video starts out at 1.3Gb/s but is compressed to rates between 35Mb/s and 100Mb/s.

Compression technology enables files to be stored more efficiently and moved around a facility quickly and easily, especially where bandwidth is limited. In many cases, the compression format has a significant effect on the rest of the production workflow.

Often choosing a compression format is based on that of the primary video coming into the facility. Determine which compression is used by your ENG camcorders. Is it MPEG-2long-GOP, MPEG-2 short-GOP, DV-based compression, JPEG2000 or AVC? Each format and ratio needs to be tested to ensure that it works best for your specific application.

Metadata dictionary

It is important that a facility develop a metadata dictionary that will accommodate past, present and future information. It should detail all of the criteria a facility needs to identify a specific piece of media.

The dictionary must match the common language developed in the newsroom over the years. In addition, it should describe where the content came from, how it can be used and what elements make up the entire media file. Many of these items are technical or operational, and journalists may not be responsible for filling in the data. However, someone will need to do so, and the dictionary's language must be understood by all.

This dictionary is also helpful when logging incoming clips. With the proper descriptors, many tasks can be accomplished and devices can be made to operate automatically, with little human intervention.

With careful planning, a file-based infrastructure can be leveraged to support a variety of business models and workflows. The key is how it's implemented. The challenge is getting all of the systems to interoperate and work together reliably. Fortunately, there is now technology available to get the job done.

Ed Casaccia is the director of product management and marketing for digital news production at Thomson Grass Valley.