With the move to file-based workflows and a rising demand for more content on more platforms, broadcasters, content owners and distributors consume huge quantities of storage, which they use for many different purposes.
The ongoing evolution of media storage systems has driven the development of highly refined management solutions that make it easier to address high-volume media storage and processing workflows and, in turn, to realize the full potential of these file-based operations. Deployed and managed intelligently to support the full media life cycle, today’s storage systems can dramatically improve users’ ability to work with and deliver media at the right time, in the right format — and at the right price.
End-to-end storage model
The ability to share access to media across all parts of the operation is a significant benefit for a truly global broadcast operation. With a centralized ingest and archive system, such companies can give all playout centers, production facilities and any other key business areas access to any content acquired or created by the organization. However, the ideal world in which a central storage architecture effectively supports all operations and media processing tasks is still some distance in the future.
Right now, the more common, practical approach is to select different tiers of storage for different purposes. This approach allows broadcasters to select the best storage solution for each use case, which means that, within complex storage environments, they work with: specialized storage for ingest and playout, production and editing; nearline medium-term media storage; and, for longer-term, archiving and retrieval.
Though this model requires the movement of media between different storage systems, the sophisticated media management and hierarchical storage management solutions now available go a long way toward alleviating some of the pain of complex storage environments. Broadcasters can implement the storage systems that provide the best mix of functionality, performance and price for particular applications. Doing so allows depending on media and storage management systems to do the work of presenting all stored content in a federated view that is easy for users to negotiate.
Media and storage management solutions further simplify operations by automating the movement of media files according to the demands of the workflow. This, in turn, lifts file-transfer responsibilities from operators. This frees them to focus only on tasks within their areas of expertise.
Because sophisticated media and storage management solutions can facilitate more effective use of storage throughout the workflow and across operations, they give broadcasters (and major networks with large-scale operations) the means to establish a high-performing, yet cost-effective, shared media access model. Such a model not only simplifies workflow, but it also ensures that automation, the ingest content repository, transcoders and quality control systems all operate seamlessly with the storage architecture.
Model in action
To address the challenges of managing an ingest and storage architecture designed for high volumes of media shared by multiple sites and workflow processes, broadcasters are turning to SOA-based media asset management (MAM) platforms. Such platforms typically manage a three-tier storage architecture consisting of online, nearline and archive stores, which are complemented by real-time ingest and playout servers. This architecture also integrates applications that comprise an enterprise-class content management, preparation and delivery platform.
The online store, which is optimized for storing, sharing and serving media files as they enter the facility and are prepared for playout, is used as a central storage pool for content that is being edited and processed. The storage dedicated to this task should provide the performance of a SAN with the simplicity of NAS, and this performance typically is enabled by parallel access across many storage nodes and network connections. In addition to ensuring fast access speeds, the online storage system also must boast the usual reliability requirements for 24-hour operations.
Storage dedicated to nearline storage typically doesn’t have the same bandwidth performance requirements as the online store. Instead, it is typically used as a holding area for content while it is either waiting to be processed or has been processed. It is usually larger in capacity than the online store, and it provides an immediately accessible cache for content in front of any robotic, tape-based (archive) storage.
Archive storage is typically comprised of a robotic tape library designed for mass long-term storage of content. While providing better cost efficiency per gigabyte than disk-based storage, archive storage does have the penalty of having to retrieve content before being able to use it, which adds potential latency.
The platform manages all transfer requests to and from the archive, with archive management software enabling the transfer of a colossal number of files of all sizes. The most common requests tend to be: the copying of files from nearline storage, and even preserving a copy of the file on tape; retrieving files from tape and copying them to disk; and restoring files from the archive to online storage. Massive numbers of requests may be processed simultaneously.
A bigger view
Combining media and workflow management tools with applications for proxy creation and management, the system provides a federated view of content across all of the broadcaster’s storage systems. Because it also can serve as a central point of control for media processing tasks — including transcoding, QC, and file transfers — it provides a single interface through which operators can maintain an awareness of the status of all jobs.
It coordinates applications across the enterprise to optimize workflows, and it provides built-in, media-aware services for metadata, rules, searches, transfers and organization. When equipped with a work-order management feature, the platform allows users to organize, control and prioritize manual tasks, such as operator-performed QC shot logging, track stacking and language translation, and processing. Once a particular task is complete, the platform automatically routes content to the next workflow step and, if necessary, automatically moves the content between storage systems. This negates the need for the operator to do so.
At the foundation of this model, a reliable, high-availability media management system unifies control and storage with automation, ingest and quality control processes and systems across the global enterprise. The use of redundant high-availability management systems, synchronized in real time, ensures continual uptime, even as maintenance and upgrades are performed. As a result, media files within this storage system are always immediately available.
Day-to-day storage use
Using tiered storage controlled by a MAM platform, broadcasters can ingest programming directly into online (shared) storage, which is linked to the appropriate playout servers. Once content has been played out, it is archived and protected on the mirror side of the store. Both sides (A and B) are subsequently backed up to tape, after which content is automatically removed from online storage to make up space for new or active programs. In addition to managing these processes, the platform communicates with the archive daily to ensure that, at any given time, upcoming programming and commercial content has been retrieved — from tape if necessary — and is available on disk.
The efficient and streamlined application of file-based operations to stored content is among the main benefits offered by a MAM platform and the services it facilitates. Acting as a workflow controller, the MAM system provides a common platform for all media processing tasks, including transfers, transcoding, caption processing, file-based QC and archive management. This combination of functionality allows complex workflows to be orchestrated and enforced; ensuring content is efficiently routed throughout a facility.
Figure 1. By maintaining connections to all varieties of “agents,” a MAM platform such as Harmonic’s ProXplore ensures that content processing is done within business and operational rules.
Figure 1 demonstrates how this platform can provide operations and engineering supervisors with custom interfaces tailored to their job roles and responsibilities. By maintaining connections to all variety of “agents” (systems and devices that perform work on stored assets), the platform ensures that processing of content, regardless of the storage system that supports it, takes place in accordance with business and operational rules. It also assures that, at any point in an asset’s life cycle, supervisory staff can check or confirm its status — and even intervene and redirect processing tasks, if necessary.
In the automated QC workflow, the MAM platform sends jobs to the QC system, creating a queue of files for evaluation and processing. (See Figure 2.) By applying a logical sequence of evaluative parameters, the QC system assesses and directs assets to the appropriate QC device.
Figure 2. Shown here is a sample automated QC workflow, where the MAM platform sends jobs to QC. This creates a queue of files for evaluation and processing.
Throughout this process, updates regarding individual assets are communicated via XML reports to the MAM platform, which in turn makes asset and job status available to operators and managers. When complex or significant issues are identified by this automated process, QC reports (including timecode and thumbnail references) can be created and exported by operators to simplify decision-making in addressing any issues with stored audio and video files.
By maintaining details about the status and result of asset review, the MAM platform also supports a complementary manual QC workflow, which can be triggered by the findings of the automated system and carried out in parallel. The reason that a file “failed” automated QC is documented, and the content is reviewed manually and typically scheduled for re-ingest. At the same time, operations and engineering receive any information about possible tape or equipment faults that could be responsible for content quality issues.
Ultimate benefit
Implementation of a MAM platform, along with tiered storage systems, allows today’s broadcast operators to consolidate all media transfers — for the entire media-delivery workflow — under a single controlling system. For a comprehensive ingest and archive architecture, this approach likewise consolidates all variety of control and job monitoring capabilities within a single interface.
Users can track media assets, no matter where they are in the broadcast workflow or content life cycle. They may quickly determine the status of a specific content piece and the virtualized view of content across all playout servers, online storage, nearline storage and even the archive makes accessing that content easy. When business rules driving media movement must be bypassed, the platform supports manual configuration of content transfers between locations.
These capabilities and associated benefits yield dramatic efficiency improvements — and cost savings — in broadcasters’ high-volume content preparation and delivery models.
—Simon Eldridge is director of product management, Workflow and Manageability, at Harmonic.
