PBS technicians Karl Gebhardt and Tammi Martin simultaneously simulate an ingest process using Avid editing solutions, MassStore and BroadView applications.
The entire broadcast industry is rapidly developing and moving to a server- and file-based environment with more emphasis placed on the technical integrity of the content. Just as important, an emphasis on the accuracy and completeness of the accompanying metadata is also emerging.
The PBS Media Operations Center recently completed the engineering, testing and deployment of a digital, server-based content ingest and editing facility. Designed to decrease and eventually eliminate the use of standard videotape, this facility aims to minimize the re-entry of content-related data, substantially curtailing the potential for data-entry borne inaccuracies while simultaneously allowing for future metadata-driven content changes and automated program reassembly via frame-accurate content segmentation.
The ingest system design is a clear departure from the standard hardware-driven signal paths into an intricate choreography of software, messaging, middleware and hardware interactions. These interactions are commonplace in standard manufacturing environments but have been missing up to this point in the broadcast environment. In fact, PBS has designed a system that totally integrates a scheduling and trafficking software program with what has traditionally been known as a post-production environment. To the user, the integration appears seamless, starting with the process of receiving metadata and a file or tape of a program through ingest of the content and on to play out for air.
Upstream process
PBS video technician Bill Andrea simulates the ingest process using Avid, MassStore and BroadView applications.
The ingest process starts well before any source material arrives at PBS when the producer submits a Media Inventory via the Web containing frame-accurate program metadata into the BroadView Software scheduling and traffic system. The Media Inventory is reviewed and later approved by the PBS programming department. Upon approval, as part of the process and also via the Web, BroadView provides the program producer with a printable bar-code label that is then affixed to the program tape prior to submission.
The tape arrives at PBS and is scanned into the tape library. One single click of the bar-code scanner unleashes processes in several connected databases that are then used to track content, allocate physical and human resources and maintain a reliable connection between program essence and its accompanying metadata. The scanning of the bar code automatically changes the BroadView media status from “Awaiting media” to “Arrived” and proceeds to trigger two separate downstream processes.
The first process uses standard Avid application program interface (API) calls to create an advanced authoring format (AAF) file with all of the necessary program metadata in the Avid Unity catalog system. The second process uses XML-based messaging protocols and Microsoft's BizTalk and triggers a message into ScheduALL, which then creates an ingest work order with the status as “Scheduled.” The work order contains all of the relevant instructions for the technician to perform the technical evaluation.
Figure 1. Metadata workflow. Click here to see an enlarged diagram.
When a technician arrives for his or her shift, the ScheduALL work orders automatically triggered by the arrival of media have been prepared and clearly lay out all the tasks for the day. If the technician opens a work order that requests a program be ingested from one of the Avid Media Composer Adrenaline Ingest/QC stations, the technician logs into the Avid Unity system via MediaManager, drags and drops the BroadView-created AAF file into its workstation and follows the step-by-step ingest procedures as prescribed by the metadata contained in that particular ScheduALL work order. (See Figure 1.)
Ingesting's technical apparatus
During the ingest and technical evaluation process, a technician uses three devices to ensure that PBS technical operating specifications are met. They are the Avid Media Composer Adrenaline, a VideoTek VTM 440 and the Dolby LM100.
The Avid environment was selected as the Ingest mechanism because it provides the economical ability to accomplish both ingest and editing using the same tools and the same streamlined workflow in SD and HD. The media composer environment also provides media management solutions.
For example, file submissions follow the same process as tapes. As the source material is ingested into the Avid environment, the technician performs a comprehensive technical evaluation. Then the technician checks the appropriate Avid locators that define the ins and outs of every segment in the program. These locators were initially defined by the producer in the Media Inventory and carried through by BroadView into the AAF structure. The technician may move the locator's ins and outs a couple frames in either direction to ensure that the audio and video represent clean segment breaks.
PBS editor Robert Sottile streams three channels and takes an IMX50 50MB file and rerecords it to an 8MB file in the flattening room.
An instance of when this may occur would be when the segment break was stepping on the secondary audio program, which occurs on channel four for PBS. By moving the locator, the technician could prevent a problem that would result in an audio burp in repackaging the program at a later date. These frame-accurate in and out locators are used to automate the replacement of such program elements as underwriters, schedule-specific promos and other dynamic spots.
The VideoTek VTM 440 with the SpyderWeb software interface provides an electronic measurement of video and audio. The Dolby LM100 measures the perceived loudness of dialog. These devices are calibrated based on PBS technical operational standard parameter settings. Logs are generated by each of these devices and become part of the metadata associated with the ingested material.
Once the ingest task and technical evaluation are completed, the technician changes the ScheduALL status to “Completed successfully,” which automatically updates the media status in the BroadView program to “Ingested” via an XML BizTalk message. It also signals BroadView to query the Unity and extract the timing information from the locators to update its database. At this stage, the program is available for review by the programming department or any other authorized user via Avid's MediaManager high-resolution Prolog browser.
Figure 2. File movement. Click here to see an enlarged diagram.
The final step in the work order involves the archiving of the program material in various formats, which will be necessary for future repackaging of the program. (See Figure 2.)
Archiving for later editing
A program is archived with all of its related Avid metadata in an I-frame only, MXF-wrapped IMX50 file. This relatively high bit rate will allow subsequent changes to be made, if necessary, to the actual program content without sacrificing picture quality.
Simply by right clicking on the completed file sequence name and selecting the “Send to workgroup” (archive) option in the Avid editor, the operator invokes the MassTech SAvI option that interfaces Avid's Transfer Manager with the Mass-Store archive manager. This interface uses Avid's dynamically extensible transfer API to move Avid media objects in native Avid MXF file format between the Unity and the MassTech system.
MassTech first moves those media objects from Avid Unity to its near-line cache and later to PBS' LTO-based ADIC Scalar 10K data tape library, while simultaneously updating its database to reflect the addition and location of these assets. This lower-cost tape archive effectively and economically extends the storage capability of the high-performance Avid Unity storage subsystem. Currently, these FTP-based Avid-to-MassTech transfers are occurring at approximately three times faster than real time, a higher rate of speed than PBS had originally expected.
Later on, if content features need to be added to that program, for example closed captioning or descriptive video services (narration tracks for the sight impaired), or if the program needs to be edited for content reasons, such as language or nudity, several events occur. First, an edit request would be created in BroadView, which holds the media status as “Ingested.” This request would initiate a message to Biz-Talk. Second, BizTalk would send an XML request to MassStore, triggering the automatic restoration of the program from the ADIC tape library back to the Avid Unity. BizTalk would also send a message to ScheduALL, triggering an edit work order with the proper instructions. When the editor arrives for his or her session and opens the appropriate work order from ScheduALL, all the required program material has already been loaded and is available on the Avid Unity in native Avid MXF format, making far more efficient use of the editor's time and unique talents.
Archiving for content packaging
After completion of the archiving in the Avid format, the technician will initiate a second process using the Avid FTP service through Transfer Manager. This FTP service transfers the file to an Omneon server in a 50Mb/s MXF format (OP 1a). Immediately upon arrival on the Omneon server, the file will automatically be archived to the ADIC tape library via MassTech's MassStore archive software. At a later date this file can be repurposed in an automated process when the program is repackaged.
Repackaging entails the producer switching out underwriters' spots within a program (historically underwriters have been part and parcel to the program). These underwriters' spots are defined by the producer in the BroadView Media Inventory as segments. The process of removing an underwriter spot and replacing it with a new spot can be accomplished strictly in software by creating a new package in BroadView. One can drop the metadata for the original program essence and add the metadata for the new underwriter spot, letting the scheduling software and automation actually stitch the new program together by following the timings from their respective metadata. Termed flattening, this automated process ends the hours and hours of content re-ingestion, editing room manipulation and tape recreation.
Archiving for content distribution
Figure 3. System design. Click here to see an enlarged diagram.
When all the needed media for a program has been ingested, BroadView triggers a series of automated events to create the final program file for distribution to PBS stations. BroadView generates a series of play-lists that are executed in the flattening process. This process consists of the 50Mb/s file being played out, stamped with the appropriate V-chip, Nielsen and other pertinent information and then immediately rerecorded at PBS' standard MPEG distribution rate (8Mb/s for 4:2:0 video and 4Mb/s-for uncompressed audio for a total payload of 12Mb/s).
The flattening room consists of automation control and the monitoring of three channels of Omneon play out, record and review ports for the packaging of programs, as well as a fourth port dedicated to dub creation. The flattening process is monitored by Evertz AVM 7760s to ensure that all of the expected audio, video, V-chip and captioning are present on the final distribution file.
Figure 4. System integration. Click here to see an enlarged diagram.
When the distribution file recording is finished, MassTech will initiate another archiving process, creating now a third instance of the content in the library, albeit this one at a lower bit rate. The file is then moved to the network operations center for both real- and non-real-time distribution.
These files will be repeatedly leveraged in our standard schedules, via future next-generation interconnection system using user datagram protocol multicast over satellite or even in on-demand member station requests fulfilled via Internet-based broadband connectivity.
Archiving for proxy viewing
In addition to the archiving process, the completed recorded distribution file triggers the creation of a Windows Media proxy, encoded at 1.5Mb/s. This proxy file is available at the desktop of PBS content screeners by virtue of dual high-quality flat screens. Every program aired by PBS can then be screened, flagged and annotated for content. This proxy viewing is triggered simply by clicking on the appropriate program in the BroadView schedule and selecting the “View media” option. The proxy is also archived in the tape library creating now a fourth copy of the content in the tape library.
When all is said and done, there are four different copies of each program in the tape-based archive. Each has unique characteristics that make it well-suited for a specific purpose, but each one carries with it the objective of leveraging one initial ingest and quality control cycle across the program's lifespan. In combination with frame-accurate, complete and coherent metadata, PBS expects to generate substantial savings in time, materials and human resources, while significantly reducing the number and gravity of metadata-related on-air discrepancies. In addition, it is expected that the elimination of codec cycles will continually increase the technical quality of the programs delivered to PBS member stations.
Creating a collaborative work environment
PBS director of the Media Operations Center Wendy Allen does management file transfers in the storage and networking room.
As you would expect in a system that integrates so many software and hardware providers, the overwhelming majority of problems were centered around the interfaces between these systems and the ability to accurately and succinctly describe what PBS wanted to accomplish. Using the proper terms to describe functions and data points across so many vendors was challenging.
The skills needed to address the entire system ranged from such classical broadcast engineering issues as drop frame time code, the relative infancy of MXF and 16-bit versus 24-bit audio, as well as typical IT concerns, such as database constructs and optimization, message passing protocols, hardware RAID optimizations, fiber-optic IP networking, sub-netting and access control lists.
Combining the finest broadcast hardware, operations management and engineering know-how with the best practices from the information technology and enterprise application integration disciplines, the Media Operations Center has enabled a revolutionary set of automated, radically streamlined and, ultimately, far more efficient content processing workflows. It required tremendous dedication by an enormous number of PBS personnel, vendors and consultants.
André V. Mendes is chief technology integration officer and Wendy Allen is director of the Media Operations Center for the Public Broadcasting Service.
