Robert Kafarski and Jennifer Vasquez operate WNET’s master control.
Station automation is nothing new, nor is the concept of asset management. However, since the introduction of automation and management systems more than a decade ago, many things have changed. These changes include widespread use of the Internet, high-speed networking and the availability of lower-cost data networks. Such new technologies allow systems and processes that were impossible just a few years ago. Industry pioneers in both the vendor and customer arenas have sought new products, technologies and systems to take advantage of these opportunities. New technologies also bring with them the challenge of new problems. Often, the solutions can be elusive. Typically, the larger the challenge, the more elusive the solution.
A new perspective on content
Such was the situation at THIRTEEN/WNET. Like most stations, it constantly faces the challenge of doing more with less. In this case, it means running not one but multiple program streams from a single, centralized automation system — a challenging situation, and one that is made more difficult by the fact that the system's ingest points are located many miles apart. The station is located in Manhattan, and its sister station, WLIW21, is located in Long Island. These stations supply viewers with several different program streams in both SD and HD through their digital and analog transmitters. The stations provide SD and HD feeds of their regular programming, a “World” channel, an educational “Kids” channel and a “Create” channel that encourages viewers to take a “hands-on approach to life.” Automation is the key that makes these multiple streams possible.
Migrating from single to multiple streams requires not just new equipment, but new thinking and processes as well. The station's management, engineers and operations personnel started with a blank slate, rather than an equipment list. Programs, sponsor breaks and other interstitial material — once simply considered as linear entities — had to be reconsidered not only as file-based assets, but assets of variable length. One reason is that while segment lengths may remain unchanged, overall program lengths can vary depending on the lengths of segments plus breaks. End breaks can easily be different lengths depending on the time of day the program airs or the format of the channel it is running on. The more they looked at the challenge, the more it was also seen as an opportunity.
Harris NetVx allows faster-than-real-time file transfer of full-motion video and audio.
Beyond simply automating, management also desired capabilities that stretched the normal operations scenario. Automating WLIW's programs streams from Manhattan was needed, but so too was the ability to provide local ingest and news from its facility into the automation system. Both real-time ingest and faster-than-real-time transfers were needed.
Ultimately, the solution came, not in a rack-full of equipment, but in a 5RU device — Harris' NetVX and a small number of additional components, including Harris' Flexicoder and ReCon software. NetVX is a modular network interface system that allows connectivity to variety of networks. Integral to the automation system is a LAN that includes a bi-directional DS3 link between the Manhattan and Long Island facilities. At either end of this link are NetVXs. Using telco circuits provided by Verizon, with limited backup through existing station microwave facilities, they allow IP traffic, network connectivity and faster-than-real-time file transfer of video and audio.
Using NetVX's Virtual Channel feature, the stations can do their own bandwidth management and set specific QoS levels on the DS3 channel. Today, the DS3 is used asymmetrically. Approximately 40Mb/s of the 45Mb/s bidirectional DS3 data channel is available for customer payload. Currently, 38Mb/s is used in the Manhattan to Long Island direction, while 32Mb/s is used the other direction. The channel is split into three services: ATSC, NTSC and 10/100 IP traffic. Traffic going to Long Island includes 19.4Mb/s of ATSC traffic. The ATSC stream goes in as ASI, is output as SMPTE 310 and feeds the WLIW21 digital transmitter. For NTSC, 12Mb/s is used, and for 10/100 IP traffic, the allocation is 7Mb/s.
WNET uses Harris’ main and backup servers for ingest and playout.
A variety of services are included in the 10/100 IP traffic, which is allocated at 20Mb/s from Long Island. Both WLIW21's digital and analog transmitters are remote-controlled using ReCon. ReCon is Windows-based software that is designed to communicate with broadcast, net-work and facility control equipment using IP and the serial interfaces found on most modern broadcast equipment. Both applications require some bandwidth, but the major user of IP bandwidth is the ingest/playout system. It is essentially a remote node on the THIRTEEN/WNET system in Manhattan. Promos and other content can be ingested into the automation system in Long Island. As this is done, rather than inputting the necessary information into a database in Long Island, the information is input directly into the automation database in Manhattan over the network. As needed, material stored in Long Island is transferred over the network and into the servers in Manhattan. In addition to the 10/100 IP channel, the Long Island to Manhattan direction has a 12Mb/s channel for NTSC, which is used for transporting live studio production back to Master Control for broadcast.
The station's automation system consists of the LAN, ADIC archive and a SeaChange storage array, as well as four servers controlling the system, a main and backup for ingest and the same for playout. A remote client and remote (temporary) storage are in Long Island. The archive is used to store content that will air in the future, but not for at least two weeks.
Content management
The storage array consists of five RAID Arrays (nodes) configured in a RAID2 Array that allows not only for automatic rebuild from a failed drive, but also for automatic rebuild from a failed node. One useful feature of the RAID2 architecture is the ability to easily add nodes for additional storage. It is a simple matter to update each node's setup, and they do the rest. Once reconfigured, the nodes will determine the necessary info and begin the process of re-striping automatically. Depending on the storage, the number of nodes involved and the number of playout requests, the re-striping process can take from several hours to several days.
The station’s automation system includes the LAN, ADIC archive and a SeaChange storage array.
Between the automation system and the NetVX in Manhattan, a Flexicoder provides the necessary compression and encoding through a modular system that, like NetVX, can be user-configured and upgraded as necessary. Flexicoder can handle a variety of signal types and is capable of compressing multiple SD and HD streams simultaneously using statistical multiplexing and HD compression management (HDCM) software that dynamically allocates compression bandwidth for optimal picture quality.
Having the equipment in place is a requirement, but it is only the first step. Those who have worked with automation understand that it is also a process. This process involves a myriad of components all working together. Some components are software, others are hardware, and still others are human. To bring everything together harmoniously requires that all components function seamlessly.
Bringing everything together requires working a week in advance. Obviously, the traffic logs are prepared for all the program streams. Once the list of programming is complete, an inventory of required content is determined and compared with the content of the servers. Lists are generated that detail what is needed and where. Examples include satellite feeds, tapes from out of house, archived content and content that is, or will be, produced in Long Island, as well as content that needs to be archived. Each day these lists are worked and the content is moved, either automatically or manually into and out of the storage and/or archive. Only a few things do air directly from tape, and these are typically items that arrive shortly before air and will only air once.
There are several “rules” that allow content to be managed in the system in a straightforward manner. In addition to the rule concerning late arrivals, all content is removed from the system in one of two ways. One way is to move it from the server to the archive; the other way is to simply delete it. All content within the storage system fits into one or more of the following categories — either it has already aired, it will be aired within the next two weeks, or both. Content that has already aired and will not air again is deleted. If it will air again, but not within the next two weeks, it is moved to the archive. Later, that content will be moved back to the storage for broadcast.
Summary
This process will repeat until it's decided the content will not air again or the broadcast rights have been exhausted. At that point the content, along with its associated database entries and metadata, is purged from the archive and automation system.
THIRTEEN/WNET/WLIW21's content library churns nearly 100 hours of content each day. Accomplishing this with a small number of engineers and a single (large) server is possible through careful integration of equipment, people and processes. Automated content transfers, a straightforward method of data entry, and automated updating of information all contribute to an efficient and streamlined operation. Integrated components such as NetVX, Flexicoder and ReCon minimize system complexity, and hence the learning curve, while maximizing onsite flexibility and customization.
Automating multiple channels requires not only new equipment, but also new processes. Integrated content management, especially across multiple facilities, requires equipment, processes and systems that can function seamlessly across the distance. With all the inter-related systems, tools such as NetVX, which integrates more than a dozen individual pieces of equipment into a single device, can simplify long-distance network management and provide the tools needed for an integrated platform that will serve the needs of broadcasters today. With their software-based modular architecture, these tools are likely to remain in service for years to come.
Equipment List
James “Mac” Privette is the director of engineering, and Frank Graybill is the chief engineer at WNET.
Harris:
NetVX
Flexicoder MPEG-2
ReCon
Design Team
Automation system for MC operations
ADIC Scalar 10K Robotic tape archive
SeaChange Video File Server Array
Avalon Archive Management Software
Thirteen-WNET
James “Mac” Privette, dir. of engineering
Frank Graybill, chief engineer
Vince Lancellotti, assistant chief engineer
Harris Broadcast
Daryl Parry, systems engineer
