IP for Broadcast: A Conversation with Thomas Edwards of Fox

LOS ANGELES —As the vice president for engineering & operations for Fox network, Thomas Edwards is responsible for engineering and development of new broadcast technologies, including file based workflows. He has authored numerous papers (including an article on “Uncompressed Video Over IP” in TV Technology) and appeared on panels at industry events, offering his expertise on adapting broadcast operations to IP. We recently spoke with Edwards on the implications of this important transition.

TVT: What are the imperatives for broadcasters to transition to IP? How important is this transition to the industry?
Thomas Edwards: The main drive for moving from SDI/AES to IP is the need to enhance flexibility and agility of the broadcast plant. The business environment is changing rapidly—new channels launch, old channels go away, and there are new opportunities like OTT, UHDTV, HDR, HFR, 3D audio. IP can carry any format, and also it enables “broadcast DevOps,” the concept of deploying “agile infrastructure” through programmatic automation. The goal is to maximize the predictability, efficiency, and maintainability of operational processes. This can be best achieved when we have a fully virtualized, mainly software-based infrastructure for all of our video processing capabilities. IP is the key to making this possible.

There are also certain density advantages on day one. For example in the IP video routed mobile trucks from Game Creek used by Fox Sports, a portion of a single rack can provide routing for 6,900 uncompressed HD video signals. One HD-SDI cable can only carry one unidirectional HD video stream, but a 10 Gbps Ethernet connection can carry eight 720p active video streams in both directions, and using MTP/MPO multiple fiber connections, six streams of 10GbE can use a single connector, for 48 bi-directional HD video signals per connector (plus thousands of uncompressed 24-bit audio channels). This kind of density is needed for the large monitoring needs of events like the U.S. Open. At the cost of a bit more latency through the switch, JPEG 2000 encoding could be used to multiply the number of routed signals by nearly a factor of 10 (as is being used at ESPN’s DC-2).

Cost savings is also a possibility, as IP switches and servers are commodity products, used by every industry on the planet. The broadcast industry should be able to follow the economies of this huge scale for hardware. Our industry will need special software, of course.

I believe that this transition is going to be very transformative for the industry. In a few years, it will be typical to have an all-IP video plant.

TVT: Many elements of media facility are already IP-based. What do we mean by an “all-IP facility?”
Thomas Edwards: File-based workflows have flowed over IP for quite a while. Some real-time machine control has also been flowing over IP. But the “all-IP facility” takes the next step of moving live audio, video and other ancillary data over IP. The availability of affordable 10 Gbps Ethernet NICs and switches was required for uncompressed HD to effectively flow over IP inside the broadcast plant. Later this year, 25 Gbps Ethernet switches will show up on the market, which have the potential to be the lowest cost Ethernet per bit per second.

TVT: What can you do with an IP workflow that you cannot do with an SDI workflow?
Thomas Edwards: The final product of an IP workflow is unlikely to look much different than an SDI workflow. The advantage would be in flexibility and efficiency, and eventually cost savings, especially as the entire broadcast plant becomes virtualized.

TVT: What does SDI do that IP can’t do? Is frame-accurate switching the last hurdle?
Thomas Edwards: There really is no “last hurdle,” in a way we are there already. Fox Sports produced the Big East Men’s Basketball Tournament in March, 2015, using all-IP video routed trucks. What the industry now needs to do now is “perfection” of the idea. That includes using commodity Ethernet switches rather than bespoke broadcast-oriented Ethernet switches, moving all devices from SDI I/O to IP I/O, and eventually moving all video processing to virtualized software on commodity servers.

TVT: You and the engineering team at the Fox NE&O have successfully demonstrated methods of frame-accurate switching over COTS IP hardware. What is the best method so far and why?
Thomas Edwards: Each of the methods of video switching over IP (source-timed, switch-timed, and destination-timed) work. But they all have different trade-offs in terms of whether they can be implemented on commodity switches, how complex they are, and how much bandwidth they use. Fox and Arista Networks demonstrated switch-timed video switching at the SMPTE 2013 Annual Technical Conference & Exhibition using an FPGA-enabled Ethernet switch, and currently one broadcast vendor ships a switch-timed capable Ethernet switch as well. Fox and Arista then demonstrated source-timed video switching at the 2014 Hollywood Post Alliance Technology Retreat using a very typical commodity Ethernet switch, and at NAB 2015 another broadcast vendor showed a product with source-timed switching. Meanwhile, other vendors at NAB 2015 were showing destination-timed switching.

Destination-timed switching is probably the simplest way to switch video on commodity Ethernet switches, but it generally requires twice the bandwidth of a single video signal to be reserved. The good news is that Ethernet bandwidth at equivalent cost is rising all the time.

It is impossible to do switch-timed video switching on current commodity Ethernet switches, but it may be possible in the future due to rising requirements of SDN in the data center and carrier industry. Source-timed switching can be done on commodity Ethernet switches, but it requires some complex communication between a controller and video source nodes.

At this point, I suspect for the next year or two destination-timed switching will be most popular for all-IP plant builds, with source-timed switching reserved for smaller networks that are more bandwidth constrained. If the Ethernet industry decides to implement precisely-timed flow changes in the standard “merchant silicon” of Ethernet switches that may allow for switch-timed video switching on commodity Ethernet switches.

TVT: How does the workflow change, both within the media content workflow and with traditionally external business-related workflows?
Thomas Edwards: In the ultimate state of a virtualized, IP-connected video plant, a business requirement comes in to launch a new channel. You fire up a “new channel” wizard, type in how many channels of contribution feeds you need, how many video server channels you need, how many video switchers and graphics inserters you need, the number of emission watermarkers and emission encoders you need. The channel is then created in five minutes, and billed to your credit card. Your programming staff gets an email to begin populating the programming on the channel. The traffic system fires up, and notifies your salespeople to start selling, as well as activating programmatic ad sales opportunities through your Internet portal. Your video operations staff can KVM from their office (perhaps not in your building) to begin doing live production switching, and the channel goes on the air. No one moved a cable; no one installed a piece of equipment on the rack. The only thing that moved were bits over Ethernet: virtual machine images moving and spinning up, and Ethernet switch configurations being pushed. The elements of the new channel are automatically set up in independent dual-chain air/protect using IP-based hitless failover. Every signal in your plant is easily monitored at any point on your network. Logging is enabled on all aspects of the workflow, which allow you to perform deep analytics on your operation to aid in continuous improvements in efficiency.

Perhaps the video processing of your new channel occurs in your own private datacenter. Perhaps it occurs at a public cloud site, or a combination of private and public cloud as required to optimize price.

TVT: Describe the impact of adopting an IP-based workflow on content distribution and measurement.
Thomas Edwards: The all-IP facility does not necessarily have a direct effect on content distribution and measurement, but certainly moving from an exclusive model of “tied up” channels that live for years to the OTT model where video feeds to consumers can pop up for a while and then go away means that enhanced flexibility of the broadcast plant will become more important; this is where an all-IP facility can shine.

TVT: How does the cost of transitioning to an all-IP infrastructure compare to expanding the capacity of an existing SDI infrastructure?
Thomas Edwards: In the short term, I suspect costs will be on par with SDI solutions, especially as we live in a transition where most legacy devices are SDI, but are only converted to IP at “rack-top” converters. As more devices get native IP I/O, we can expect they will be increasingly packaged in commodity servers. The move away from bespoke broadcast hardware-like closed-caption inserters, frame syncs, D/As-will certainly save some money.

Many broadcast engineers have told me something along the line of, “automation never saved money on current operations, but it let us do more with the same pant.” It is likely the all-IP facility will be similar, costs may stay the same, but flexibility will become tremendously enhanced. But at some point we are likely to see real cost savings from hardware commoditization.

TVT: If you were designing a new facility now, would it be fully IP-based?
Thomas Edwards: You can certainly build an “IP video routed” facility today, but most devices would still need a “rack top” conversion between IP and SDI. Perhaps a year from now, it would be reasonable to build a playout facility with limited graphics insertion using all-IP I/O. In three years, I suspect there would be no question that a new facility would be built “all-IP,” regardless of type. At NAB 2015, we already started to see camera control units with IP I/O as well as production video switchers.