LOS ANGELES--About a year ago I wrote about REMI (REMote Integration) sports production (also known as “At Home”), which is a trend for covering live sports that requires less equipment and personnel on-site and shifts much of the live production activities to the broadcaster’s central production center/network center.

Enabled by robust and affordable telecommunications capabilities from the stadium/arena, the camera (and microphone) signals are sent back to the central production center, integrated with graphics, replays, effects and b-roll materials. Usually a small utility truck is sent to the remote site, with cameras and microphones as well as video encoders.

Local camera operators and utility crew roll out the cameras and microphones while an A2 is usually on-site (and may have driven the truck) to set up coms and in some cases provide a sub-mix of the natural sound (crowd). Examples of this type of production are used extensively in college sports (except football and men’s basketball) as well as some professional sports. In smaller productions (1-5 cameras) the truck is replaced by a portable field pack.

BEYOND THE CENTRAL PRODUCTION CENTER

However in many cases REMI still uses first-class professional cameras and production gear. These require operators who are well trained in the use of this equipment. For some sports, this is still a high bar in terms of cost, especially the huge capital cost to build such facilities.

In response to REMI, a new type of live sports production has evolved, one that does not need a central production facility and therefore can be produced anywhere there is a solid internet (IP) connection. Further, instead of hard-wired cameras and microphones, this new production called “Virtual Production” by Sony uses wireless connectivity from camera (and microphone) to a cloud-based production service. No heavy production equipment, no wires!

The sports event can be produced via an app on a tablet, laptop PC or even a smartphone. The “control point” can be augmented with multiple video monitors and audio speakers/headsets. The solution even includes return channels for communications (IFB) and video returns. Further, there can be multiple productions simultaneously using the same camera and mic feeds but outputting specialized line cuts. Finally, this may not be limited to just a professional producer, the same cameras and audio can be available to fans to generate a personalized feed that can be shared via social media.

Taking advantage of this freedom of wires and professional gear, even professional productions can cover parts of an event that are near impossible to cover using conventional TV coverage techniques. For instance, at the Red Bull Alpenbrevet alpine motorcycle race where there isn’t space or room for a production vehicle, a roving camera operator--connected via 4G wireless modem--can cover any part of the event that is accessible.

REQUIREMENTS

Is this Virtual Production (VP) the future of live sports coverage? What are the challenges and downsides of going virtual in the cloud? Some necessary technologies that enable Virtual Production are large wireless bandwidth on site with low latency, connectivity to the internet that is reliable and high bandwidth, and a VP service available and affordable in a data center relatively close to the event (to minimize delays). Finally, what is also needed is a production client app that can enable all the features needed by a production, with low latency of both signals and commands, that can run on typical laptops/tablets/smartphones with online support during the production in case of problems as well as help in setting up the VP service.

For now the offering from Sony is somewhat limited to switching up to six camera feeds, graphics insertion, limited effects (picture in picture, transitions) and audio mixing. Camera synchronization--and to some extent error recovery--requires around 2 seconds of delay, which is deemed to be acceptable for some productions. The other unique advantages that VP provides is a “pay as you go” approach to cost, thanks to implementing VP on the Amazon AWS cloud. For those interested in delivering via the internet using social media live features, this approach has a clear advantage of already being in the cloud. In addition, new features and functions can be rolled out without bringing the truck back to base and wait for the retrofit. Once upgraded on the hosted servers, the new features are immediately available.

[Read: HPA 2016: Remote Live Production]

The costs associated with VP are based on a monthly subscription for a certain number of hours of production. Included are up to six camera feeds, two playout servers, two logo keyers as well as the VPS production user interface. Additional costs may occur due to the wireless camera feeds that use wireless carrier 3G/4G (and in the near future 5G) data services. Finally, there is the cost to distribute the live feed over the internet, which could be free for some social media services (i.e. YouTube Live) or if using a commercial grade CDN such as Akamai, the cost is typically based on the number of concurrent streams. In addition, if the show is archived there will be costs to store the content and cost to retrieve it later.

OTHER VARIATIONS ON THE VP MEME

While the Sony VPS is a full IP/cloud-based approach using wireless cameras, there are other applications that take advantage of the new video technologies. NEP in their facility outside Amsterdam, offers a production facility for REMI production using their own data center, connected via fiber to major stadiums and arenas. The production facilities are rented (much like a remote truck) but only charged for the time used by the production. NEP has also used wireless IP cameras using point-to-point links (Ubiquiti Wireless) at the Coachella music festival to enhance coverage of the event. The use of IP connectivity allowed the production to remote control the cameras (Panasonic PTZ type). This obviated the need to run fiber optic cables to each camera in order to cover the sprawling event.

Teradek has a live production solution called Live: Air Action. Using their bonded wireless encoder/modems(VidiU or CUBE), connecting cameras (including iPhones/Android Smartphones) via a cloud server (Teradek CORE Integration) the Live:Air Action application lets a producer switch cameras, with limited effects, add graphics, record and playback, and has an audio mixer. Stream live over social media sites or via CORE.

Globo TV, the biggest TV network in Brazil, wanted to cover The World Surf League 2017 finals from Honolulu. Using three Sony FS5 cameras with VidiU Pro encoders with 4G LTE modems, Globo integrated with Live: Air Action to switch the cameras and stream using a Bond 4G/LTE modem to connect to Facebook live. Live:Air Action enabled the producers to add graphics including stats as well as integrate commentary and interviews with the third camera feed. Using the iPad app for Live:Air Action meant the producers could be close to the beach and the live event.

Newtek Tricaster has been used for years to provide alternative coverage, usually for social media or OTT streaming. They have developed an IP video networking environment, NDI, which allows for the use of IP cameras as well as interconnecting third-party equipment.

Recently Thaler Media used Tricaster and NDI along with LiveU wireless camera systems to cover the 2018 Western Amateur Championship in Northridge Illinois (near Chicago) while producing the live feeds at their facilities located in North Palm Beach Fla. The streams were sent to the Golf Channel’s digital platform.

On site at the Sunset Ridge Country Club were a producer, editor, four camera operators and sound mixer. Three of the cameras were mobile and used LiveU’s LU600 HEVC and LU500 IP-based transmission system to send the camera signals back to a LU2000 server located at the master control facility in Palm Beach.

The camera of choice was a Sony F55, which provided first class imagery. They set up a talk show set in the Sunset Ridge clubhouse and had the program’s co-anchors provided commentary and analysis, while a virtual set (greenscreen) provided the set background. An additional third anchor position was set up at the master control room in Florida, where the local program anchors could throw to the anchor at the main control studio. Tying it all together was a NewTek TriCaster TC1 IP production switcher with 16 inputs and 4 M/E decks, chromakey, graphics/titles, transitions, video server and virtual set support. NewTek 3Play replay systems provided SLOMO replays as well as aerial footage captured by camera flying on a drone over the course. Four Skype channels were used to provide return feeds to the Chicago on-air announcers including mix-minus and IFB.

While not up to the level of a major PGA event broadcast coverage, for the first time the Western Amateur Championship was covered live in HD, an event that in the past drew future golf greats including Tiger Woods, Jack Nicklaus, Phil Mickelson and Ben Crenshaw.

HOW WILL SPORTS BROADCAST PRODUCTION EVOLVE?

Clearly today there are many choices to capture live events in video, from big broadcast TV trucks to REMI to REMI in the cloud and finally, smartphone cameras and live video apps. For large events, with commensurate rights fees and audience sizes, professional broadcast OB vans, connected via satellite and/or fiber-optic telecom, will be the choice of live production for the foreseeable future.

On the other end, with all the available tech available in our hands (smartphones) coupled with social media sites providing distribution for live video, your child’s AYSO soccer match can be covered with multiple cameras, graphics and replays (and sound) to be seen by your friends and family on YouTube or any of the other social websites. In between big time pro sports productions events and Little League games, productions are using lower-cost solutions that use mobile wireless networks or WiFi networks, along with cloud-based apps to produce a video program that has many of the aspects of the big pro productions as well as providing professional level tools such as graphics and stats, IFB, and replays. Ingenious production teams, motivated to cover events that in the past were not economically viable for live coverage, are leveraging these new tools and providing to modest sized audience, content that in the past would have been just highlight clips in the evenings sportscast.

The keys to success is finding the right balance between cost and coverage, having robust wireless connectivity, reliable and deterministic Internet bandwidth, and a team able to adapt and learn this new way of producing live content. 

Jim DeFilippis is CEO of TMS Consulting, Inc., in Los Angeles. He can be reached atJimD@TechnologyMadeSimple.pro. See more at hisauthor archive.

NEW YORK — Even though the price of UHD/4K TV sets continue to fall, the fact remains that there’s still a dearth of available ultra-high resolution content and what content is — even in 2018 — is reserved for high-profile events such as the FIFA World Cup.

Except when it comes to over-the-top streaming services.

“4K streaming is leading the way right now,” said Will Law, chief architect of media cloud engineering at Akamai. “Satellite is bandwidth constrained at the present, and while cable is less constrained, these are still fixed infrastructure.”

Streaming services don’t have the same fixed infrastructure, even if the content is delivered via cable broadband, and as such it can reach a wider audience.

“OTT makes it easier to get the 4K content,” added Law. “Netflix has hundreds of titles already, and we’re seeing the other providers steadily adding content.”

Streaming services may be ensuring that content is future-proofed, so that resolution and overall picture quality don’t appear dated to future viewers.

“Netflix is already one of the biggest distributors of 4K content and treats higher-resolution content as premium,” said Richard Brandon, CMO of Edgeware, a Swedish provider of content delivery networks. “The OTT giant’s guidelines for ‘Netflix Originals’ state that programming has to be shot in native 4K so it’s been continually investing in UHD content to add to its library. At the same time the BBC is leading the charge in the U.K., streaming all its World Cup matches in 4K and will do the same with its Wimbledon coverage.”

BEYOND RESOLUTION

To TV manufacturers — and even the media that covers the market — 4K and UHD are being used interchangeably, but in essence 4K is just part of the bigger UHD picture.

“4K is the resolution, and this is one of many parts of UHD,” explained Law. “4K really is just focusing on the number of pixels. With UHD it is important to also discuss what this means for framerate — 30fps or 60fps — as well as High Dynamic Range, which is far more perceivable to viewers across devices than just the number of pixels.”

However, HDR isn't completely tied to UHD.

[Read: “To Be Or Not To Be UHD,” Is The Question]

“HDR is not part of the current UHD specs, but the two come hand in hand,” said Colin Dixon, founder and chief analyst of U.K.-based research firm nScreenMedia. “Everything that is UHD on Netflix is HDR as well, and we’re seeing that becoming true for Amazon and the other services as well. HDR is something you can see on all screen sizes, which isn’t really the case for just the resolution.”

The biggest issue with UHD/4K delivery is not enough bandwidth.

“The ability to deliver 4K is part of the attraction of internet-delivered services — it’s one of the ways operators can deliver an OTT service that goes beyond broadcast,” said Brandon. “The extra stress it brings to the availability of network bandwidth is an issue however, especially when streams are being delivered to several kinds of end-user devices.”

The issue will become more serious as more viewers access higher-resolution content, according to Jim Defilippis, CEO of TMS Consulting in Los Angeles.

“UHD content doesn’t represent a big cost increase to OTT services,” he said. “The costs are related to the number of customers, so when you have only a few customers it really doesn’t cost much. The bad news is that the costs will go up as you gain customers.”

DATA LIMITS

In addition to availability of UHD content, another limiting factor is data limits.

“You are good to go if you are streaming a movie or two a week, and in that case you won’t reach your ISP’s bandwidth limit,” said Dixon. “If all of the content you are watching is in UHD you’d have a problem.”

Dixon estimates that no more than 20 percent of the current U.S. market that regularly watches streaming services is consuming UHD content. “That market is growing, and if 100 percent of the audience suddenly embraced UHD, the broadband providers would have to build up the pipes,” he said.

“4K streaming is leading the way right now.” —Will Law, Akamai

Some vendors, including Harmonic, provide solutions that handle the growth in adoption of UHD, and reduce latency. This has included its EyeQ system that is designed to reduced bandwidth consumption by up to 30 percent without requiring any additional changes in the existing H.264 delivery infrastructure or the vast array of decoders on the market.

“We are working to improve UHD streaming, especially for sporting events,” said Eric Gallier, vice president of marketing at Harmonic. “Our solution provides latency that is comparable to what you’d see in traditional broadcasts. This is not only about the encoding but also about the playout.”

At the 2018 NAB Show, Harmonic unveiled its new end-to-end UHD HDR solution for live sports streaming. The solution features Harmonic’s ViBE CP9000 contribution encoder for VOS 360 media processing SaaS for encoding, packaging, origin server capabilities and OTT distribution.

HURDLES LEFT TO OVERCOME

Bandwidth aside there are still other issues to resolve, especially as there are a lot of technologies at play.

“Streaming isn’t all unicorns and ice cream cake right now,” said Akamai’s Law. “We need to resolve the standardization of the various HDR protocols as there are currently too many. This creates fragmentation in the market and that could drive up production costs.”

Copyright protection and DRM could also remain an issue, but so too could piracy of UHD content.

“Pirated content is usually degraded in quality from redistribution and added compression, but when a stream starts in 4K, it can be degraded but still remain high enough quality to watch,” said Edgeware’s Brandon. “Content providers can overcome this by deploying solutions that allow them to add a digital watermark to each stream. This can then be used to identify where a stream was stolen from, letting users better protect their investments into 4K content.”

THE NET NEUTRALITY FACTOR

Since FCC Chairman Ajit Pai announced last November that net neutrality would be repealed, many industry observers have warned that streaming media could suffer but others are now saying the issue is being unnecessarily politicized.

“We are working to improve UHD streaming, especially for sporting events.” — Eric Gallier, Harmonic

"Many are concerned about distribution equality," said Law. "Net neutrality can impact the economics of how content is delivered from source to the last mile."

High-bandwidth content, however, could end up costing more just to get to that last mile.

“When there is an outlay from a service such as Netflix that is delivering a lot of content to other providers, this is when it becomes an issue,” said Dixon at nScreenMedia. “If a lot of 4K content crosses the wires from any internet company there will be an outcry from the ISPs for the transit cost. That is where some companies have had to buy ‘preferential pipes’ — that is in essence what Netflix is already doing.”

Part of this ongoing issue reaches back decades to how the internet was developed as an interconnected “network of networks,” where content passes through from one provider to another.

“Net neutrality is the key to the operation of gateways between internet providers and the peering agreements that insure equal treatment for traffic as long as the traffic is balanced between the providers,” said Defilippis. “In a non-net neutrality situation, an internet provider is free to charge for improved gateway performance (bandwidth and latency). 4K content is sensitive to these increases due to the size of the files and the bandwidth required.

“UHD will amplify the throttling issues which are bound to happen with the change to net neutrality,” Defilippis concluded.

[Want more information like this? Subscribe to our newsletter and get it delivered right to your inbox.]

LOS ANGELES—NEWS FLASH: There is a huge shift in viewing of media (TV and movies) from traditional distribution outlets (cable, satellite, broadcast) to the internet, including both fixed and wireless as well as mobile devices…OK, we all know this, just wanted to get your attention.

While the established modes of media distribution are well understood and engineered, governed by open standards (ATSC A/53, SCTE 23 (Docsis), DVB-T, DVB-S, MPEG) the internet is an amalgam of “open standards” (IETF) and quasi proprietary approaches (Apple HLS), as well as a variety of software protocols and applications (web browsers, media players, etc.). Further there is a loose form of registration (ICANN) and operational principles (peering) that govern traffic and routing on the internet. 

This looseness, with a minimal of standardization, has encouraged the development of new feature-rich applications that empower people to consume media content anywhere and anytime they have access to the internet. On the minus side, the “core internet,” which has evolved over time, is not designed to expand at the rate of consumption of video. Unlike other forms of internet traffic, media streaming/downloading requires certain performance criteria:

1. Low latency

2. High end to end bandwidth

3. Low packet loss

4. High storage capacity

ONE TO ONE

One of the key differences between the internet and traditional video distribution is that the internet is based on datagram transmission protocols, not switched circuit technology nor linear channel distribution (cable, OTA, satellite). In datagram transmission protocols, the video content is put into sequential packets of limited size and then each packet is launched over an IP network. Each request for content requires unique packets be sent to that user, thus 1 million viewers means 1 million distinct video streams. Linear channels can support multiple receive points with one common video feed.

Along the way, these packets are buffered and routed to their ultimate destination. Because there is no centralized routing or path assignments, the route of any two packets of the same video stream may or may not traverse the same set of routers and links thus experiencing different transit delays. The IP packet video receiver has to buffer the incoming packets, perhaps re-ordering them, prior to processing (decoding). This buffering introduces additional delay. In contrast, in a linear channel or circuit-switched transmission, the routing of video data is fixed thus the received video packet order is fixed, minimizing buffering and delay. There may be an initial delay in setting up the circuit or tuning to the desired channel, but once the streaming has begun, a minimum of delay will occur from receiving the compressed video data and decoding back to base band image data.

THREE METHODS OF SEGMENTATION

To overcome some of the challenges of data packet transmission, a technique known as segmentation has evolved. There are three competing approaches: Microsoft Silverlight, Apple HLS and Adobe HDS. Basically, the media is encoded in bundles of discrete segments of content length (from milliseconds to several seconds usually) and are then sent as a “package.” Once the first package is received and verified, decoding starts while the next package is being received. As long as the transmission delays are less than the segment content length, the receive buffer is kept from emptying, which prevents stalling the decoder output. 

However this only solves the delay/re-buffer problem. If the end-to-end IP bandwidth is not stable and drops below the criteria of delivering packets faster than the decoder is pulling them out, then buffer underflow will occur causing freeze frames or black frame output. An advantage of the internet is the inherent bi-directional connection; the receiver can report back to the video server statistics on the recovery of packets. If the packet receive rate is too slow for the current encoding bitrate, the server can “switch” to a lower bitrate. For the segmented protocols, this usually means swapping bit rates at segment boundaries. Likewise, if there is more than sufficient bitrate, the server can raise the bitrate and improve the overall video quality. For circuit-switched or linear channel-based delivery, fixed bandwidth is guaranteed along with a fixed transmission path.

From a practical point of view, given that encoding is processor-intensive and each bitrate rendition require more storage at the server, video servers have a fixed number of encoded bit rates available, typically four. The server selects the appropriate bitrate segment for the viewer encoded in the bitrate below the reported receive bitrate. A further refinement of this approach, called CAE (Content Aware Encoding) optimizes the bitrate per segment, with multiple tiers of quality performance. For a given quality, the bitrate will fluctuate between segments; if the available bitrate is insufficient, the server will select a tier (or ladder) of less quality/lower bitrate.

Other techniques to manage video delivery over the internet include transcoding at the edge (where the origin video is re-encoded to fit down local IP connections), multicast (similar to a linear channel approach), shared caching (where receivers share data streams) and peering (similar to Bit Torrent). 

HANDLING THE CROWDS

The reality of internet delivery of media content is a complex topic. There are physical constraints, such as bandwidth, storage, processing, as well as electrical power (it is estimated that in the U.K., 16 percent of all power generated is used by the internet data centers). While there are traffic flow models for the internet, media streaming does not fit well into these models, so we have to use empirical measures of how well the internet is handling video. We know many “peak event” viewings of live content (season premieres, live sporting events, etc.) cause either slowdowns or disruptions to the video feeds. The causes are manifold—maybe over-subscription of the origin server or if the client side connectivity degrades or the internet core routing is over-taxed or if the ISP gateway is overloaded. CDNs in part, mitigate these peak flows and attempt to route around bottlenecks. But the root cause is larger audience sizes coupled with increasing bandwidth requirements of video content (HD, 4K, high frame rate). Since each viewer receives a unique media stream, as the viewership grows, the total internet bandwidth grows. 

The current estimate is that around 70 percent of all internet traffic is due to media consumption. This means that everything else (email, financial transactions, web browsing, etc.) is 30 percent. However, it is interesting to note that the 70 percent represents many duplicative feeds—if 1 million people are interested in viewing the World Champion Darts competition finals, then that represents 1 million streams at some bitrate (1Mb/s to 4 Mb/s typically). But at the same time, there can be many other instances of stream viewing with an audience of a single viewer to over 1 billion viewers. While CDNs can scale and create multiple delivery pipes—and multiple origin servers can be made available, thus spreading the load over a broader set of servers and data connections—there is ultimately a finite resource of bandwidth and processing nodes. 

THE BOTTLENECK

Much like the traffic on the 405 in Los Angeles, there are finite resources (lanes) that can carry cars. We can carpool or reduce the size of the lanes (compression) but this solution has a linear effect on the problem while the growth of internet video traffic is exponential.

Another way to look at this is Compound Annual Growth Rate (CAGR). Video streaming is growing at a CAGR of 30-35 percent; video compression over the last 25 years has improved around 7 percent CAGR (halving the video bitrate every 10 years). Unless video compression has a major breakthrough (15:1 improvement in compression efficiency) compression alone cannot solve the internet video bottleneck. (Note that 15:1 compression improvement does not take account of the increase in bitrate due to increasing video format size or frame rate.) More and more content delivered over the internet is HD and above (4K, eventually 8K), which needs more bandwidth than the current mix of internet video streams.

The solution has to be found in alternate video protocols for distribution of video over IP. The 1:1 relationship between the viewer and a unique video stream is unsupportable with the growth of consumption of video over the internet. The challenge is the desire for personalized delivery, one that can be controlled by the viewer using many different devices (mobile, fixed, PC), connection types (wired and wireless) and viewing conditions. Some proposed solutions are variants of multicast protocols, but multicast means “appointment TV” viewing or having to record each multicast feed locally in an appliance. However this does not support “impulse viewing” if that multicast feed was not previously subscribed to by the viewer. Perhaps a hybrid approach where there is a “linear feed” multicast to millions and 1:1 individual streams for those that either come late to the party or want a unique viewing experience (such as a “cut down” version of a live event, minus all the game breaks). Maybe.

Well let’s hope that 2018 will be the year that someone figures out a solution to the bottleneck.

LOS ANGELES—Sports broadcasting is one of the more demanding activities in television. Specially built trucks (or “OB Vans”) roll in the night before or perhaps the early morning. Utility crews and technicians converge and start rolling out cables and gear to be installed for the game later that day. The production truck (or trucks) is powered on, sometimes with shore power, sometimes with a generator, sometimes both. Inside, the production crew preps for the day, prebuilding effects and graphics, editing b-roll packages, checking the cameras and microphones, setting up comms, connecting back to the network center via fiber or satellite. As the game nears, the director meets with his camera crew, the producer works with the announcers, and everyone else is busy transitioning from setup to operations.

Crews are either locally sourced or travel in; hotel arrangements, local transport and meals are additional logistics to contend with. Veteran crews make it seem easy for the ‘A’ game, but not so much for the B- and C-level productions that may look more like an amateur team using antiquated gear.

It’s a crazy way to cover events that are prescheduled at locations visited many times during the year. But what if there was a way to produce the event from a central fixed location, maybe thousands of miles away? What if the local requirements could be reduced to the core functions while all the elaborate production is done from a full-time production facility with teams of well-trained production personnel? There would be no need for a large specialized vehicle, representing millions in television production equipment on wheels and no travel and hotel or meals to organize and pay for. 

TWO DECADES OF EXPERIENCE

Well, it’s been happening for quite some time. In 1996, NBC Sports took advantage of Atlanta’s proximity to New York and leveraged their new digital equipment infrastructure (Genesis) rebuild at 30 Rockefeller Center to do preproduction and graphics builds for the Olympics, as well as highlight packages remotely in New York. All it took was multiple digital video circuits between the International Broadcast Center at the Georgia World Congress Center in Atlanta and 30 Rock, the will to do it, and the coordination and communications to make it feasible. 

In the last few years, sports networks have increased their efforts to enable “At Home” or REMI (REMote Integration) production, especially for college sports. The key factor is access to high bandwidth/low latency connectivity coupled with a flexible work force that can be trained to do the localized work needed at the remote site. Fixed facilities coupled with a small vehicle to transport the production gear (mainly cameras and microphones) to the stadium or arena enable the central production facility to put on live sporting events while located miles away. 

Big Ten and Pac 12 were two of the first collegiate sports leagues to go down the At Home path. Then ESPN with the SEC Network as well as other college sports have deployed REMI to take advantage of the newly built all-IP production facility in Bristol, Conn. (DC2). But it hasn’t been just for college sports; ESPN used REMI extensively for the 2016 X-Games.

HOW DO THEY DO IT?

Aside from fiber interconnect with bi-directional Gb/s IP pathways, At Home productions have taken different approaches to the on-site equipment and operation. For some college sports, such as swimming/diving, wrestling, volleyball, etc., Pac 12 developed a mobile rack with all the terminal gear to interface to a high speed internet connection and deploy up to six cameras and microphones.

For larger events or outdoors, a small van or truck arrives on the scene and a small local crew deploys the equipment (cameras, microphones, terminal gear) and then operates the cameras and creates audio submixes. At some events, a hybrid approach is taken where there is both on-site production and camera and microphone signals are transmitted back to the central production facility.

For those events where fixed connectivity is not available, wireless connectivity via 4G/5G mobile bonded cellular can be used to stream camera signals to a central production facility. NESN, with production facilities in Boston, uses cellular camera backhaul to produce pre- and post-game coverage, live reports and press conferences at major sports events.

“The Switch,” a global media and transmission solutions provider focused on high quality live video transport, offers a cloud-based live production environment called “Cumulus.” Coupled with “Home Runs,” their video transmission service uses DTM (Dynamic Synchronous Transfer Mode) to enable At Home production. DTM, unlike IP, provides dedicated bandwidth with guaranteed QoS. By enhancing Home Runs with Cumulus, The Switch demonstrated a complete television production at the 2017 NAB Show that included a SkyCam aerial camera system--"Sky Command"--in Denver (Dick Sporting Good Park), with remote integration from The Switch studio facilities in London and Los Angeles with SMT's designed 3D graphics and patented Camera Tracker Technology, which allows virtual graphics to be inserterd over a moving camera, controlled by personnel located at The Switch booth in the Las Vegas Convention Center.

Meanwhile, over in Europe, an example of REMI—as well as cloud production—is in operation. The NEP Hilversum facility, just outside Amsterdam, has connectivity with a telecom provider that has access to high-speed IP connectivity to a variety of venues, including major soccer stadiums. By leveraging existing facilities at a media center in Hilversum, producers can book studios and control rooms, as well as telecom and data center functionality to produce an event, pay only for the time used and then move on while the next production moves in. 

The latest event in Europe involved a REMI “proof-of-concept” that proved that live uncompressed camera signals can be sent over 1,000 miles to a production center where a live sports event is produced, with replays and graphics as well as enabling streaming, 360-degree VR and archiving functions. Gearhouse Broadcast, using gear and software from Snell Advanced Media (SAM), delivered live signals from the UEFA Under 21 Championship Final in Krakow, Poland, to the BT Sport Centre at the Queen Elizabeth Olympic Park in London. Five Sony HDC 4300 4k/HD camera signals were sent via dual redundant 100GB Ethernet links, which were provisioned by Level 3.

THE CHALLENGES

Live television requires tight teamwork; being on-site, working in close quarters, and knowing you are on your own at a remote site brings out the best in everyone. The downside is the cost and time lost due to travel, setup and breakdown. REMI and At Home brings the remote camera and audio signals back to a fixed facility, which can be used for multiple events throughout the year without having to travel thousands of miles, perform set up and breakdown multiple times. The production team is sourced locally so travel is minimized but the quality of production can be consistent. 

But there is a clear requirement to make At Home work: reliable, affordable and low-latency transmission facilities between the remote site and the central production facilities, as well as the need for reliable, low-delay communication systems to coordinate between the remote site and the central production team. In addition, local camera operators, audio/coms specialists and other skilled operators have to be sourced. There is a growing shortage of people trained in these skills as the role of traditional broadcast operations shrink. Training new hires to replace retirees is going to be needed for At Home productions to be successful. 

WHAT DOES THE FUTURE HOLD?

As the “‘big kahuna” of production, live TV requires planning, reliability and attention to detail, while having the ability to pivot and deal with any contingency. At Home mitigates some of the risk by centralizing operations and facilities, thereby improving efficiency and reducing costs; but there is still the need to rely on local talent to operate the field equipment. 

With the industry transitioning to a “cloud” approach, where all signals are in a data center, individual production team members can be anywhere as long as there is reliable, fast connectivity. However the key to successful REMI production will be access at all sites to a reliable, high-speed, low-latency telecom network that is both ubiquitous and cost-effective.

Jim DeFilippis is CEO of TMS Consulting, Inc., in Los Angeles. He can be reached at JimD@TechnologyMadeSimple.pro. See more at his author archive.

LOS ANGELES—Not a day goes by there is not an article or announcement regarding video distribution over the internet. Coined, “OTT” as in, “over-the-top,” these offerings include video-on-demand, live streaming, user-generated content (e.g., You Tube), as well as news clips, marketing and advertising.

The key feature of OTT is that it’s individually targeted. The consumer chooses what, when, where and on what device to view content. Unlike “one-to-many” broadcast technologies, internet protocols are designed to move data point-to-point. OTT video distribution can deliver content to each viewer, tailored to their connection speed and viewing device, as well as collect viewing information relevant to marketing and advertising.

BOTTLENECK CHECKS
As OTT viewership has increased, internet limitations have become bottlenecks. The first solution was to move content closer to the users, or “the edge,” using private content distribution networks, or CDNs. Last-mile bandwidth choke points are another issue. The solution has been the use of dynamic adaptive streaming over HTTP, or DASH, techniques, including Apple HLS, Microsoft Smooth Streaming and Adobe Dynamic Streaming. These employ racks of servers to create versions of the content, at different resolutions, bitrates and frame rates; that can be switched out dynamically, depending on bandwidth availability to the client device.

In parallel, there have been improvements to video and audio compression that allow for lower bitrates, HEVC, VP9, MPEG-H, AC-4, for example. However, there has also been an increase in demand for higher quality video, up to 4K (3180x2160p24/25/30/60) as well as in high dynamic range and high frame rate. The latest MPEG standards for video, HEVC and Google’s VP9, can deliver 4K at bitrates from 4 to 20 Mbps. So while CDNs and DASH have improved streaming performance, the demand for live streaming and 4K content will continue to challenge OTT delivery.

As the video streaming over the internet load has increased, IP switches and routers have mitigated this increase by increasing the port line rate, up to 10 to 25 Gbps, as well as adding additional buffering to prevent packet loss. However buffering increases the end-to-end delay, or latency, which degrades the quality of service of the video delivery. Higher port line rates require higher internal router and switch bandwidth, as well as faster processing speeds.

MORE POWER, MR. SCOTT!
What other challenges exist? Higher video data rates require more processing power for compression, more fiber capacity—from 10 to 100 Gb—and more energy:
CPU power = CV2fc, where C is capacitance, V is voltage and fc is the CPU clock frequency.

Additionally, more storage is required for the additional versions that are generated (from high-dynamic range 4K to mobile versions). Higher capacity fiber interconnections require IP switches that can handle these higher line data rates, which increase their clock speeds (power) and, as mentioned above, increase the throughput latency.

Potential solutions have been proposed to reduce the overall internet loading due to video streaming, reduce latency and increase video quality of service. One potential solution is to use multicasting techniques. Multicasting is a mode in IP protocols that uses reserved IP addresses to “publish” a common data stream. This technique has been used within closed private data networks, but has yet to be done effectively for internet delivery of OTT services.

IPv4 SQUEEZE
One challenge is that multicasting requires internet service providers (ISPs) to agree to support specific multicast protocols, some of which are proprietary. The second challenge is that it would require either a one-size-fits-all approach of a single video version, or multiple versions of multicast streams to enable DASH protocols. There are 16.8 million IPv4 addresses assigned to global, internetwide multicasts, which may seem more than enough, but if there are just four versions of each video stream, that limits the number of simultaneous video streams to 4.3 million. IPv6 multicast address space is 1.4 x 1036 , so this will not be a problem once IPv6 is in wide use. Another issue with multicast delivery is the need for a protocol to distribute the equivalent of a program guide to all devices for providing information on which video streams are assigned to which multicast addresses.

Another solution is to use peer-to-peer (P2P) techniques. In P2P, edge terminal devices can communicate with other edge devices and share data streams, thus increasing the number of processing nodes, storage and individual bandwidths. This approach would need the user to agree to use a specialized application that can enable P2P. The challenge with P2P is congestion at the local area network as well as privacy and security.

HYBRID BROADCASTING
In Europe, there is an alternate approach to OTT called “HbbTV,” for Hybrid Broadcast Broadband TV. In HbbTV, programs are transmitted using traditional broadcast methods as well as IP streaming. The HbbTV standard allows for the synchronized reception of both signals, thus supporting second-screen viewing as well as delivering enhancements to the broadcast signal that also support interactivity.

In the United States and Korea, the ATSC 3.0 standard has hybrid technologies built in; thus ATSC 3.0 services can be delivered OTT or over-the-air. In addition, ATSC 3.0 employs watermarking technologies to carry the linkage and synchronization information within the program video and audio signals. ATSC 3.0 also incorporates the ability for interactivity, audience measurements, targeted commercials and second-screen applications. ATSC 3.0 is also working on a specification for an application frame work for receiver TV applications.

The OTT ecosystem has many parts—content preparation, content curation, origin servers, distribution servers, CDNs and user applications and players. Some key vendors in the OTT space include Akami and Limelight for CDNs; Elemental and Enivio for encoding and transcoding; MLBAM and Neulion for platform and app development; and Netflix and Amazon as content distributers. There are a myriad of medium to small companies that provide software or services to content distributors.

THE WORLD WILD WEB
Finally Net Neutrality. The internet is not only an amazing technical development, it is also a novel social, business and legal construct. A key component of the internet has been the concept of “Net Neutrality.” Net Neutrality is the principle that all bits are equal, that there is no preference for one set of bits over the other. From a consumer standpoint, this means that all OTT providers have the same opportunity to supply content without any discrimination.

However because Net Neutrality requires any improvement to be available to all internet services, there is no incentive for any one service to pay for improving internet performance, such as increasing the bandwidth limits at an ISP gateway. Some wireless providers have offered “unlimited” access to certain video services, while other services would be under the customer’s data cap. Some cable ISPs have attempted to limit the amount of data for some customer activities, such as P2P streaming. Net Neutrality is in a state of flux and being re-defined by the FCC and the Congress. Stay tuned for updates as they occur.

In summary, OTT has opened up many new opportunities to distribute content to viewers. Viewers have choice not only in programs but in how and when they can watch content that they are interested in. Unfortunately, the IP technologies that the internet is built on, are not easy to scale as OTT becomes more popular. OTT and VOD workflows have changed the distribution workflow for TV. No more is it tape machines, playout servers, master control and automation. Now, we have software-based content management, transcoding, origin servers, authentication servers, distribution servers and CDNs.

There are business, legal and societal challenges that are in continual evolution. The current challenge for OTT is true live streaming of popular content such as sports and news. Live streaming requires lowest latency possible, while handling millions of simultaneous viewers, and delivering good quality video and audio.

Perhaps OTT will replace broadcast distribution technologies. However, today’s existing broadcast TV distribution methods of OTA, cable, satellite and fiber, use well-proven and widely deployed technologies to deliver TV programs, live or prerecorded, to the vast TV audience, without limits to viewership, quality and with minimum delay.

LOS ANGELES—If you’ve spent more than a microsecond in TV, you know something about time code. It seems to have always been around and indispensable to TV production. Whether it is used as a time reference on recordings, on an edit time line or part of a program delivery format, time code is ever present.

There was a time before SMPTE TC (B.S.TC). These were dark days of using film (foot and frame counts) or basic control track tick marks or just a stopwatch. We’ve grown up with SMPTE ST 12-1 (originally ANSI C98.12.1975), but what is time code exactly? I’d like to give some background and then discuss new methods that will replace ST 12-1 in the not too distant future.

COUNTING FRAMES

First and foremost, ST 12-1 is a counter. At its base, it counts frames of video (not fields per se) as well as relates these frame counts to wall clock time, either absolute or relative. Born of videotape editing requirements, the first version of SMPTE TC was a linear signal, designed to be recorded on an analog audio track of a videotape machine. Limited to just 80 bits of data, the format is a bit arcane, using truncated BCD (binary coded decimal) notation for the frame counts and time values (H:M:S:Fr) along with some flag bits while the leftover bits were assigned as “user bits.” One important flag bit that is still in use today is the field bit. The field flag bit indicates the “phase” of the field cadence, where zero can mean either the top field (25Hz systems) or the bottom field (30Hz) systems and 1 means the other field in the sequence. In a revision of ST 12-1, the field flag has been defined for progressive video systems to increase the frame count to either 50Hz or 60Hz. 

User bits have been assigned in public documents as well as for private use. One has to be very careful in using user bits, because their meaning can change depending on the context of the project. 

As time went on, linear time code gave way to Vertical Interval Time Code (‘VITC’). Originally VITC was an analog waveform, superimposed on an active line in the vertical blanking interval of a TV signal. VITC was present in both fields of an interlaced video signal.

As video became digital, it was somewhat awkward to digitize an analog wave form of a digital data stream. So, a variant of VITC, called Digital Vertical Interval Time Code(DVITC) was developed. Because component digital video signals did not have a vertical interval, time code had to be re-packaged as pure data into the Vertical Ancillary Data Space (VANC) but formatted to “look like” video samples. 

Then along came file formats. Here time code would be decoded to binary code values, sometimes converted to different formats, and then embedded into files, either as a “time code track” or in the header/footer of a media file. MPEG transport streams can also embed time code in the ancillary packet header of a MPEG TS packet, thus time code and compressed video can be distributed either as a bit-stream or as a file type. 

Recently, SMPTE ST 12-1 was extended to provide frame counts above 60Hz. This is now documented as ST 12-3, and uses some of the binary flag bits as a modulo counter of “sub-frames” that are extensions of the classic “super-frames:” 24, 25, and 30. Thus any frame rate that is a multiple of 24, 25 and 30 can be extended using the five binary flags, for maximum frame count of 960 (30x32).

Okay, you may have noticed that I have not discussed “drop frame” time code. So not to go too deep into the ancient reasons why in NTSC countries the frame rate is really 29.97Hz (30*1000/1001), this causes a disconnect between the time code, time values and the real time clock time. The accumulated error is approx. 3.6 seconds per hour or 1.5 min per day. To compensate for this error, SMPTE ST 12-1 defines a special counting of frames that “skips” certain numerical frame counts on a regimented basis, so as to “catch up” to real time. While not perfect, there is residual error at the end of the day of about 86 mS or 2.59 frames, it is “good enough” to edit to time and deliver shows and commercials accurate well within a frame time. (By the way, drop frame time code is only corrected up to midnight. After midnight the error will accumulate unless a “jam-sync” is forced to reset the time code clock to real time. This has been acceptable practice for many years and most practitioners know that if there is a recording over midnight, there will be a time code jump (which usually is fixed by re-stripping the time code when ingested into an edit system).)

MODERN TIMEKEEPING

Starting back 10 or so years ago, some engineers thought of a better way to both synchronize video (and audio) as well as represent frame counts (or sample counts) and to have a system that is not only time accurate but in fact is traceable back to a master clock reference such as GPS.

First, some background on the nature of modern clocks as used in computer networks: You may be familiar with Network Time Protocol (NTP); this is the method that computers can sync their clocks over an IP network to a master clock. It works pretty well for conveying time with reasonable accuracy (~1-10 mS) but is not good enough for synchronization of video signals due to the variable latency present in IP networks. IEEE, realizing that the precision, accuracy and stability of NTP was not sufficient for critical real time applications, developed Precision Time Protocol. PTP or IEEE 1588 defines a protocol over IP as well as the requirements for master clocks (Grandmasters) and slave clocks. PTP provides accuracy and stability of timing (~0.1-10 uS) sufficient to synchronize video as well as audio and other signals. 

Another important feature of PTP is the concept of an Epoch. An Epoch is a “start point” in time that defines the “zero” count. Time is then measured from the Epoch to the present using a precise frequency of any unit desired. One can use seconds, milliseconds, pico seconds or frames of video (with a defined frame time = 1/frame rate) or audio samples (1/sample rate). SMPTE has defined the Epoch to be Jan. 1, 1970, midnight Greenwhich Mean Time (GMT). Further, SMPTE has defined the “start point” of the Epoch to be the start of vertical sync reference for all formats and frame rates. The SMPTE ST 2059-2 defines the profile for the use of IEEE 1588 for Professional Broadcast Applications.

Thus by counting time from the Epoch, and knowing the exact frame time, one can determine the offset between two video signals, providing that each of the video signals were created while connected to a known PTP reference (see SMPTE ST 2059-1 Generation and Alignment of Interface Signals to the SMPTE Epoch). Of course it is not always possible to be connected to a reference, but we can always “time sync” and bring the unreferenced video back into alignment with the Epoch.

NEW TIME LABELS

Back to time code. The SMPTE standards committee for Time Labeling and Synchronization contemplated modifying ST 12-1 time code to carry some information to be able to tie back to PTP and the Epoch. While ST 12-1 time code has user bits and binary flag bits that could have been redefined, an exhaustive search for all the current (and legacy) use of these bits showed that this would be very disruptive. Further, this approach would be carrying forward a legacy of a digital signal coded for an analog world into an all-digital IP packet environment. So the committee decided to develop a new time code, based on PTP and the new video sync standard. The criteria they developed included:

§Simple

§Adaptable

§Human Readable

§Carry additional information

§Convertible to/from ST 12-1

§Packetized for delivery over SDI, IP, MPEG TS, etc.

§Stable and accurate over long periods (weeks, months, years)

Two approaches have been developed by the drafting groups. One is a called Generic Time Label (GTL) the other Time Related Label (TRL). These time labels represent a running count of media units such as video frames and audio samples, counting from a known time reference such as the SMPTE Epoch or a private Epoch. The difference between these time labels is that the GTL carries just the count along with rate and origin metadata. The TRL defines a set of objects that can be used to carry a variety of time code and label information as well as a media count. Both TRL or GTL time labels can be used to relate the media counts to real clock time and dates as well as legacy ST 12-1 Time Code values, however the TRL allows for the direct embedding of these alternate formats within the label as well as additional metadata related to the media, including legacy ST 12-1 flags and binary groups. For both TRL and GTL the underlying time reference is based on the ST 2059 SMPTE PTP standard.

Recently Howard Lukk, the new SMPTE Director of Standards, organized three “summit” meetings, one each in Los Angeles, London and New York. The purpose of these meetings was to gather input from the user community with regards to their needs for a new time label. For those that could not attend, you can take the time label survey at www.SMPTE.com/timecode.

So we’ll have ST 12-1 Time Code around for the foreseeable future, but be ready to deal with these new time labels as well as the new SMPTE ST 2059 time and sync reference standard based on IEEE 1588 Precision Time Protocols.

Jim DeFilippis is CEO of TMS Consulting, Inc., in Los Angeles. He can be reached atJimD@TechnologyMadeSimple.pro. See more at hisauthor archive.

LAS VEGAS—The video industry is moving into more—more resolutions, more color, more contrast, more frames. Ultra high-def 4KTVs are the new HD. More and more sets feature high dynamic range. Wide color gamut is on the way. The questions now revolve around the workflow—achieving these dynamics glass-to-glass—whether or not it’s worth it, and if so, what’s the best approach.

That’s what four top broadcast engineers take on during the NAB Show in a Tuesday, April 19 at 2:30 p.m. Super Session entitled, “4K, UHD, HDR and More—The Future of Video.” TV Technology’s Deborah D. McAdams will grill video expert Mark Schubin, SMPTE President and CBS Vice President of Engineering and Advanced Technology Robert Seidel, broadcast veteran Jim DeFilippis and Ericsson’s Senior Vice President of Technology for TV & Media Matthew Goldman with crowd-sourced questions from their peers.

Here, Jim DeFilippis takes a crack at the list.

CROWD MEMBER: What provides the best bang for the buck: 2160p resolution, high dynamic, wide color gamut, or high frame rate (120 fps)?
DeFILIPPIS: Good and interesting content! Ok, each provide a unique ability to capture and display content.

CM:The Canadians are now doing baseball, basketball and hockey in 4K, and distributing it via cable. What is holding up the U.S. broadcasters? What are their immediate challenges?
DeFILIPPIS: The difference is that this is one cable/broadcaster (Rogers Communications). DirecTV and Comcast have committed to delivering 4K (Masters Golf, Rio Olypmics). 

CM:What about standards? What’s needed?
DeFILIPPIS: We have many standards, but what we are missing is clarity of the problem they are supposed to solve.

CM: Beyond sports, what other genre will benefit and have a business ROI?
DeFILIPPIS:Cinema, perhaps documentaries.

CM:Will broadcasters be “forced” into 4K/UHDTV adoption similar to the evolution of HDTV? If so, by when?
DeFILIPPIS: Different situation today. Back then, with government and private cooperation, the spectrum sell-off traded the mandatory digital tuners. This time around, government is taking spectrum and providing a one-time payout or compensation for re-location to the under-500 MHz ghetto. No mandate to update the digital tuners nor upgrade of the broadcast technology.

CM:What are the key dominos in the chain that need to be knocked over for the consumers to feel this is real and start spending? Is it availability of “great titles,” consumer devices, branded premium channels, marketing or something else not yet concocted?
DeFILIPPIS: Well the key growth driver is mobile/OTT; so how will 4K play in these devices? When a Google Nexus supports native 4K decoding, when Apple promotes their “8K” screens and iTunes UHDTV offerings, then UHD will take off.

CM:The young adults of today are notorious for consuming social content on small, portable devices. They are the future money for this. Why do they need it?
DeFILIPPIS: Cause as they get older, need to have larger screens so they can read their tweets.

CM:How do we as an industry start our advancement to UHD and HDR?
DeFILIPPIS: Run around, start many organizations that advocate people go out and spend more for larger TV’s.

CM:What is the immediate opportunity for a TV group with regard to these technologies?
DeFILIPPIS: Let the other guy go first. 

CM:Are we now headed toward 1080p60 with HDR and wide color as a first step, with 4K used in production and on-set display etc.?
DeFILIPPIS: Maybe.

CM:What are your thoughts on HDR HD facilities versus native 4K HDR?
DeFILIPPIS: Well while HDR HD is more affordable, so much of the industry is in shell shock and can’t rationalize spending any capex. So if a channel has the opportunity to upgrade, might as well go for 4K.

CM:Is the notion of HDR for HD now officially dead?
DeFILIPPIS: Stillborn perhaps?

CM:Given the file sizes, do you think we should be significantly compressing in acquisition and post, just as we did in the early stages of HD?
DeFILIPPIS: All aspects of 4k makes the workflow more of a problem. However if there can be a universal file format for UHDTV, this alone would speed adoption.

CM:HDCam, while not perfect, was a pragmatic way of getting to HD and getting material out to audiences. What’s the equivalent for UHD?
DeFILIPPIS: Good news is that in the time from HDCam (130Mbps) we are now able to support 250Mbps in production. Given that video compression is not linear—4K does not take four times the bitrate of HDTV—the production bitrate maybe close to the 250Mbps range.

CM:4K, 8K, UHD, and HDR require a very large network topology to support file transport. How do we justify that investment when the delivery to the home will not support the transport without heavy compression for years to come?
DeFILIPPIS: A bit of apples and oranges. Yes live UHD requires large amounts of bandwidth and storage. Given that HDTV/MPEG-2 routinely uses 11-18Mbps, UHDTV using HEVC can be delivered without much more (or less) bandwidth. The more difficult problem is agreeing to the distribution modalities.

CM:Will 4K become a standard transmission? 
DeFILIPPIS: Sure. Just numbers on a piece of paper.

CM:What are you views on affordability of transmission methods for 4K, that is, broadcast, broadband and/or satellite?
DeFILIPPIS: 4K is as affordable in it’s time as HD was in the late ’90s. The real issue is the cost for any change to the established infrastructure.

CM:Do any of you on the panel think there is benefit to using HDR and/or 4K to author a better HD product that can be delivered to consumers without significant changes to current infrastructure, instead of racing to provide 4K?
DeFILIPPIS: This is happening today… use of 4K cameras to capture and then downsampled in post to 2K, while preserving the 4K digital negative.

CM:Shooting UHD side-by-side with HD is too costly. How can we reduce the cost of producing sports in UHD and HDR? 
DeFILIPPIS: Ultimately, side-by-side is not practical. So we have to develop the ability to interoperate between UHD, HDR and HDTV.

CM:How can we do live production of HDR sports without needing separate “shaders” for an HDR/wide color gamut output and an SDR/normal color gamut output for legacy TVs?
DeFILIPPIS: We need to develop tools and monitoring techniques to allow video ops to “shade” for both HDR and SDR.

CM:Do you think that high frame rate is suitable for all genres?
DeFILIPPIS: Yes, in the context that HFR capture can be used for any frame rate (24, 25, 30…). However, artistically we are accustomed to seeing some content at low frame rate and other content at high frame rate.

CM:Given the propensity of advertisers for brighter brights and whiter whites, how do we avoid the CALM Act for brightness?
DeFILIPPIS: Have Congress work on more important matters such as the budget, hunger, terrorism, etc…

CM:How do we integrate interstitials into programming without brightness and color wars?
DeFILIPPIS: Ultimately, this is a self-correcting problem. If commercials are obnoxious, people will tune out, or cut cords or go read a book.

CM:How well can we produce for both HD and HDR without serious compromises, particularly in graphics or saturated colors?
DeFILIPPIS: By it’s nature, HDR is a different pallet than SDR. One could “paint” within the Rec. 709 lines, and HDR will convey these colors but no real benefit. Or paint outside the Rec. 709 lines and try to downsample to SDR, with mixed results.

CM:Given the constraints of the television, cable and satellite system, how comparable will the broadcast experience be to the ultra Blu-ray?
DeFILIPPIS: Apples to oranges. Ultra Blu-ray is not limited by bitrate nor have to support both HDR and SDR.

CM:Given the wide disparity of displays and capabilities on the market, how well can we author once and use everywhere, and how much will the user experience vary?
DeFILIPPIS: Hey, just like NTSC, Never the same color!

CM:As HDR displays get better and do a nice job of presenting SDR-graded content better than it would look on a legacy UHD display, does this challenge the value of specific HDR grading? What percentage of average consumers will appreciate the difference?”
DeFILIPPIS: Unless some one has a side-by-side set of UHDTVs, one showing the SDR upsampled and the other HDR graded, who knows? Try watching a movie on DVD and then Blu-ray HD. Not exactly the same.

CM: When will lower-end 4K cameras offer servo lenses?
DeFILIPPIS: Not sure what lower-end 4K cameras exist… but the key issue about 4K cameras is the optics not the cost or electronics.

Also see…
April 16, 2016
“Q&A: Matthew Goldman on 4K, UHD, HDR and More”
“Is it worth the investment to build a native 2160p infrastructure for the gain in user experience? This is what the broadcaster needs to grapple. This question is very different between a Hollywood studio production and live TV production or distribution.”

April 11, 2016
“Q&A: Mark Schubin on 4K, UHD, HDR and More”
“In my opinion, HDR and HFR offer the most bang for the buck. HDR might be easier to implement, but it can increase motion artifacts, bringing us back to HFR.”

LOS ANGELES—This month, I thought we’d tackle one of the less controversial subjects in the media business today, high dynamic range. Who could argue from a creative perspective or from the viewer’s perspective, the ability to capture and delivery stunning video imagery with a dynamic range far in excess of the current HDTV system.

Sorry, the world is never that simple. While there is agreement that HDR is one of the key components that should be included in UHD TV, the details of what HDR consists of, what the specifications will be or which encoding and compression methods will be standardized, has not been settled. In an attempt to help frame this situation, I’ll start at the beginning….

CRT and GAMMA
In the beginning of television, the one and only display technology was the cathode ray tube. An incredible technology for its time, it did have certain limits, one of which was the peak brightness possible, limited by the maximum spot size (resolution) as well as the power supply. CRTs have a characteristic electronic to optical transfer function (EOTC) implicit in the physics of the CRT, which we know as gamma(γ). The gamma transfer curve has been documented officially in ITU Rec. BT 1886:

The dynamic range of this curve is typically from 0.1 cd/m2 to 100 cd/m2 , which can represent three orders of magnitude or about 10 f-stops. Those of us who have had to shade cameras know that there are limits to where to set iris (exposure), pedestal (minimum black) and the knee point (peak white). Beyond the limits of minimum black and peak white, the video signal clips and all detail is lost.

The CRT limitations constrained the design of the original analog TV systems (NTSC, PAL and SECAM) as well as early digital video systems (ITU Rec. BT 601) and HDTV systems (ITU Rec. BT 709). These systems implicitly limited the range of display luminance values to match the CRT capabilities and using the inverse gamma (0.45) to convert the linear light values from the electronic sensor (CCD/CMOS or tube) to electronic values (voltage or digital code numbers).

With the development of flat panel technologies (LCD, plasma, OLED) along with improvements in display brightness including LED backlighting, display performance is no longer a limitation to reproduce the brightest scene elements while still handling the dark shadow details providing a dramatic increase in overall picture contrast ratio.

HDR
High dynamic range tonal reproduction goes beyond the brightness of standard dynamic range, both in the toe (minimum black) and shoulder (peak white/highlights). SMPTE has standardized a new transfer curve defining the conversation to/from linear light values and video levels (code values) in ST 2084. Known as PQ (perceptual quantizer), this curve is based on the perceptual quality of the human visual system and was defined so as to minimize detectable errors (banding and contouring) over a large range of light values from .0005 cd/m2 to 10,000 cd/m2.This range covers most of the range of human vision as well as natural scene lighting and represents eight orders of magnitude or 28 f-stops! PQ preserves this extreme range of light values while fitting them within of the 10- or 12-bit values of digital video. Below is a comparison of different electrical-to-optical transfer function (EOTF) curves:

However PQ is not strictly a display transfer curve. For each display, there has to be a conversion from the PQ values back to linear light and then another conversion from linear light to display light values. This is sometimes referred to the electronic-to-electronic transfer function (EETF).

There are two meta-data items that are needed by the display to correctly convert the PQ values to display light values, MaxCLL (peak white) and MaxFALL (frame averaged light level). These two metadata items anchor the scene tonal range so that the mid-tones are reproduced correctly and the overall scene light balance is maintained.

So far so good.But what about displays fixed to the old gamma curve? And will a program produced for HDR look right if converted to the more limited CRT display tonal range?

The BBC and NHK have proposed a compromise curve between PQ and gamma, which uses the lower part of the gamma curve and then mapped to a logarithmic curve to extend the upper range of peak white. Called HLG (hybrid log gamma), the definition of the curve can be found in the ARIB B67 standard, “Essential Parameter Values for Extended Image Dynamic Range Television.” While Rec. 709 is anchored in a definition of the peak white value (100 nits) and PQ is a mapping of absolute light values, HLG defines the code value of 0.5 to represent a reference white level (relative to the SDR code value of 1 = 100 nits). Typically the range of HLG encoding is from .001 nits to 1000 nits for a dynamic range of 20 f-stops.

Similar to Rec. BT 709 (SDR), HLG is a “relative light” transfer curve, so the display device tonal range characteristic has to be known at the time of mastering. While this works for a HDR display, what about SDR displays?How will HLG look on these legacy (and some of the first generation 4k displays)? Tests by the BBC and NHK report that HLG, when displayed on a SDR monitor, produce reasonable results. Can HLG might be a good candidate-mastering curve for live television production as a common format for both HDR and SDR?We’ll see….

So far, we’ve gone over the concepts in terms of capture and display of HDR, but what about delivery? How can HDR programs be delivered to the viewer?s it possible to simultaneously deliver a signal that can be faithfully reproduced on any display, HDR or SDR? What about all the different types of content distribution such as Blu Ray, OTT, VOD, cable, satellite and of course over the air?I’ll address HDR distribution and delivery in my next article. Stay tuned.

PACIFIC PALISADES, CALIF.—For this month’s blog, I thought we’d go over mezzanine video compression. It’s become a hot topic again, especially in the area of moving real time video over IP networks.

First, some background on mezzanine compression.

Mezzanine compression is defined as a level of video compression that reduces the video bitrate (bandwidth) so it can fit into professional videotape or storage devices while not being too aggressive, thus retaining the overall quality of the video. Typically, mezzanine compression ratios are from 2:1 to 8:1. Various digital video tape formats used mezzanine compression: D5-HD (4:1), HD CAM (2:1), DVC Pro (5:1), HD CAM SR (3:1), DigiBetacam (8:1).

Key parameters for mezzanine compression are:

• Support for 10-bit depth (or higher)
• 4:2:0/4:2:2/4:4:4 chroma resolutions
• Intraframe encoding
• Multiple pass with minimal degradations
• HD and UHDTV format support

Today, the focus is on compressing live video signals being transported and distributed over IP networks. As has been discussed, the current transmission bandwidth of IP interconnects is 10 Gbps, which can carry up to three uncompressed HD signals, but is not enough to carry even one UHDTV signal. There are multiple implementations for mezzanine compression over IP:

• JPEG 2000 from Evertz
• TICO from IntoPix
• LLVC from Sony
• NDI from Newtek

The first three are wavelet-based, while NDI is described as ProRes-like (DCT-based). JPEG 2000 used in the Evertz EXE-VSR routing system is based on the ISO standard and supports multiple resolutions and formats, including HD and 4K.

TICO has been developed by IntoPix and is based on a lightweight implementation of JPEG 2K. Tico supports HD, 4K and 8K video formats and has low latency encode/decode with low complexity for implementation in firmware (e.g, field programmable gate arrays) as well as in software. TICO is in process of SMPTE standardization.

Low Latency Video Compression, or LLVC, by Sony, has been document in SMPTE RDD34. Using wavelet compression and a line-based entropy encoding, LLVC accomplishes low latency with line-based encapsulation.

NDI’s compression is designed to work in software and is the basis for Newtek’s Tricaster production system. NDI’s compression and SDK supports multiple video and audio streams transported over IP networks. NDI is supported on 32- or 64-bit Windows operating systems with Intel i3 Sandy Bridge CPU or better and a minimum of 4 GB of RAM. While not a I-frame-only codec, it provides for transparent delivery even with packet loss. The codec is based on a 8x8 DCT transform and supports 10-bit, 4:2:0,4:2:2 and 4:4:4.

The JPEG Standards Committee is also looking at standardizing mezzanine video compression.

While the criteria above are necessary, the key issue is going to be interoperability. Will multiple mezzanine methods be used and thus require multiple licenses and implementations? Or will one of these methods become the dominate method and become the de facto standard? In addition, will these compression standards be applicable for the storage and NLE applications? It could be very desirable to have one mezzanine compression standard, but is this necessary for adoption?

In one of the first large adoptions of mezzanine compression, ESPN uses it at the core of its new DC2 facility in Bristol, Conn. ESPN uses Evertz JPEG2K encapsulated in MPEG-2 transport, and delivered over the core EXE-VSR router based on 10Gbps switching. With ESPN’s more 4,000 simultaneous sources requiring simultaneous transmission to almost as many outputs, mezzanine compression in a 10Gbps switch fabric is a necessity.

While compression opens up bandwidth, there are also choices in terms of the amount of compression/bandwidth, bit depth (10-bit or higher) and the format of the video (4:2:0, 4:2:2, 4:4:4, HD, UHD-I or II). While HD generally is compressed between 100 and 200 Mbps, UHDTV is compressed at 250 to 500 Mbps. But at these rates, many compressed video signals can fit in a 10 Gbps infrastructure.

What does this mean? Well, mezzanine compression has proven to be workable for storage of video. Compressed video is in use for distribution and final delivery to the consumer. But will mezzanine video compression be acceptable for live video streaming inside a video production?

LOS ANGELES— Happy New Year 2016! Well, I had thought to start the new year with a fresh topic, maybe something non-controversial such as high dynamic range or 8KTV. But as 2015 was wrapping up, new developments in IP transport of live uncompressed (or mezzanine compressed) video have compelled me to continue on with this theme of video-over-IP.

In my last few musings, I compared the real-time transport of video streams using IP standards, such as SMPTE 2022, versus the conventional use of serial digital interface over coax. In conclusion, while SDI is still relevant and has recently been upgraded to 12 Gbps rates, IP transport of video is an upcoming method proposed by many of the major equipment vendors. (See “12 Gb SDI Cost, Cables and Capabilities,” Dec. 9, 2015.)

In late 2015, there were announcements from two organizations promoting video-over-IP. One is called AIMS (Alliance for IP Media Solutions) and the other is called ASPEN (Adaptive Sample Picture Encapsulation). AIMS is supported by Imagine Communications, Grass Valley (Belden), Cisco, Arista Networks, Snell Advanced Media, EVS Broadcast among others. ASPEN is led by Evertz with support from For-A, Ross, Abekas, AJA Video Systems, ChryonHego, Hitachi, Sony, Tektronix, VizRT among others.

While both organizations support the idea that the future of video transport will use IP-like protocols for switching and delivery, they differ in how they manage the important requirement of video synchronization. AIMS builds on SMPTE 2022 (which, if you recall, came from the Video Services Forum) as a method to use standard IP packet distribution for live video. The core timing is derived from the use of IEEE 1588 Precision Time Protocol to generate RTP with RTMP timestamps. While SMPTE 2022-6 specifies the packetization of the whole SDI signal into packets, there are ongoing efforts in SMPTE to standardize the individual packetization of video, audio and associated data (VSF has standardized this approach in TR-04).

ASPEN builds on a proprietary method developed by Evertz, based on MPEG-2 Systems transport over IP. Currently documented via RDD 37 (Registered Disclosure Document) as well as SMPTE ST 302 and 2038. MPEG-2 Systems (ISO 13818-1) synchronize video and audio streams via an embedded system clock (27 MHz) using a 90 kHz counter as part of an adaptation header.

Both proposed standards also include mezzanine compression to reduce internal bandwidth. AIMS uses TICO, a wavelet-based intraframe compression method (undergoing SMPTE standardization). ASPEN uses JPEG 2000, an ISO standard (15444-1). JPEG 2000 has a range of compression ratios from “mathematically lossless” 2:1, to a more aggressive visually lossless compression of 4:1 or higher.

What are the important considerations? First, SDI is still a valid approach, one which is standardized, well understood and is routinely used for video production. The second consideration is that each of the video-over-IP approaches uses different methods to encapsulate video, audio and data and to provide proper synchronization. Finally, video-over-IP methods have been demonstrated as workable and have been adopted by several notable video production facilities.

Will these two approaches be interoperable? Lets hope so, if only via the brute force method of converting to SDI and then re-packetizing. Clever solutions should evolve that can enable the interchange between AIMS and ASPEN encapsulated video and audio signals.

There is also uncertainty regarding the use of commercial off-the-shelf IP switches for transport of live video and audio signals. Which IP switches will support which protocol? And there are other considerations regarding IP distribution of video and audio streams such as routing optimization algorithms, directory services (what’s in which packet) and latency/bandwidth management.

So our wish for 2016? Let’s hope that all the promises of AIMS and ASPEN come to fruition.

Jim DeFilippis is CEO of TMS Consulting, Inc., in Los Angeles. See more of his contributions in the Author’s Archive.

Corridenda: Steve Reynolds, ​chief technology officer of Imagine Communications, has pointed out that AIMS is focused on un-compressed video and audio transport and routing, and while they have may eventually support compressed video on their roadmap, they have not selected TICO or any other mezzanine video compression method. Further the transport of audio is standardized using AES 67 with the use of SMPTE 2059 (IEEE 1588 PTP) as the common synchronization method. The primary documentation for AIMS encapsulation methodology is found in VSF TR-03 (component RFC 4175) and TR-04 (SMPTE 2022-6).

Mo Goyal, director of product marketing for Evertz, speaking on behalf of ASPEN, would like to clarify that ASPEN defines the mapping of uncompressed UHD/3G/HD/SD over MPEG-2 Transport Stream (ISO/IEC 13818-2) while SMPTE 2022-2 provides the mapping of TS to IP. Further, at this time ASPEN has not documented nor selected JPEG 2000 or any other mezzanine compression technology at this time for use of transport video over IP. ~ Jim D.

LOS ANGELES—As a follow-up to last month’s double dose of SDI vs. IP (and all the follow-up comments; thank you kind readers), I thought I’d focus a bit more on the future of live video signal distribution.

In my last post, I stuck to currently deployed technologies, 3G SDI and 10 Gb Ethernet. I compared and contrasted the use of these transmission and distribution technologies for live video and audio production. The key take away was that while it is possible to use 10 Gb IP for live video, the design of such IP gear and systems is not as straightforward as designing a facility with SDI. SDI being the older, more mature technology, has the advantage of simplicity and ease of use, including test and debugging, although restricted to just carriage of serial digital video with embedded audio.

Last month, my belief was that 12 Gb SDI (SMPTE 2082) was still nascent in the market. Surely, I thought with 10 Gb Ethernet now and 25 Gb standards to be delivered in 2016, the future may well be an all-IP world. At the SMPTE Annual Technical Conference in late October, I walked around the exhibits and found 12 Gb SDI product on the stands, working and very much available. Talking to the primary suppliers of SDI chip sets, Semtech (Gennum) and Macom, I learned that they are sourcing the critical chipsets for SMPTE ST 2082—which became a standard earlier this year—and have been working with their customers to integrate into the products, both at the SMPTE show as well as products to come on the market in the next year.

So let’s look at what 12 Gb SDI can do today. 12 Gb SDI uses regular 75 ohm coax cable—e.g., Belden 1694—and fiber. Distance on coax is the usual 100 meters or about 300 feet, while single-mode optical fiber is rated in kilometers. Further, ST 2082 is flexible in how it can be used, given that it multiplexes eight 1.5 Gbps SDI links across the coax. Thus any signal in the SDI hierarchy is supported, including 270 Mbps, 1.5, 6, and 12 Gbps. Keep in mind that 12Gbps supports video signals up to UHD-1 (2160x3840, 10-bit, 4:2:2) at 60 fps.

I asked both Semetch and Macom about the cost of 12 Gb SDI versus 3G SDI. The chipset cost is 1.5 to two times the cost of 3Gbps chips, partly due to low volume but also because the 12 Gb chipset combines both the receive and equalization modules. I inquired about system costs for 12 Gb SDI products from vendors such as Ross and Black Magic Design. They reported that while the cost is higher for 12 Gb SDI because of the increase of internal bandwidths and processing, greater functionality is available for multiviewers, MADI audio routing, audio embedding/de-embedding, as well as grouping of multiple SDIs into one 12 Gb SDI output. The resulting total cost of 12 Gb products compared to standard 1.5 Gb/3Gb products using outboard gear to accomplish these functions is the same or less.

Oh, and if you need a SMPTE ST 2022 IP connection, sure enough, there are SFP ports that can convert from SDI to IP.

Where’s 25 Gb Ethernet? Still in committee; standardization early to mid-2016. The physical layer consists of twin-axial cables with two lengths: 3 and 5 meters or 100 meters on multimode fiber-optic. The target design is for data center intra- and inter-rack wiring to reduce the number of top-of-the-rack switches. Cost to be determined.

So to summarize, 12 Gb SDI is ready today, costs in line with other SDI solutions, can carry multiple SDI signals, as well as support UHDTV 1 signals (up to 60 fps); 25 Gb Ethernet—to be determined.

Oh, one thing you can’t do with 12 Gb SDI is update your Facebook, Tweet or post a selfie on Instagram.