Editor’s note: The following is an excerpt from a recent white paper “Comparison of the Media Archiving Options and Costs for M&E,” commissioned by Quantum. 

ALEXANDRIA, VA.—Television stations, video producers and post houses are awash in content data, and with each passing year the demand for storage only increases. A mid-market TV station producing two hours of news each weekday will create 15 TB of data per year, which means even a small station group with six or seven stations will churn out 100 TB of content to store on a yearly basis.

That figure is significant because after only a few years this hypothetical small station group will reach the 500 TB point, a threshold—give or take several terabytes—at which time it begins to make financial sense to store this content on a lower-cost medium and free up higher-performance, more costly storage resources for more productive uses.

“Now that everyone has woken up to the fact that content is king and has long-term value, they are beginning to understand that it also has a significant cost to store,” said Steve Davis, an Atlanta- based media storage consultant who formerly was senior vice president and CTO of Crawford Communications. “It grows and grows and grows. And the bit rates are high.”

When media enterprises reach their threshold, they have three options for long-term media archiving: on-premise digital magnetic tape-based archival storage in a Linear Tape-Open (LTO) library; the cloud–either public or private; or some hybrid combination of the two.

Before choosing, these enterprises must consider four important factors: cost, performance, accessibility and security. Additionally, in some situations the operational model of a business is a fifth factor that must be part of any evaluation. Specifically, how a business operates will determine whether it is better served by making an upfront capital investment in storage technology or by choosing a recurring monthly operational expense to archive media assets.

IN THE CLOUD

Public cloud storage services, such as those offered by Amazon, Microsoft and Google, have captured the attention of media and entertainment enterprises for archiving content, largely because of pricing. However, determining the true cost of cloud archiving can be tricky. First, an M&E enterprise must decide whether it needs fast access to data stored in the cloud or it can wait a longer time to retrieve its content.

Amazon Web Services illustrates the point. Its low-cost Glacier storage ($0.004 per gigabyte per month as of this writing), which is intended for archiving applications, typically can take three to five hours for a standard retrieval. Its more expensive S3-IA (Simple Storage Service-Infrequent Access) service ($0.0125 per gigabyte per month as of this writing) promises low latency and high throughput.

Other costs, such as the per gigabyte price of data retrieved from storage, what it cost to make retrieval requests and the cost of bandwidth to connect an enterprise’s site with the cloud, must be factored in.

When it comes to security, cloud service providers contend it is impossible for a media enterprise to replicate the level of data protection they offer. They argue no media business will outspend them on the steps they take to keep data secure, and they have the certifications to prove it.

Public cloud services also can provide geo-spacing which protects archived data against the ravages of localized natural disasters, such as tornadoes, hurricanes, floods and earthquakes, by distributing data across multiple geographies.

ON-PREMISE STORAGE

M&E enterprises considering LTO-based digital storage for on-premise archiving will face an unavoidable capital expense to put a system in place. They will need an LTO robotic loader, tape drives and a tape management system.

Other associated expenses include the costs of LTO tape, the square footage needed for the system and personnel to attend to it, said Jim Casabella, president of The Jim Casabella Consulting Group in Memphis, Tenn. Enterprises should also plan for the expense of migrating to newer generations of LTO storage with time, incurring new capital costs in the process, he said. However, it should be noted that a provision of the Tax Cuts and Jobs Act of 2017 allows certain businesses to expense 100 percent of the cost of most business equipment in the first year.

Still, when measured on a per gigabyte basis, the cost of storing archival data on LTO tape maintains its advantage against other alternatives, said Casabella, who specializes in consulting with media organizations about storage archive solutions and asset management.

In terms of performance, retrieving data stored on-premise from an LTO-based library typically takes seconds or at most a few minutes. The latest generation of LTO storage technology, LTO 8, offers a data transfer rate of 360 MBps native and 750 MBps compressed.

When it comes to protecting data from entropic damage, monitoring and managing data stored on tape must be considered. Technology exists to scan LTO-stored data to validate its integrity and takes the necessary steps to protect it by automatically making a new copy of the data on a fresh tape.

Beyond this level of data protection, there is the question of maintaining the physical security of LTO tapes from theft or natural disasters. Multiple copies of tapes can be stored at different locations to protect against calamities. However, doing so also multiplies the steps needed to safeguard against theft and piracy, all of which can be planned for and professionally executed.

A HYBRID ARCHIVE

It’s common for mid-size and large M&E organizations to have multiple workflows, each with its own storage requirements. As a result, many organizations have chosen to leverage the value of a hybrid archive solution based both on cloud and on-premise storage resources. A hybrid strategy also can offer M&E enterprises the benefit of taking advantage of the best of both archiving alternatives.

Security is a good example. “You can write a copy to AWS and two to tape,” explained Nils Carson, system architect at Quantum, a provider of LTO-bases archiving solutions. “Take one copy out and send it to Iron Mountain or some other repository with the same physical security. That way you have ultimate control and can pull back any one of those copies at any time.”

Similarly, organizations can mold a hybrid approach that leverages the strengths of both the cloud and on-premise archiving when it comes to performance, accessibility and cost.

Cloud bursting, a technique to scale up processing power by leveraging more CPUs and GPUs as needed in the cloud to accomplish a compute-intensive task, offers a good use case that illustrates how M&E enterprises can take advantage of the best of both storage options.

For instance, a hypothetical media enterprise that wishes to use cloud-based artificial intelligence tools, such as speech-to-text functionality and facial recognition, to generate useful metadata describing archived video assets, can take advantage of processing power in the cloud and continue to use on-premise storage to achieve its specific cost, accessibility and security goals.

In this hybrid application, a media enterprise can temporarily push a portion of its archive—most likely lower-resolution proxy files—to the cloud where AI processing can create additional metadata. The metadata can then be retrieved and married to the on-premise archival copy, while the version of the metadata in the cloud as well as the proxy files can be discarded. The process can be repeated as budget allows until the entire LTO-based library has been updated with enhanced metadata, explained Casabella.

MAKING THE RIGHT CHOICE

While the pros and cons of each archiving option are sure to change as pricing models fluctuate, networking technologies evolve and storage media advances, enterprises without an archive strategy in place are at risk of spiraling cost structures, a loss of productivity and missing out on business opportunities.

However, with a solid archiving strategy in place, M&E enterprises can free up costly storage for more productive, revenue-generating uses while ensuring their valuable archives of media content remain safe and accessible for the long term.

SAN MATEO, CA.—Broadcast professionals responding to a survey at last month’s NAB Show foresee increasing reliance on AI and machine learning as well as hybrid cloud storage and a falloff on their reliance on tape-based storage in their future.

The survey, based on in-person interviews of more than 300 people at the show, revealed that 78 percent of broadcast professionals plan to use a combination of on-premise and cloud-based storage, also known as hybrid storage, to speed up media management.

Eighty percent plan to use AI and ML technologies to enrich metadata, the survey found. Among users of tape storage, 51 percent indicated they plan to move away from tape media over time.

[Read: The Next Big Step For AI? Understanding Video]

Among post-production professionals interviewed, 73 percent revealed they are frustrated with how desired media is located and retrieved and identified the issue as their primary storage challenge. Fifty percent said media management is more time-consuming today than it was three years ago.

Eight in 10 are thinking about using AI and/or ML technology to assist in tagging media, which indicates healthy interest in storage offering embedded rich metadata tags and metadata-bases search tools, the survey said

The survey also revealed a significant upturn in those who expect to use hybrid storage when compared to three years ago –78 percent versus 16 percent in 2015. Exclusive use of cloud storage appears to be headed lower, however, as 9 percent said they would be cloud-only in three years while 17 percent are today, it found. When it came to disk- versus tape-based storage as a primary storage medium, the former was clearly the preference, with 53 percent favoring disk and 32 percent choosing tape.

Among tape users, 51 percent said they plan to move away from tape in the next three years, the survey said.

Cloudian sells a scalable storage platform that consolidates, manages and protects enterprise data.