Editor’s note: The following is an excerpt from a recent white paper “Comparison of the Media Archiving Options and Costs for M&E,” commissioned by Quantum.
ALEXANDRIA, VA.—Television stations, video producers and post houses are awash in content data, and with each passing year the demand for storage only increases. A mid-market TV station producing two hours of news each weekday will create 15 TB of data per year, which means even a small station group with six or seven stations will churn out 100 TB of content to store on a yearly basis.
That figure is significant because after only a few years this hypothetical small station group will reach the 500 TB point, a threshold—give or take several terabytes—at which time it begins to make financial sense to store this content on a lower-cost medium and free up higher-performance, more costly storage resources for more productive uses.
“Now that everyone has woken up to the fact that content is king and has long-term value, they are beginning to understand that it also has a significant cost to store,” said Steve Davis, an Atlanta- based media storage consultant who formerly was senior vice president and CTO of Crawford Communications. “It grows and grows and grows. And the bit rates are high.”
When media enterprises reach their threshold, they have three options for long-term media archiving: on-premise digital magnetic tape-based archival storage in a Linear Tape-Open (LTO) library; the cloud–either public or private; or some hybrid combination of the two.
Before choosing, these enterprises must consider four important factors: cost, performance, accessibility and security. Additionally, in some situations the operational model of a business is a fifth factor that must be part of any evaluation. Specifically, how a business operates will determine whether it is better served by making an upfront capital investment in storage technology or by choosing a recurring monthly operational expense to archive media assets.
IN THE CLOUD
Public cloud storage services, such as those offered by Amazon, Microsoft and Google, have captured the attention of media and entertainment enterprises for archiving content, largely because of pricing. However, determining the true cost of cloud archiving can be tricky. First, an M&E enterprise must decide whether it needs fast access to data stored in the cloud or it can wait a longer time to retrieve its content.
Amazon Web Services illustrates the point. Its low-cost Glacier storage ($0.004 per gigabyte per month as of this writing), which is intended for archiving applications, typically can take three to five hours for a standard retrieval. Its more expensive S3-IA (Simple Storage Service-Infrequent Access) service ($0.0125 per gigabyte per month as of this writing) promises low latency and high throughput.
Other costs, such as the per gigabyte price of data retrieved from storage, what it cost to make retrieval requests and the cost of bandwidth to connect an enterprise’s site with the cloud, must be factored in.
When it comes to security, cloud service providers contend it is impossible for a media enterprise to replicate the level of data protection they offer. They argue no media business will outspend them on the steps they take to keep data secure, and they have the certifications to prove it.
Public cloud services also can provide geo-spacing which protects archived data against the ravages of localized natural disasters, such as tornadoes, hurricanes, floods and earthquakes, by distributing data across multiple geographies.
ON-PREMISE STORAGE
M&E enterprises considering LTO-based digital storage for on-premise archiving will face an unavoidable capital expense to put a system in place. They will need an LTO robotic loader, tape drives and a tape management system.
Other associated expenses include the costs of LTO tape, the square footage needed for the system and personnel to attend to it, said Jim Casabella, president of The Jim Casabella Consulting Group in Memphis, Tenn. Enterprises should also plan for the expense of migrating to newer generations of LTO storage with time, incurring new capital costs in the process, he said. However, it should be noted that a provision of the Tax Cuts and Jobs Act of 2017 allows certain businesses to expense 100 percent of the cost of most business equipment in the first year.
Still, when measured on a per gigabyte basis, the cost of storing archival data on LTO tape maintains its advantage against other alternatives, said Casabella, who specializes in consulting with media organizations about storage archive solutions and asset management.
In terms of performance, retrieving data stored on-premise from an LTO-based library typically takes seconds or at most a few minutes. The latest generation of LTO storage technology, LTO 8, offers a data transfer rate of 360 MBps native and 750 MBps compressed.
When it comes to protecting data from entropic damage, monitoring and managing data stored on tape must be considered. Technology exists to scan LTO-stored data to validate its integrity and takes the necessary steps to protect it by automatically making a new copy of the data on a fresh tape.
Beyond this level of data protection, there is the question of maintaining the physical security of LTO tapes from theft or natural disasters. Multiple copies of tapes can be stored at different locations to protect against calamities. However, doing so also multiplies the steps needed to safeguard against theft and piracy, all of which can be planned for and professionally executed.
A HYBRID ARCHIVE
It’s common for mid-size and large M&E organizations to have multiple workflows, each with its own storage requirements. As a result, many organizations have chosen to leverage the value of a hybrid archive solution based both on cloud and on-premise storage resources. A hybrid strategy also can offer M&E enterprises the benefit of taking advantage of the best of both archiving alternatives.
Security is a good example. “You can write a copy to AWS and two to tape,” explained Nils Carson, system architect at Quantum, a provider of LTO-bases archiving solutions. “Take one copy out and send it to Iron Mountain or some other repository with the same physical security. That way you have ultimate control and can pull back any one of those copies at any time.”
Similarly, organizations can mold a hybrid approach that leverages the strengths of both the cloud and on-premise archiving when it comes to performance, accessibility and cost.
Cloud bursting, a technique to scale up processing power by leveraging more CPUs and GPUs as needed in the cloud to accomplish a compute-intensive task, offers a good use case that illustrates how M&E enterprises can take advantage of the best of both storage options.
For instance, a hypothetical media enterprise that wishes to use cloud-based artificial intelligence tools, such as speech-to-text functionality and facial recognition, to generate useful metadata describing archived video assets, can take advantage of processing power in the cloud and continue to use on-premise storage to achieve its specific cost, accessibility and security goals.
In this hybrid application, a media enterprise can temporarily push a portion of its archive—most likely lower-resolution proxy files—to the cloud where AI processing can create additional metadata. The metadata can then be retrieved and married to the on-premise archival copy, while the version of the metadata in the cloud as well as the proxy files can be discarded. The process can be repeated as budget allows until the entire LTO-based library has been updated with enhanced metadata, explained Casabella.
MAKING THE RIGHT CHOICE
While the pros and cons of each archiving option are sure to change as pricing models fluctuate, networking technologies evolve and storage media advances, enterprises without an archive strategy in place are at risk of spiraling cost structures, a loss of productivity and missing out on business opportunities.
However, with a solid archiving strategy in place, M&E enterprises can free up costly storage for more productive, revenue-generating uses while ensuring their valuable archives of media content remain safe and accessible for the long term.
