INDIAN WELLS, CALIF. – What’s the problem, Jim DeFilippis rhetorically asks Ando Setos. “There are plenty of shows coming out of Hollywood these days.”
Setos, sitting in front of the podium where DeFilippis is standing, misses his line, which is, “today, we are managing, but look at the trend. We are heading for a ‘data tsunami.’”
But Setos is going off script. There’s a storage challenge, he says, and let’s not oversimplify.
“We say storage, but it’s the speed of getting it from and to the storage,” he says. “ It’s about capacity, but it’s about terabytes and megabits. And the archiving of data, which we’ll get into later.”
Setos and DeFilippis, former colleagues at Fox, brought the 411 on file-based workflows to the Hollywood Post Alliance Tech Retreat in the desert. The three big issues pertaining to file-based workflows, according to Andy and JimD—Storage, archiving and file formats.
STORAGE, OY
What do you do with all those bits? An HD show can generate as much as 30 hours of media at 200 Mbps for 2.7 terabytes of data. Double that for 3D, and quadruple it for 4K, for which a quarter of a petabyte would be needed for storage. A DCI-formatted movie can generate as much as 1 to 2 PB of media per title. 3D more than doubles the necessary storage capacity. Add a high-frame rate and double or even triple that.
Distribution can then require more layers of storage based on what format each new online distributor requires, usually at 1 to 4 Mbps. For a 1,000-hour library, this can be as much as 0.5 to 2 PB for each distribution deal, DeFilippis said
And then he showed a slide entitled, “FLASH: Terror-Bites of Media Files Threaten Hollywood.” It showed an escalation in storage reaching nearly 2 million exabytes for TV shows and movies by 2016.
Where will it go, the cloud? In order to store files on the cloud, they have to be moved across the Internet, Setos said.
“Internet managers talk about ‘large objects,’ he said. “They cost a fortune and create huge problems on the Internet. And they’re called ‘tv shows,’ and take four or five GB. We’re dealing with another type of object, ‘humongous, incredibly big objects.’”
“The cloud assumption,” he said, “is incredibly dangerous.”
As for physical storage media, DeFilippis said the industry couldn’t wait for “the mythical grain of salt that going to store the Library of Congress.”
Each physical storage format has its benefits and drawbacks. Solid state memory, used mostly in acquisition, is reliable, fast and power-efficient, but it’s costly relative to, an not suitable for, long-term storage because of “alpha particles,” DeFilippis said.
Hard disk drives, used mostly in post production and playout systems, reliable, reasonably affordable and handle large bandwidth inputs, but they’re fragile and power hungry.
Optical storage is low cost and can last up to 20 years, but has slow read/write access. Data tape has both cost and read/write benefits, but needs to be “incorporated into a library with a migration strategy,” DeFilippis said.
WHAT ABOUT ARCHIVING?
DeFilippis and Setos described two approaches: Migration versus fundamental archive storage, or “bits vs. atoms.” The benefit of digital file storage migration is continuous verification of content, available for all to use. The final version and components are there, Setos said. The downside is that it’s an ongoing process that needs prior planning. Fundamentally archival media is simple, durable, long-lasting; store-and-forget. The downside is not having a reading device in the future.
“Because we’re dealing with files now and not things that are consumable by a human, like fixed art and looking at images on Mylar… we’re going to need complex devices to read anything that comes up in the future,” he said. “The problem is not the media, it’s ‘will I have the machine.’ Think of trying to build a quadraplex recorder today.”
“Digital file storage is bits: Data tape, tape drives or archive,” he said. “Data tape is just tape. A drive is just a drive. We’re talking about systems.”
FILE-FORMAT ANARCHY
Why so many file formats, DeFilippis asked, again, rhetorically, though it’s a ubiquitous musing in the TV industry. There are different format requirements for each stage of production, competitive issues between manufacturers, and diverse formats for distribution platforms.
“I hear people talk about transcoding like a fix-all Band-Aid, but you lose picture quality,” Setos said. “You have to be careful about how you use transcoding…. There are no simple solutions. We work in possible the most complex audio/visual environment ever.”
Will there ever be one common file format? JimD, said, “no. The goal should be to minimize file types, especially the number of codecs.”
~ Deborah D. McAdams
