INDIAN WELLS, CALIF. – What’s the problem, Jim DeFilippis rhetorically asks
Ando Setos. “There are plenty of shows coming out of Hollywood these days.”
Setos, sitting in front of the podium where DeFilippis is standing, misses his
line, which is, “today, we are managing, but look at the trend. We are heading
for a ‘data tsunami.’”
But Setos is going off script. There’s a storage challenge, he says, and let’s
not oversimplify.
“We say storage, but it’s the speed of getting it from and to the storage,” he
says. “ It’s about capacity, but it’s about terabytes and megabits. And the
archiving of data, which we’ll get into later.”
Setos and DeFilippis, former colleagues at Fox, brought the 411 on file-based
workflows to the Hollywood Post Alliance Tech Retreat in the desert. The three
big issues pertaining to file-based workflows, according to Andy and
JimD—Storage, archiving and file formats.
STORAGE, OY
What do you do with all those bits? An HD show can generate as much as 30
hours of media at 200 Mbps for 2.7 terabytes of data. Double that for 3D, and
quadruple it for 4K, for which a quarter of a petabyte would be needed for storage.
A DCI-formatted movie can generate as much as 1 to 2 PB of media per title. 3D
more than doubles the necessary storage capacity. Add a high-frame rate and
double or even triple that.
Distribution can then require more layers of storage based on what format each
new online distributor requires, usually at 1 to 4 Mbps. For a 1,000-hour
library, this can be as much as 0.5 to 2 PB for each distribution deal,
DeFilippis said
And then he showed a slide entitled, “FLASH: Terror-Bites of Media Files
Threaten Hollywood.” It showed an escalation in storage reaching nearly 2
million exabytes for TV shows and movies by 2016.
Where will it go, the cloud? In order to store files on the cloud, they have to
be moved across the Internet, Setos said.
“Internet managers talk about ‘large objects,’ he said. “They cost a fortune
and create huge problems on the Internet. And they’re called ‘tv shows,’ and
take four or five GB. We’re dealing with another type of object, ‘humongous,
incredibly big objects.’”
“The cloud assumption,” he said, “is incredibly dangerous.”
As for physical storage media, DeFilippis said the industry couldn’t wait for
“the mythical grain of salt that going to store the Library of Congress.”
Each physical storage format has its benefits and drawbacks. Solid state
memory, used mostly in acquisition, is reliable, fast and power-efficient, but
it’s costly relative to, an not suitable for, long-term storage because of
“alpha particles,” DeFilippis said.
Hard disk drives, used mostly in post production and playout systems, reliable,
reasonably affordable and handle large bandwidth inputs, but they’re fragile
and power hungry.
Optical storage is low cost and can last up to 20 years, but has slow
read/write access. Data tape has both cost and read/write benefits, but needs
to be “incorporated into a library with a migration strategy,” DeFilippis said.
WHAT ABOUT ARCHIVING?
DeFilippis and Setos described two approaches: Migration versus fundamental
archive storage, or “bits vs. atoms.” The benefit of digital file storage
migration is continuous verification of content, available for all to use. The
final version and components are there, Setos said. The downside is that it’s
an ongoing process that needs prior planning. Fundamentally archival media is
simple, durable, long-lasting; store-and-forget. The downside is not having a
reading device in the future.
“Because we’re dealing with files now and not things that are consumable by a
human, like fixed art and looking at images on Mylar… we’re going to need
complex devices to read anything that comes up in the future,” he said. “The
problem is not the media, it’s ‘will I have the machine.’ Think of trying to
build a quadraplex recorder today.”
“Digital file storage is bits: Data tape, tape drives or archive,” he said.
“Data tape is just tape. A drive is just a drive. We’re talking about systems.”
FILE-FORMAT ANARCHY
Why so many file formats, DeFilippis asked, again, rhetorically, though
it’s a ubiquitous musing in the TV industry. There are different format
requirements for each stage of production, competitive issues between
manufacturers, and diverse formats for distribution platforms.
“I hear people talk about transcoding like a fix-all Band-Aid, but you lose
picture quality,” Setos said. “You have to be careful about how you use
transcoding…. There are no simple solutions. We work in possible the most
complex audio/visual environment ever.”
Will there ever be one common file format? JimD, said, “no. The goal should be
to minimize file types, especially the number of codecs.”
~
Deborah D. McAdams