media is created
digitally, content production
processes must now leverage
distribution at a level of
quality equal to or better
than high definition.
As a consequence, a migration is taking
place across the industry that includes TV
stations, film studios, production houses,
and digital intermediate (DI) facilities.
High-performance intelligent storage
platforms are now helping media organizations
and service providers optimize
workflows and improve performance.
An integral component in the workflow
process, these storage systems provide
the scalability, performance, density, and
power efficiency necessary to store and
manage enormous amounts of unstructured
data, which power the all-digital entertainment
We’ve witnessed extremely rapid growth
in very large media files. Digital images, video,
and audio now account for 85 percent
of the global storage capacity. Most consist
of unstructured data as opposed to the
transaction-based data, which previously
dominated many organizations. This quantum
shift in the types and volumes of data
has changed storage in a dramatic way.
Traditional storage architectures, once
optimized for transactional data, are not
well suited for digital media. Media storage
platforms now handle very large files in real
time. They must be able to expedite workflows
and provide paths to staying ahead of
the exponential growth in files and media.
Systems are called upon to support hundreds
of users, store millions of images, videos,
and music files; and then deliver them
with sub-millisecond response times.
These systems must also stream massive
amounts of high -definition content without
ever missing a single frame.
Storage platforms need to deliver write
performance that equals read performance.
They must provide a guaranteed quality-of-service (QoS) with predictable performance
on large files, while simultaneously
ingesting multiple sets of data streams in
real and faster-than-real time.
Motion picture and digital media content
creation is made possible by the collaboration
of teams that make up a massive mix
of production activities. Their activities include
visual effects, 2D and 3D animation,
full titles, trailer clips, and sound
The releases consist of feature
films, television, advertising, interactive
visual media and video
games. In order to meet the often
stringent deadlines, real-time collaborative
workflow is essential.
No longer can operations tolerate
system downtime, inaccessible
data, or reduced throughput because
of data transfer bottlenecks.
Today’s workflows create content
of increasing complexity that
feed a distributed environment in
a mix of resolutions ranging from mobile/Web resolutions through HD.
Some systems must support the 2K, 4K
and 8K (UHDTV) formats of digital cinema.
As a result, organizations are finding they
are technically challenged when it comes
to supporting a mixture of real-time workflows;
concurrent multi-user editing or
colorists; and growing file sizes that begin
at ingest and expand throughout the post-production process.
Conventional storage systems are found
to be inadequate for this level of post production,
with users turning to intelligent,
high-performance storage solution platforms
for their answers.
Post-production environments have
moved to a central storage system that supports
edit-in-place functionality for dozens
of uncompressed HD workstations; all the
while delivering multiple streams for transmission
and distribution to other services.
In extreme environments, system performance
can require as much as 240 GBps of
scalable storage capacity while supporting
open systems with multiple CPUs, operating
systems, and file systems—all in a collaborative
Broadcast entities also find themselves
preparing content for multiple formats, and
then distributing those multiple streams in
support of consumer demands for on-air services,
video on demand, streaming to mobile
phones and media players, and for delivery
to Web portals for Internet distribution.
These expanding activities are growing
at an alarming rate, requiring another breed
of high-throughput scalable storage systems
that can support real-time collaborative
workflows, and the storage of myriad files of
unstructured images, videos, and music.
This caliber of storage system handles
very large files at a very high rate of throughput.
In some operations the volume of newly
created content easily exceeds multiple
terabytes a day, which is fed by concurrent
digitization on multiple ingest stations.
These next-gen storage platforms concurrently
deliver multiple terabytes of data to
editing stations, playout servers, transcode
and render farms, and content distribution
networks. And since video playout depends
on the flawless streaming of video, these
storage platforms must deliver real-time performance
with uncompromising reliability.
Heretofore, graphics and effects production
had operated in a parallel job-task-related
fashion using silos of individual stores
attached locally to the workstation or a small
NAS, or on a SAN.
By contrast, news editing has traditionally
operated on a sequential basis with storage
on a central SAN or NAS in support of multiple
edit stations. New-era intelligent storage
systems now provide for parallel storage
solutions, which are changing the news and
graphics/effects models through a tightly
coupled centralized repository of managed
files with performance equaling and usually
exceeding their archaic functionality.
Storage platforms now employ storage
accelerators to handle parallel paths of data
between hosts and storage arrays. These accelerators
utilize solid-state caches and coherency
algorithms to optimize read and
By coupling scalable high-performance
storage platforms to storage accelerators,
the throughput and performance requirements
can be better met; and can further
provide for redundancy and failover paths
that allow operations to continue should a
host bus adaptor or switch fail.
Karl Paulsen is a technologist and consultant
to the digital media and entertainment
industry. Contact him at [email protected].