Integrating high-performance storage
solutions into rich-media, file-based workflows bring a range of topics to the
surface, one of which is why and how to select a storage system.
With the proliferation of disk-storage systems throughout the marketplace,
prescribing the correct storage platform is becoming more important than ever
For small-scale systems, selecting a storage solution may be relatively
straightforward. However, for systems that are expected to grow significantly
and require many different forms of clients and workflows; and for systems that
bridge into other platforms, the configuration of a proper storage solution
with few limitations becomes much broader.
We are beginning to see the impacts caused by improperly sizing the performance
requirements as they relate to both the storage and the server subsystems. Poor
storage-system performance will slow workflows significantly. While bottlenecks
may not show up initially as users become accustomed to the advantages of
file-based workflows, deficiencies in sizing and scaling of the storage system
will certainly lead to reduced overall system performance.
An obvious issue in storage-performance deficiency might at first seem to be
insufficient storage capacity. However, there are other hidden issues that
become performance-killers once a large-scale storage solution is required.
Small-scale, dedicated storage platforms are usually tailored for the specific
components and workflows that are initially attached to the storage system.
Some systems will combine editing interfaces, simplified media asset management
and storage into a single “package.” These solutions are
often groomed so that the subcomponents are matched for specific throughput,
access and collaboration. Such systems are usually optimized mainly for the
initial requirements of a specifically defined workflow. Often these systems
are configured into a compact, economical system that meets the objectives of
the users in their current working environment.
When a system is configured for only a few editing workstations, optimal
storage performance parameters are relatively easy to achieve. Smaller systems
generally do not require a significant number of spindles (hard drives) to achieve
what are usually modest bandwidth requirements aimed at general editing or
When a facility’s needs reach enterprise levels, the storage and
server systems might now handle dozens of craft editors, graphics systems,
closed-caption processing, audio sweetening, external MOS-interfaces and more.
These may also employ multiple ingest, play-out servers and transcoders. Once a
system approaches this size and scale, the storage system architecture becomes
a critical factor in defining end-to-end performance.
In larger, “enterprise class” systems, those
with a high degree of media asset management (MAM) services, the requirements
to handle and process significantly more activities may be orders of magnitude
greater than when just 5 or 10 editing stations are attached to a small storage
area network (SAN) or network-attached storage (NAS).
Many of these activities may be in the background, consuming system bandwidth
even though they may not directly impact storage access or capacity at a
significant scale. As a facility’s overall system requirements
increase, some dedicated “purpose-built” storage systems
(those comprised of spinning disks in a straight RAID configuration) may become
incapable of managing what becomes an ever-expanding need for the migration of
data (content) between various subsystems and workflows.
For systems incorporating a MAM, an enormous number of content, data and
metadata files will be continuously generated. Here, the system data and
metadata files are typically handled by IT-centric server hardware sized to
manage the various processes that occur throughout the various workflows.
1: A modest scale “enterprise class,” digital media
file-based production system showing the common components, servers, clients
and storage systems. Inserted text boxes describe some of the typical data
rates and stream counts for a system of this size. The total bit rate
(including some additional items not shown on this diagram) for a system of
this dimension might easily approach 2.3 Gbps of system
bandwidth—plus additional services for near line NAS or DLC gateways
in the 140–150 Mbps range.
Additional IT-servers can be deployed to manage the content (raw files, proxies
and renderings) that shuffle between devices (encoders, decoders, transcoders)
and storage systems (SAN, NAS or local direct-attached storage (DAS) caches on
In larger systems involving numerous workflows occurring simultaneously, all of
the various services involving file exchanges must be thoroughly accounted for.
Examples (see Fig. 1) include ingest operations with proxy generation and
transcode functions; migration of files from local storage to central storage
and archive; edit-in-place and/or file exchanges between central storage and
editing workstations; layering and rendering; and video servers in support of
QC, baseband play-out, video-on-demand or streaming.
Fundamental to specifying overall system performance, especially as it relates
to the bandwidth of the storage system and the network interfaces, is the task
of accurately predicting the total number of data reads and writes expected
from the system.
System sizing involves not only storage capacity, accessibility or
bandwidth—but now includes the network storage interfaces,
accelerators and the capability to correct for ambiguous data, which can occur
during writes to the disk drive itself.
The overall system design needs to manage and mitigate all potential limitations
in performance. Design parameters must include the number of spindles
supporting the storage solution; the number of read and write cycles per group
of disks; the type of interface to the storage (e.g., Fibre Channel vs. GbitE
vs. InfiniBand); file sizes and formats; and the number of input/output
operations per second (IOPS).
As a benchmark point, when the aggregate read+write bandwidth reaches
1–1.5 GBps, conventional commodity storage components probably become
insufficient—and an enterprise-class, media-centric storage solution
needs to be considered.
Having the right parameters increases the ability to share access to all the
content, both at high resolution and at proxy levels, and at all times.
Balancing all these demands among all the clients and users, whether attached
directly as “thick-client” workstations or when in a
Web-based interface, is essential to maintaining system stability, achieving
high performance and providing sufficient availability.
Karl Paulsen (CPBE) is a SMPTE Fellow and chief technologist at
Diversified Systems. Read more about other storage topics in his latest book
“Moving Media Storage Technologies.” Contact Karl at