A performance measurement commonly used to benchmark hard disk drives, solid state drives, and storage area networks is called Input/Output Operations Per Second, or IOPS. Each type of storage device can have its own particular set of benchmark metrics and in some cases they can vary dramatically depending on the application which they will support and the physical makeup of the device.

It goes without saying that every millisecond spent in a process will impact overall net system performance. In our highly software-centric world, designers strive to achieve the fastest processing while simultaneously minimizing every aspect of latency possible. This can be accomplished through the effective use of software applications and by harmonizing the proper components into a complete package (as a “system”) that meets the expected targeted utilization requirements.

As mentioned in past articles, much of the IT-world centers on high volume transaction processing for data that is typically “structured” data which is organized, consistent and predictable. With more and more “video” media activity, that model is changing and systems are now required to manage different forms of data, otherwise known as “unstructured data.”

TRANSACTIONAL VS. BATCH PROCESSING
Processing media assets is fundamentally almost the reverse of the processing encountered for transactional activities. Transaction processing manages small sets of highly organized data-bits which are divided into individual, indivisible operations. Each “transaction” must succeed or fail as a complete unit; and can never be only partially complete.

Transactional processing, those supporting banking, ATMs, order processing and financial activities, continue to look for increased performance and capacity. Most storage performance benchmarks are therefore centered on transactional processing activities. It’s no wonder that users can now find solutions that achieve, for example, 1 PB (petabyte) of flash storage providing 22 million IOPS with tens of thousands of read/writes—all in a single rack.

In the IT world, batch processing—the opposite of transaction processing—handles requests that are cached (prestored) and then executed all at the same time. Batch processing typically occurs unattended, for example, the reconciliation of all the bank’s ATM transactions for a given day are compared against all the customer’s bank records and are then reported at a time period somewhere outside of normal business hours. Conversely, transaction processing is interactive, requiring a user input such as depositing money into a checking account or ordering airline travel where flight times, travel locations, seat selections, must occur from user responses.

LOTS OF IOPS NEEDED
Depending upon the application, these activities can require a lot of IOPS; which can be throttled or adjusted based upon priorities, loading and overall system demand. However, the system must be designed to handle the peak demands by providing sufficient IOPS to meet the application’s activities. In transactional activities, the emphasis on storage metrics can be very predictable and consistently managed, except for failures or overly high demands system wide.

Handling media files and content requires a different set of processing concepts; even though in certain conditions there are finite elements of transaction-like activities still occurring such as for metadata inputs or reports management.

Generally, users of magnetic disk and solid-state storage systems are seldom concerned with the details of processing speeds and latency for individual storage devices. However, if you’re building a spec server for continual operations where high volumes of transactional or batch activities are necessary, these storage performance metrics become more relevant.

When the system requirements are elevated to a higher order RAID system (e.g., RAID 5, 10, 6, etc.), the aggregate performance influence associated with IOPS, disk rotational speed and latency become key factors. And now with increased solid-state storage integration, there are added factors related to read/ write performance that are considerably different from rotating magnetic disk storage.

FIGURING UP THE IOPS
Disk and solid-state storage devices have a theoretical maximum IOPS value based upon certain key factors. For magnetic spinning disks, these factors are: rotational speed, average latency and average seek time (Fig. 1). The formula for computing IOPS equals 1/(average latency in milliseconds plus the average seek time in milliseconds). Since most enterprise class storage systems employ several drives arranged as arrays, you then multiply the IOPS for a single drive times the number of drives to obtain the total IOPS figure. For example, if a single drive is capable of 180 IOPS, then 10 drives in a JBOD configuration would be 10 x 1,800 = 1,800 IOPS; assuming this is a RAID 0 configuration, which incidentally would be considered unusual in this level of system.

Fig. 1: Disk and solid-state storage devices have a theoretical maximum IOPS value based upon certain key factors. For magnetic spinning disks, these factors are: rotational speed, average latency and average seek time.

When you up the RAID level to, say RAID 5, other factors will then impact the performance figures. When data is striped across drives (typical for levels above RAID 0), each write-action becomes a multiwrite operation which must occur to each drive in the array. This places a significant impact on the raw IOPS figure. For RAID 1 and RAID 10, there will be a two-to-one IOPS penalty. In RAID 5, there are four IOPS per write-action; and in RAID 6 (with double fault tolerance protection), there will be six writes per write-operation.

For read-activities, in configurations less than RAID 5 or 6, there is no performance impact and the array essentially yields a 1:1 ratio for each read operation.

When drive systems are configured, the balance (or ratio) of reads and writes must be considered, along with the RAID penalty associated with the input/output operations (IOPS). For example, if your specific application requires 250 IOPS in order to process a 50 percent read and 50 percent write workload—and you want RAID 6 for resiliency—you’ll need to resolve the array so that a total of 875 IOPS could be delivered (see the calculation example in Fig. 2). One spec that impacts this performance is spindle rotational speed. Typically there are three choices in spindle speeds depending upon the disk drive: 7,200 RPM, 10,000 RPM or 15,000 RPM. Having selected a drive specified with at least 250 IOPS, the sample calculation shows the total IOPS target (875 IOPS) necessary for this specific application. The calculation is based upon the number of spindles (i.e., the quantity of drives at a specific rotational speed), the drive IOPS, and a RAID 6 configuration.

Large storage systems with dozens to multi-hundreds of drives are not unusual for today’s growing storage systems. This is not just for capacity purposes, but also for IOPS. The range for a complete system can in fact be quite broad: anywhere from between 20,000 to multi-million IOPS. However, in Tier 1 storage implementations, where a set of enterprise applications are enabled on both virtual and physical servers, IOPS performance may be well below 100K IOPS. Typically, the numbers will range between 30 to 40K IOPS.

It is easy to see why systems tend to employ higher RPM, enterprise-class SAS drives when looking for peak performance in a minimal footprint.

Fig. 2: When drive systems are configured, the balance (or ratio) of reads and writes must be figured, along with the RAID penalty associated with the input/output operations.

DOES IT REALLY MATTER?
Some storage professionals may claim “IOPS don’t matter”; and they suggest you don’t need to consider the benchmarks and stress test statistics in designing arrays to meet bandwidth. If this statement were entirely believable, then you’d probably not see any performance impediments between the various drive systems provided by most high-performance storage system providers. However, this statement just doesn’t seem to make sense, and when you dig further you’ll probably find that what is really inferred is that the consistency in the methods and benchmarks used to state “performance” are just that: only benchmarks. This is why storage manufacturers and vendors who provide systems for specific applications—media or transactional workflows—will almost always evaluate not just the individual HDDs and the arrays which are composed of those drives, they will brutally test them using their specific applications for the environments in which the arrays will be deployed.

There is much more to these metrics than space permits and with SSDs there are an entirely new set of dimensions which we find will change with age and the number of erase cycles. In a future installment, we’ll take a look at how flash memory and SSDs stack up in the IOPS perspective. Some of the statistics may indeed amaze you.

Karl Paulsen, CPBE and SMPTE Fellow, is the CTO at Diversified Systems. Read more about other storage topics in his book “Moving Media Storage Technologies.” Contact Karl atkpaulsen@divsystems.com.

File-based loudness processing, as one might expect, operates on audio files, either standalone, or extracted from an audio/video file such as MXF. The strength of a file-based processor is that it can gain foreknowledge about an entire audio file (or segment)—start to finish—before doing any processing. Because of this, file-based processing could potentially be less intrusive than its real-time counterparts.

Typically, file-based processing happens in multiple passes. First, the audio signal is analyzed, then—based on preset rules—the processor determines what type of functions or operations is needed and then applies them. Depending on the adjustments needed, further analysis and processing steps may occur in an iterative fashion, to arrive at the final result. While this may sound time-consuming, typical processing times are faster than real time. What kinds of measurements and processing are typical of file-based loudness processors?

DEFINED LOUDNESS TARGET
The first is fairly obvious—loudness. Adjusting the audio signal to reach a defined loudness target is fairly straightforward for file-based loudness processors. Unlike a real-time processor, which effectively rides gain throughout, a file-based processor inserts either attenuation or gain. if it’s needed, to reach the target loudness. This is done after performing a loudness measurement on the file, per ITU recommendation ITUR BS.1770-3, “Algorithms to measure audio programme loudness and true-peak audio level.”

Fig. 1: Example of a file-based workflow that includes loudness measurements and adjustments, and true peak limiting for the AudioTools family of products (Courtesy of Minnetonka Audio Software, Inc.)

This single gain shift (scaling factor) doesn’t change the dynamic range. “It’s like adjusting the gain control on a fader and leaving it alone for all of the content,” said Bob Nicholas, director of international business development for Cobalt Digital in Urbana, Ill.

In addition to loudness, ITU-R BS.1770-3 describes how to measure another parameter, true peak level. Both file-based and real-time processors can provide this measurement. According to ITU-R BS.1770-3, “true-peak level is the maximum (positive or negative) value of the signal waveform in the continuous time domain; this value may be higher than the largest sample value in the 48 kHz time-sampled domain.”

If only the sampled peak value were used, problems such as inconsistent peak readings, unexpected overloads, and under-reading and beating of metered tones could occur. Again from ITU-R BS.1770-3: “The problem occurs because the actual peak values of a sampled signal usually occur between the samples rather than precisely at a sampling instant, and as such are not correctly registered by the peak-sample meter… [The] use of a true-peak indicating algorithm will allow accurate indication of the headroom between the peak level of a digital audio signal and the clipping level.”

DYNAMIC RANGE CONTROL
File-based processors can be used for other related functions as well. One example is dynamic range control (as with the AERO. file option for RadiantGrid or the Wohler loudness appliance solution). Another is measuring and controlling maximum short-term loudness, maximum momentary loudness, loudness range, and dialog level (as with Minnetonka’s AudioTools family, which includes AudioTools Server, AudioTools FOCUS and AudioTools Loudness Control for Harmonic ProMedia Carbon, for another example.)

Simply scaling an audio file, even though it meets a loudness target, may not be enough to stop viewer complaints, if dynamic range remains wider than their systems can handle, as Tim Carroll, Telos Alliance chief technology officer and Linear Acoustic founder, pointed out. That’s the reason for adding dynamic range control to loudness processors. For a theatrical release movie, for example, dynamic range may need to be narrowed to make it a better fit for TV, while a sitcom may not need much, if any, dynamic range adjustment.

File-based processors are capable of making a variety of measurements to determine how to adjust the final output to reach a predefined target. This can often be an iterative process, as one adjustment may affect another parameter. Fortunately iterative processes are well-suited to software-based products. If the target isn’t reached after a certain number of attempts, the processor can send out a notification for human intervention.

File-based loudness processing is typically just one part of an overall file-based ingest or quality control workflow that can include audio up or down-mixing, Dolby E decoding and encoding, metadata adjustment, channel assignment detection and conforming, watermarking, pitch and time control, sample rate conversion, channel management (muting, copying, reconfiguring, replacing), and third-party functions and integration, according to Oliver Masciarotte, director of customer experience at Minnetonka Audio Software.

Fig. 2: Example of a more complex file-based workflow that includes multiple languages, program correlation and channel order verification, downmixing, loudness measurement, processing and peak limiting for the AudioTools family of products (Courtesy of Minnetonka Audio Software, Inc.)

As an example, the AudioTools family can integrate with Telestream Vantage and Harmonic ProMedia Carbon. File-based processing is also amenable to complex automation, according to Masciarotte.

The audio functions can occur alongside any video file-based functions such as transcoding, aspect ratio, standards conversion or quality control.

BRANCHING AND ITERATIVE WORKFLOW
Branching and iteration are keys to efficient and flexible workflows in file-based processors. Let’s look at a couple of workflow block diagrams for AudioTools to see how this works.

Referring to Fig. 1, start at the left side of the block diagram, where the audio essence is extracted from an MXF file. The audio is then checked for the presence of Dolby E. If Dolby E is detected, then the channels are first decoded to PCM before being sent to the measurement and loudness normalization stages. According to Masciarotte, loudness measurement and control, for the most part, needs to happen with PCM (pulse-code modulated) digital audio signals.

While not shown in Fig. 1, the workflow could be set up with another branch to check if the loudness meets the required target. If “yes,” the signal would continue on to the next stage, but if not, the audio channels would be subjected to further loudness level adjustments and measurements.

After the loudness adjustment sections, the workflow checks to see if the signal needs to be converted to Dolby E. Depending on the answer, different true peak limiting is applied. If encoding is required, it happens after the true peak limiter, and the signal gets re-wrapped into the MXF file.

Fig. 2 shows a more complex workflow that includes multiple languages, program correlation, channel order verification, and downmixing, in addition to loudness measurement and processing and true peak limiting.

With software-based systems, workflows such as these are defined according to each customer’s specific requirements. According to Masciarotte, these types of file-based systems can be modular to some extent, scaleable, and easily interoperable across WANs, MANs, LANs and SANs. The file to be processed can be local or remote depending on the system.

Some IT knowledge is typically needed to set these up and to ensure a good user experience.

Mary C. Gruszka is a systems design engineer, project manager, consultant and writer based in the New York metro area. She can be reached via TV Technology.

As a practical matter— to ensure compliance with the CALM Act or to produce deliverables to a specific loudness target—stations and content-producing facilities are relying more on loudness processors to automatically make adjustments to the audio content.

There are two broad categories of loudness processors: real-time and file-based. Within each type, loudness processing can be a stand-alone function or incorporated as part of a total processing package.

Real-time loudness processors operate in nearly real time (with some buffering and processing delay) to meter loudness of an incoming signal (mono or multichannel) and then typically continuously make audio level adjustments depending on the rules or presets it is given.

Fig. 1: General signal flow of a multiband real-time loudness processor, Linear Acoustic Aeromax

A file-based loudness processor, on the other hand, analyzes loudness from audio that was recorded as a digital file, typically as part of a video file. The analysis occurs over the entire length of an audio piece and then a scaling factor (gain or attenuation), if needed, is applied to the entire content, based on preset rules, so that the audio output is delivered at that new (target) loudness level.

REAL-TIME PROCESSORS
While they can be used for treating loudness on archival material on tape (instead of in a digital format), real-time processors are typically installed in the last stage of the audio signal chain, before an encoder, to catch any loudness problems that would make the program non-compliant.

A real-time processor fixes loudness “by adjusting dynamic range,” said Tim Carroll, Telos Alliance CTO and Linear Acoustic founder.

The processor continuously measures program loudness of the input audio signal according to the ITU-R BS.1770 standard (currently version 3, from August 2012). Then automatic gain control, or in other words, compression, is applied to adjust the level to meet the target. Think of this as riding gain with an audio fader.

Fig. 2: Comparison of multiband (left) and wideband multiloop (right) block diagrams for real-time loudness processing

“The [processor] is constantly adjusting the level,” said Peter Pörs, managing director, Jünger Audio GmbH. “The fader is never in a fixed state. It’s moving so slowly that you don’t perceive it.”

For content at relatively controlled levels, adjusting the AGC too fast will be audible, yet if a loud transient occurs, the processor must be able to react quickly to pull it down. This is why processors typically have an output limiting stage.

WIDEBAND AND MULTIBAND PROCESSING
Processors differ in how the AGC is applied. Some apply AGC across the entire audio spectrum (wideband). Others use the multiband approach where they break up the full audio spectrum into sections or bands and apply AGC individually to each band. Different attack and release times can be set for each band.

According to Carroll, if the processor runs in straight wideband this, in general, can make processing adjustments more audible. As an example, a loud thump could bring down the level of a whole program, even though it’s only the lower frequencies that caused the level to spike. With multiband processing in this scenario, only the lower frequencies would be reduced, (Fig. 1).

Four or five bands generally are adequate for multiband loudness processors, according to Carroll, and this is what is typical.

Another idea behind multiband processing is the way us humans perceive sound. We don’t hear linearly across the audible frequency range. We are more sensitive to mid-range sounds compared with those of higher and lower frequencies, but the difference changes as the audio level changes.

For example, for normal hearing at low audio levels, a sound at 100 Hz must be raised about 15 dB higher than one at 1,000 Hz for the two tones to be perceived as equally loud. As audio levels increase, the lower frequencies don’t need to be raised quite that much compared to the mid-range to be perceived as equally loud.

Two researchers from Bell Labs, Harvey C. Fletcher and Wilden A. Munson, studied this phenomenon and in the early 1930s published their results with graphs of equal loudness curves across the audio spectrum and at different audio levels. These have come to be known, not surprisingly, as the “Fletcher-Munson curves.”

Fig. 3: Block diagram of Jünger Audio Level Magic loudness management

That’s why if you compress the entire signal you change the relative levels of the high and low frequencies to the mid-range, and that, according to Bob Nicholas, director of international business development for Cobalt Digital, changes the character of the sound. This can have a negative effect on intelligibility.

“Multiband is not trying to keep things spectrally flat, but to keep things spectrally balanced,” Carroll said.

Nicholas said that multiband AGC is more applicable to a sound source that’s a mix of different signals and wideband is more for single source signals, like that used on a channel strip of an audio console.

Taking a different tack, Pörs said that the multiband approach can produce anomalies when the different frequency bands are summed together. “The possible difficulty is that [with] the overlapping zones of the filters, a precise summation of the signals is nearly impossible [and that] leads to coloration,” he said. “That’s why we came to wideband.” (See Fig. 2.)

Wideband with a twist, that is. “We have a different processing design approach,” Pörs said. “We call it multiloop design.” This design incorporates a series of gain controls, with each “fader” controlled separately. (See Fig. 3.)

“The various loops each work over the entire frequency spectrum,” Pörs said. “They work in parallel, each with a different set of attack and release parameters. Each loop develops a control signal which is then summed with the controls from the other loops to produce a single gain control signal applied to one gain control element.”

The algorithms in the processor provide automatic adjustment of the attack and release time based on how the input signal changes over time. “This is called ‘adaptive dynamic range control,’” Pörs said. “By monitoring the waveform of the incoming audio, the system can set relatively long attack times during steady-state signal conditions, but very short attack times when there are impulsive transients.”

In addition, the Jünger multiloop design allows for a very short time delay to be put in the audio signal path. “This lets the gain-changing elements ‘look ahead’ and determine the correction needed and to apply it to the delayed signal just in time to control even the fastest transients,” Pörs said.

No matter what design, it must be set and used correctly. More on this later.

Mary C. Gruszka is a systems design engineer and consultant based in New York. She can be reached via TV Technology.