If your livelihood depends on a computer, you have the worrisome issue of whether it is securely backed up. The content can be video, audio, graphics, photographs, scripts, or whatever. If you’re like me—and your professional and personal life are mostly in files—you redundantly back up data multiple ways. 

I do backups to several hard drives, SSDs and two cloud services. Every year, I put one of my main SSD drives in a safe deposit box, in case everything else fails. Yet, I’m still afraid of losing data through some oddball thing I haven’t thought about.

The fact is, sooner or later, all storage media fails. Nothing invented yet is forever. Smart users (or should I say paranoid?) turn drives over every few years. But then you have to ask, how reliable is that new drive you just bought? Which brands are most reliable? Can I depend on a drive not failing? Sadly, the answer is a resounding NO. Every drive will fail sooner or later. The only way to protect your detail is through redundant backups to different drives.

Minimal Failure Rate
Fortunately, there are people measuring the failure rate of both hard and SSD drives. One of the best is Backblaze, a cloud storage and data backup company, which I use in addition to Apple’s iCloud on my Mac. Backblaze measures the failure rate of its 203,928 hard drives by annualized failure rate, or AFR. AFR = (drive failures /(drive days / 365)) * 100. 

Backblaze publishes a report every year about drive failure and has just released its results from 2021. For the complete report, Google “Backblaze Drive Stats for 2021.”

Out of over 200,000 hard drives, a total of 1,820 of these hard drives failed last year. The most reliable drive in the Backblaze arsenal was a 6TB Seagate (model ST6000DX000) with an annualized failure rate (AFR) of only 0.11%. This model also happened to be the oldest in its fleet, with an average age of 80.4 months use. 

Also performing well is a pair of drive models, including the 16TB WDC drive (model WUH721816ALE6LO), with an average age of 5.06 months. It has an AFR of 0.14%. The second is a 16TB Toshiba drive (model MG08ACA16TE), which has an AFR of 0.91%.

The total AFR for 2021 for all Backblaze’s hard drive models was 1.01%, which is slightly higher than in 2020 when the AFR was 0.93%. Overall, HGST and WDC are the best performers over the last three years, with an AFR under 0.5% in all but a couple of quarters. When you dig into the data, you find that there are drives from each manufacturer that perform well.

As for SSDs, Backblaze measured them for the first time this year. The company uses HDDs for its data storage and the SSDs are exclusively used as boot drives. These boot drives also store log files and temporary files produced by the storage server. Every day, a boot drive reads, writes and deletes files. As of the end of 2021, Backblaze was using 2,200 SSDs.

The annual failure rate for SSDs overall is similar to HDDs. In 2020, it was 0.78%; last year, it increased a bit to 1.22%. However, there are two glaring outliers in 2021: The Crucial (CT250MX500SSD1) and Seagate (ZA2000CM10002) SSDs failed at rates of 43.22% and 28.81%, respectively. 

The overall number of these SSD drives, however, is much lower than hard drives. Backblaze uses only 20 Crucial drives, and they were all installed in December. While there’s a high AFR, there’s a low sample size and a wide confidence interval. The situation is more extreme with the Seagate drive in question. There are only four of them in use and one of them failed.

Backblaze uses 1,090 of the Seagate ZA250CM10003 drives and it had a 1.04% AFR in 2021. The company also has 562 Seagate ZA250CM10002 drives. Its AFR was even more impressive at 0.36%. Backblaze has 250 Dell DellBoss VD SSD drives. Not a single one failed.

No Guarantees
Though the Backblaze data can help us small-scale users determine which models to buy, there is still no guarantee the drive won’t fail. The rule of thumb is all magnetic media, like that used in hard drives, has a half-life of about five years. No magnetic drive company claims its products can store data reliably for more than 10 years. So somewhere between 6-10 years, your data is going to disappear because the magnetic field is no longer strong enough to retain the data.

So what’s a computer user to do?  Essentially there are two choices today for serious archiving: either spend big money to record your data on a professional magneto-optical or other archival media for long term backup or use a combination of hard drives, SSD and cloud backup. For most normal PC and Mac personal computer users, the cost dictates the second option.

Another factor is there is no such thing today as permanent storage technology for the masses. The creation of a very long-term, highly reliable archival storage medium is the subject of a vast amount of research these days; it’s an ever changing technology. What’s state-of-the-art in storage today may not be a year from now. 

At the same time, everyone’s need to archive a vast amount of data is exploding. Though specific estimates vary, there’s a consensus that the rate of growth is about 50% each year. There’s no recession in the digital data storage business. 

Today, multiple backups and good luck are what’s needed for the most reliable data storage. 

MELBOURNE—Atomos and Nextorage have formed a partnership centered on the latest generation of Nextorage AtomX SSDmini, designed specifically to maximize performance and reliability of Atomos recorder devices.

Engineering teams from both companies considered all pertinent details to create a drive that meets the needs of demanding productions, Atomos said.

To fit the compact media slot of the NINJA V and NINJA V+ and the NEON series products, the Nextorage AtomX SSDmini is 20% smaller than conventional 2.5-inch SSDs. With the optional handle, they can be adapted for use with the SUMO 19, NINJA CAST and SHOGUN Studio 2, it said.

The housing is constructed from die-cast aluminum. It is shock resistant (up to 1500G/0.5ms), able to survive extreme temperatures and has been designed to dissipate heat during operational use. The reinforced, durable connector has been insert tested over 3,000 times to ensure that it can stand up to the rigors of daily use on set. For a filmmaker who believes the drive is taking a beating, the Nextorage AtomX SSDmini fully supports SMART data, allowing future AtomOS drive health monitoring, it said.

The firmware of Nextorage AtomX SSDmini also has been tuned to support sustained transfer rates required by modern formats (550MB/s read and 500MB/s write). Available in 500GB and 1TB sizes, the drives have been certified for all of the formats currently supported by Atomos recorders. Those include Apple ProRes RAW HQ 8K at 30fps and 4K at 120fps, which accommodate different file sizes and suit different types of production.

The 500GB SSDmini is priced at $229, and the 1000GB SSDmini has a price of $439. Nextorage provides a three-year warranty for the AtomX SSDmini.

More information is available on the Atomos website.

In my April column (“Putting the IOPS Where They Count”), the generalities of Input Output Operations (per second) or IOPS were explored from the physical device that is the hard disk drive (HDD) perspective.

We learned how the combination of disk drives in arrays can support the specific workload requirements of a system. Also discussed was a seldom-addressed factor called the “RAID penalty,” the impact of selecting a given RAID-level against the overall total system IOPS.

Continuing on this theme, solid-state devices (i.e., drives or disks)—known as SSDs—have a different set of impacts on the overall normalized IOPS equation. It is a fairly well-known fact that SSDs will age based upon the number of erase cycles the device is put through over the course of its normal use. Flash memory physics, a primary component of the SSD, is a contributing factor as to how IOPS performance will change.

Specifications for read/write performance are often given for SSDs, and Flash, at the time of their initial placement into service. What is surprising is that performance can (and will) change, sometimes dramatically, once the device is placed into service. This figure is irrespective of the number of erase cycles, but certainly dependent upon the SSD time in service and the number of read/write cycles, with or without erasures.

DIFFERENTIATING IOPS

Table 1: Dramatic IOPS comparison of the same equivalent storage sizes for SSD versus HDD. (A) is for a 12-drive, 50/50 read-write workload; (B) for a 60-drive, 50/50 load; and (C) for a 60-drive 70/30 workload. Note the variations in HDD rotations speeds (15K, 10K and 7K RPM) and the impacts on IOPS for each set of spinning disk arrays.

There are amazing differences in IOPS for SSD versus conventional spinning disks. Take, for example, a spinning disk array made of 10K RPM SAS drives, each in the modest 528 GB capacity (pretend they make such a drive). Given a 50/50 workload ratio of read/write and 12 drives in the array, for a RAID 6 configuration the usable capacity would be 4.7 TB—and the I/O performance would be 429 IOPS. A RAID 5 configuration would be the same usable storage, but IOPS increases to 600 (thus the meaning of the term “RAID Penalty”).

Apply the same configuration in terms of the same number of SSD drives, the same total usable capacity and a 50/05 workload, and the IOPS climb to 34,286 for RAID 6 and 48,000 for RAID 5. That is nearly an 80x increase at RAID 5 and RAID 6 over the HDD.

For a different perspective, change the read/write ratio from 50/50 to 70-percent read and 30-percent write. For the same HDD configurations, the performance for RAID 6 now increase from 429 to 600 IOPS; and for RAID 5, the change goes from 600 to 789 IOPS. For the SSD equivalent, the RAID 6 performance goes from 34,286 to 48,000 IOPS; and for RAID 5 the change is from 48,000 IOPS to 63,156. The increase between HDD and SSD is still an 80x improvement regardless of the RAID configuration. (See Table 1 for additional comparisons.)

CHANGING THE LANDSCAPE

You can easily see why the popularity of SSD in personal laptops, tablets and smartphones (whether you realized they were installed or not) has dramatically changed the overall system performance landscape. But how are these specifications measured, and how are they marketed?

A technical tutorial provided by SNIA (Storage Networking Industry Association) at the January 2015 Storage Visions conference in Las Vegas revealed some surprising issues about how the SSD is rated—especially in its new versus “broken-in” or “steady” states.

Performance of an SSD is about how well it functions at accessing, saving or retrieving the bits. This should be understood independently from the network, the application or any other processes or operations that could influence, negatively, the net overall performance. There are specific metrics used to judge or measure performance—more on this in a bit.

Fig. 1: This graph shows the impacts on drive aging. The STEADY STATE region is the time period used for the measurement, per the suggestions in the SNIA testing document. Note the TRANSITION period between FOB (“fresh-out-of-the-box) and the STEADY STATE (preferred test range) where the IOPS performance will dramatically change.

However, it should be recognized that SSD performance changes over time. There is a period between the “fresh-out-of-the-box” (FOB) and the “steady state” period where there is a “transition” period. The normalized IOPS performance, depending upon the type of memory’s multilevel cell structure, will actually do a step-function—negatively—from a peak FOB IOPS figure to the steady state IOPS measurement. (See Fig. 1 for selected relative comparisons.)

When developing a “level of performance” metric, the measurements will depend upon the type of IO request. The influencing metrics include whether the SSD access is sequential or random in nature, and varies based upon the size of the blocks, the read/write workload (as exemplified from the previous comparison) and how the IO access blocks are physically arranged on the media itself (known as “block alignment”).

Another factor for the performance is the duration of the writes in terms of continuous activity, based upon the number of hours of 100-percent writes across a specified block size (e.g., 4KiB = 4096 kilobytes; the kibibyte [KiB] being a 1000-unit byte for quantities of digital information).

SNIA has addressed a standardized measurement model for ensuring that the same perspectives can be applied across many SSDs. This procedure is detailed in the document “Solid State Storage Performance Test Specification” (PTS), developed, published and validated by SNIA.

How the SSD is used and for what purposes has a lot to do with how well and how long the device will perform and at what performance levels. Enterprise class implementations require much higher overall performance than what is generally put into the market for generic or consumer applications. Enterprise class devices are expected to be run at full workload and under steady state conditions 24/7; so they are tested (and hopefully spec’d) in those environments.

These are the types of devices most often found in professional applications, and for SSDs can come aggregated in all forms of storage configurations (DAS, NAS, SAN) and in as RAID or Tiered storage solutions.

IOPS OR LATENCY

IOPS become important metrics in both consumer and enterprise operations. That said, the management of response times (i.e., the mitigation of latencies) becomes a most important and overriding factor for what are considered high-level IOPS devices. Enterprise class SSDs with comparable IOPS may in actual practice differ significantly in response times and latencies.

Which leads us to the application—another contributing factor that can substantially change the need for some operational statistics in the storage device and physical media type. Random instances of activities that generate high response-time “spikes” become evident and obvious as the spinning gear appears and activities wait for the settling time in memory access.

For media-centric activities, this impact is apparent in editing systems, MAMs, VOD and other activities where high volumes of metadata read/writes are present. When the file is of a contiguous large size—stored sequentially on a storage medium—the read/write performance impact can be relatively insignificant.

However, for the editorial process or for graphics rendering, there are often hundreds of smaller files that are being acted upon, and are often spread randomly across the storage device. Any latency in writing the blocks or in accessing the file chunks will dramatically change performance, making SSDs ideal for metadata and high-volume database activities and potentially unnecessary for contiguous file retrieval.

For video server devices with content caching just ahead of playout, an HDD will provide a suitable real-time performance, assuming it was selected properly and designed for those activities. SSDs can provide improvements (as in VOD systems), but the HDD is, so far, a more economical solution—at least for most long-form content.

That, in a nut shell, is an overview of the issues surrounding SSD performance and a brief introduction on why the IOPS for the HDD is significantly different for an SSD. Later, we’ll look at more recent developments in the cell-structure of NAND, Flash and other SSD devices, and how those improvements influence the performance of SSDs.

In the meantime, choose your SSD wisely and know your application before deciding to wholesale forklift your space-taking, power-consuming, maintenance-intensive disk arrays.

Karl Paulsen, CPBE and SMPTE Fellow, is the CTO at Diversified Systems. Read more about other storage topics in his book “Moving Media Storage Technologies.” Contact Karl atkpaulsen@divsystems.com.