The future of data storage

How broadcasters survive and thrive in future markets will depend not only on their ability to create, aggregate and brand unique digital content, but also on how effective they are in making strategic decisions about where and how their content is stored. In today's connected economy, it will be crucial for broadcasters to be able to quickly repackage and deploy content to new markets.

The required business and creative decisions will need to be managed by an integrated set of ASPs supporting micro-payments, advertising and subscription models. If a content owner can't perform these creative and business functions economically and rapidly, the competition will.

Some history

Many established mainframe computer companies initially ignored new microprocessor technology and assumed it was similar to projects they were already doing internally. The entire PC revolution occurred because thousands of electronics people who were not previously in the computer industry saw it differently and created an entire new range of products. Before the 1990s, arrays of processors were mostly used in scientific computers or DSP systems. The introduction of multiprocessor-based desktop workstations in 1992 by Sun Microsystems, along with its version of Unix, allowed software developers to use multiprocessing as a mainstream business environment.

It was the commercialization of the Internet in 1995 that set off the real dotcom revolution. Nowadays we take it for granted that every household and every cell phone can have an Internet connection. Serving this worldwide marketplace offers enormous business opportunity.

Solid state disks (SSDs) are data storage devices that can currently store data at the same density per module as hard disks. SSDs have been around in one form or another for more than 30 years. Because of their high cost, they were initially used as accelerators in high-end servers. Today, retirement beckons to the hard disk drive. Hard drives have been a cornerstone of the computer market for 54 years. Hard drives represent the safes in which computers store data. Over the decade 2000 to 2010, the maximum capacity of a hard drive grew 17 times (from 180GB to 3TB). In 1999, a 1TB hard disk array cost $79,000. By the end of 2010, a 3TB hard disk cost less than $250. That's a 1000 times cost reduction for the same storage capacity.

But hard disks haven't gotten any faster or more reliable. Random access times are exactly the same, and user replacement cycles are the same as they were a decade ago. This is despite improved MTBFs, which have been cast in doubt by some large-scale user studies. Simply put, the data integrity of hard drives has failed to keep pace with their growing capacity. Large disk populations are vulnerable to random uncorrectable data losses.

The solution is that drive manufacturers simply add wraparound layers to protect users from data loss with the unfortunate side effect of slower data access times. Despite the success of the hard disk and its apparently dominant position in the storage market today, over the next five years, sales of hard disks for enterprise applications will fall by about one half and by 2019 will cease to be used in enterprises altogether. The main market for hard drives will become consumer products, which have lower performance, data integrity and cost-of-ownership requirements.

What's next?

Look first for SSD accelerators initially to be added to larger storage systems. This will allow users to realistically increase application speeds by a factor of three and increase burst Web apps acceleration rates by 40 times.

In the second stage of the storage revolution, SSDs will be added to notebooks, allowing them to run faster and provide longer battery life. Initially, SSD-equipped PCs will cost more and, therefore, appeal to a small subset of the notebook market, but prices will drop.

Consumer SSDs have actually been used in high-end laptops since 2007. Unfortunately, market acceptance has been slow because of poorly designed notebooks. In many cases, mediocre SSDs were simply dropped into motherboards, which had been designed for hard disks. In nearly all commercial notebooks shipped up to the third quarter of 2010, notebook chipsets wasted most of the theoretical speed advantages of SSDs. It will require several more product generations of notebooks before the real advantages of SSDs will become obvious to users. In the last stage of the SSD revolution (2015 to 2019), we'll see an entirely new class of SSDs. I call this product class bulk storage, and these SSDs will replace hard disks in the data center. They will offer higher storage density than HDs, lower operating costs and faster performance. Internally, these SSDs will have different architectures, which will be optimized for low power consumption, data healing and long operating life rather than just raw speed.

In many ways, the software architecture of tomorrow's bulk storage SSD rack will more resemble that of tape libraries. Instead of a robot mechanism grabbing a tape cartridge, some sections of the SSD will simply be powered down and then powered up only when the information is needed. Another key factor in this scenario — the atomic data interchange between the bulk SSD and its neighbor, the auxiliary acceleration SSD (which sits on the SAN) — will by today's standards be a huge chunk of data. This data will sit two levels away from the main acceleration SSDs and appear as a DAS connection to the applications.

In tomorrow's data-driven content factory, storage will be regarded as a profit center and not an overhead. The ability to redeploy content quickly along with micro accounting will create new revenue opportunities for content owners. This will include the ability to manufacture (customize) new content through software agents to prepopulate anticipated user demands and fully embrace the any content, anywhere at any time business model.

Going forward, storage architectures will remain just as complicated as they have always been, with several classes of SSDs being optimized for different roles. A key difference, however, will be that the only spinning devices in the storage cabinets will be the cooling fans.

Zsolt Kerekes is editor

Growth of SSD manufacturers

The number of SSD manufacturers has continued to increase over the years and is projected to explode during the next three years. One reason for the rapid growth over the 2010 to 2013 period is that OEMs will be attracted by growth in SSD revenue and a lack of clear monopolistic style leaders. There will be many different market leaders in many different SSD segments.

1998 - 10 companies

2005 - 30 companies

2007 - 63 companies

2008 - 92 companies

2010 - 200 plus companies

2013 - 1000 plus companies (forecast)