New storage options

Eons ago, when cameras had scanning electron beams and electronic recording of sound was in its infancy, television was essentially a live medium. The only option for preserving and storing content was to shoot a high-resolution monitor with a camera synchronized to the frame rate of the scan. Kinescopes can only be found now when looking at archival footage of old, and originally live, shows.

To overcome the missing link that storage represented, the production community turned to the only other viable option — producing on film in the first place. There were two reasons for this. First, the transfer characteristics of film and television are not the same, which rendered making kinescopes with correct tonal representations challenging. Second, the quality of the recordings was at best marginal. Film originals transferred directly to television worked much better, a fact that was universal until the last decade of technical advancement.

In 1956, when Ampex stunned the NAB crowd with the first practical electronic recordings of monochrome images, the game changed forever. Although the evolution of electronic recording has been slow, in the last 20 years, the pace of technological change has quickened considerably.

New advancements in recording have flowed in large part from a fundamental shift in the processing of images from analog to digital. Can you even buy an analog professional recorder today? Magnetic tape has begun to disappear. Some manufacturers have ceased producing professional videotape. These market forces will play an increasing role in the future, in no small measure due to the popularity of tapeless recording in the consumer domain.

Borrowing from CE

Professionals used to disdain the technology developed for consumers as not up to the standards they needed, but economics have changed manufacturers' model. Where once broadcast and consumer electronics were developed in isolation, increasingly, the R&D dollars invested in the huge consumer electronics space has benefited the full vertical market all the way to broadcast and professional products.

For example, DV recording, which now covers 25Mb/s to 100Mb/s, was originally a consumer electronics initiative. The economy of the scale of CE production and development allowed the professional market to benefit before it even took hold in CE applications.

Similarly, MPEG was developed not for professional use, but for distribution of content in digital networks to the home. MPEG encoders were initially very expensive, but decoders have always been available at CE prices. MPEG's model was to put the expensive and difficult part of the process in the encoder and let simple decoders read the stream. But in the last decade, the cost of MPEG encoders has dropped sufficiently to put them in reach for both professional and consumer camcorders (for less than $500), and even in HD consumer camcorders. We all benefit from the necessity of sharing R&D cost.

Solving the issue of bandwidth

The professional market has much higher bandwidth requirements than the consumer marketplace, which will need to be addressed in the next decade. High-bandwidth, lightly compressed HD absorbs as much as 10X the bandwidth of even the 100Mb/s DV100 recorders.

The solution lies in new physics with new economics. We are lucky again to be able to ride the development of high-capacity hard disks and lower-cost flash memory. Today, consumers are often content to record on flash memory as well, but at much lower data rates than professional applications. With single-card memory solutions of 8MB, and as much as 64MB by the end of this calendar year, a consumer might record many hours.

By ganging multiple memory cards in a single package, and then stacking several in an access drive with appropriate operating system software to write across the complex media that results, professional applications can record much longer than a camera operator could hold a camera. And the research has achieved sufficient cost/benefit economies so that it has become quite practical.

Of course, hard disks compete in the same space. With appropriate packaging for field use, a hard disk record solution can be quite capable, though unlikely at the same cost as tape in the foreseeable future. It can compete in general terms with high-capacity memory recording solutions. One company has adopted a strategy of using either flash memory or a removable cartridge storage solution, which hits about the same economics as flash and is cheaper than repackaged hard disk systems.

The migration to file-based studios

In studio applications, the landscape is quite different. There is a strong movement to file-based recording, with large-capacity, integrated systems achieving almost arbitrarily high record/play bandwidth. On my desk, a $600 RAID 1 NAS system provides high availability to 1TB. Scaled to large capacities, the cost of the disks themselves becomes almost an academic matter. The infrastructure and software needed to manage the total network of storage is the most challenging problem from both an economic and technical perspective.

MPEG expert and author John Watkinson once spoke at a SMPTE conference about the inevitability of storing in complex environments. He said we would eventually drop our fixation on physical media and realize that all that is important is that the content we record comes back when we ask for it.

He has been shown to be very perceptive on this point. Content can be spread across many disks, or it may have been moved to nearline or deep archive, but if it is retrieved when we ask for it, it does not matter what the system had to do to deliver the content.

What's in store

Physics holds other developments at our fingertips. Among the most intriguing options is holographic storage. One of the attractions of holographic systems, and other optical storage, is that the life of the media itself is very long, perhaps long enough that it will not matter so long as a player still can be built to retrieve the data. Migrating to new media might be needed twice a century. Another key differentiator is that the media can hold a huge amount of content. Current products can store about 1.6TB and access it at 120MB/s (960Mb/s), making it much faster than real time for most video applications.

At the edge of the physics affecting storage are interesting options, including memristor memory. Built on a concept theorized more than 30 years ago, a memristor has a different resistance depending on the direction of the current. Memory built on this concept today would have 100Gb in 1cm2 and operate at the speed of dynamic random access memory (DRAM). More importantly, it would be inexpensive and nonvolatile. Computers, or video devices, built using this technology would never need to be booted. When power is restored, the system simply acts precisely as it did before it was shut off.

John Luff is a broadcast technology consultant.

Send questions and comments to: john.luff@penton.com