Data-Defined Storage

One of the hotter topics in the professional broadcast world is the generational transition to IP technologies for video.
Publish date:
Social count:

One of the hotter topics in the professional broadcast world is the generational transition to IP technologies for video. When examining this paradigm shift in video production and transmission technologies, the area of control management can be found to be the single most critical component in the entire system. This central controlling architecture for video-over-IP (or “IP-video”) is that mechanism, steeped in software, that makes the system work. Professional broadcast video infrastructures will soon rely on this most important element, because without it, connected video, audio and metadata cannot move throughout the IP-network.

Image placeholder title

The foundation of a new network SDS in an enterprise environment allows owners to utilize existing (legacy) storage in a VM configuration. Network control management is intrinsically structured around “software-defined networking” or SDN. The basis of SDN originally comes from a routing decision platform which is directly associated with networking. However, in recent times the term SDN for video networking has become an over simplification associated with the general statement of how broadcast video routing technologies will be controlled going forward. Ironically, we didn’t (in the past) refer to a video router control system as “software-defined,” despite the fact it is indeed entirely software-based (sans the human interface push-buttons on the control panels).

So why the change? Probably a cross between actual networking technologies and a considerable amount of marketing hype.

Not to be left out from other “something-defined-systems” domains comes another prominent terminology known as “data-defined-storage” or “software-defined-storage.” Storage, like video and networking, continues to grow in all dimensions. Thus some of the terms including “scale-up” and “scale-out” are now being augmented by newer more “network” or “cloud”-centric solutions.

When we start exploring other “something-in-the-cloud” models, we also find “storage-as-a-service” (STaaS)—sometimes known also as “SaaS” which is also used in cloud frameworks to indicate “software-as-a-service.” It doesn’t take a lot of imagination to easily see that technology and marketing continually appear to be on the usual collision course, which unfortunately masks the actual functionality, while adding to frustration, inconsistency, and sometimes confusion for the entire set of “X-defined-Y” solutions.

To make better sense of this entire landscape, let’s look at storage-related “X-defined-Y” functions and services as the computer world sees them, in particular with “data and/or software-defined-storage.”

The storage industry considers software defined storage (SDS) to be a new approach to the value propositions associated with managing and protecting data, as it relates to the storage architecture. For quite some time we’ve grown to realize that not all storage systems are created equal. Storage solutions are generally assembled to pro vide specific functionality, and as such are tailored to allow users to perform certain tasks based upon the applications which address that storage.

One example is the editing system which needs relatively fast throughput (input/ output or “I/O”); but not necessarily in high volumes of storage capacity. This model relates to software defined storage when the value of associating the applications and devices addressing this tier are coupled to a repository of captured metadata and essence. Software-coupling empowers users to access, query and manipulate the critical components of the data to transform it into useful information, while providing a flexible and scalable platform for storage of the underlying data itself.

The continual grown in data creation is driving the enterprise and its users to alter their storage strategies beyond traditional file- and block-level storage. A growing interest in object stores is now leveraging new software-defined storage mechanisms. Object storage is rapidly gaining traction within organizations that routinely generate massive amounts of unstructured data, especially when it comes to archiving that data. Data defined storage is a variation to object storage which is also gaining increased attention as systems move from dedicated host/client/server models to virtualized machines with global or consolidated storage.

Data-defined storage focuses on metadata. In this context, storage smarts now shift toward an emphasis on the content, its meaning, and its informational value versus what was traditionally only the media’s type or location.

By media type, one could infer that the storage medium might be a spinning disk, an optical disc or a linear digital tape. And by location, the store could be a local DAS drive, a high-volume/high-capacity central SAN in a data center or a cloud-based store whose geographic location could be anywhere and most likely completely unknown.

However, when considering the more important value proposition, data-defined storage would reflect more about what the content actually is (e.g., static images, moving images, text, etc.,) and what meaning that data has as it is organized to the context by which it would be used.

Data location is becoming less important as organizations become more globally diverse as evidenced by those enterprises who have begun migration to cloud technologies, whether private or public. As user bases expand, the enterprise now begins to depend upon a more data-centric management approach as it enables a single, more unified means of managing its data across expanding and distributed locations.

The enabling technologies that drive content use and metadata utilization include “media independent data storage” whereby the media-defined storage eliminates the storage boundaries typically anchoring media to specific storage media. Those boundaries, touched upon earlier, can be categorized within or across devices such as solid-state drives (SSD), hard disk drives, cloud storage and tape storage platforms. This essentially “scales-out” storage across many domains and leverages the capabilities of various storage systems, irrespective of their location.

Another formidable advantage of data-defined storage is that of data security and identify management. By employing data-defined storage, organizations are clearly able to address end-to-end identity management right down to individuals and devices themselves. This concept is very helpful in managing mobility requirements as it enhances both information governance and data security in a more unified and comprehensive fashion.

In the next installment, we will look at “software-defined-networking” which enables indexing and complex metadata association and derivatives. Don’t be surprised as you start to hear more and more about these topics going forward.

Karl Paulsen, CPBE and SMPTE Fellow, is the CTO at Diversified. Read more about this and other storage topics in his book “Moving Media Storage Technologies.” Contact Karl at