As a kid living in
San Francisco I
the Main Library card
catalog in search of
books on electronics
and telephone systems. Of
course, “analog cards” are
not computer friendly so
starting in 1995 library associations initiated
the mother of all metadata standards
to replace the basic card.
This standard is now called the Dublin
Core Metadata Set (ISO 15836) and
defines only 15 elements. These are:
title, creator, subject, description, publisher,
contributor, date, type, format, identifier,
source, language, relation, coverage, rights.
Metadata has two broad definitions. One
is “structured data about data” or as some
say—“the bits about the bits.” Another definition
is more literal and comes from the
Greek meaning of meta—alongside, that
is “alongside” that being described.
From these basic 15 elements bibliographical
information can describe any
The Dublin Core is not fine-tuned to describe
media, so during the last 10 years,
SMPTE, the European Broadcast Union
(EBU), PBS, BBC and others have broadened
(or reduced) these basic elements in
several important ways for the production
and distribution of media. Beyond Dublin,
media-friendly metadata supports frames,
time code, A+V sequences, mark_in/mark_
out positions and more.
SMPTE publishes RP210, a metadata
dictionary. This document (www.smptera.org/mdd) defines 10 different classes
of metadata relevant to describing media.
Of the ten, these four are common: descriptive,
structural, spatial-temporal (of
capture source) and rights.
SMPTE's RP210, a metadata dictionary, defines 10 different classes of metadata relevant to describing media.
Ideally, this dictionary can be referenced
by others to build facility metadata
schema. In reality, RP210 is one of several
dictionaries in use today. Due to legacy,
there are literally thousands of in-house
schemas that don’t have a pointer to a formal
Of the elements, title and length are
fundamental. Other basic elements are
descriptive and structural (or technical).
Descriptive is a form of narrative about
the essence; who, when, where, what with
frame accurate annotation in principal.
Structural is more about the media format;
MPEG-4, 5 Mbps, 4 PCM audio channels
and so on.
Both the EBU and PBS have developed
their own “media core” metadata sets
based on the Dublin Core. The EBU version
is called EBUCore and for PBS, PBCore.
Both of these have been widely accepted.
The Framework for Interoperable
Media Services (FIMS) refers to EBUCore
to describe metadata (wiki.amwa.tv/ebu).
One special kind of metadata is the
object ID. These are short, registered,
searchable, codes used to identify media
files. Examples of these are Ad-ID (spot
ID, tracking), EIDR (movies, TV pgms) and
ISAN (general AV). At registration time, essential
metadata is linked to the ID code
in a Web-accessible database.
Another special type is compositional
metadata. It is used to describe the details
of an A/V sequence including all edit related
data. The old-fashioned EDL and newer
AAF and Apple XML formats are typical.
With files ruling the facility, metadata is
paramount for searching and locating content,
implementing workflows, limiting access
rights, facilitating reuse, and describing
the contents of files.
Without metadata, content loses much
of its value if it can be located at all in a
sea of files. In what movie (file) did Clint
Eastwood say “Go ahead, make my day”?
Metadata is essential to answer this question,
or ask Ken Jennings of “Jeopardy”
Fig. 1: Metadata integration operations
In Fig. 1, the lower half of the diagram
shows the flow of media and metadata in
a typical production workflow.
A most common production operation
is a keyword search and subsequent
AV file play. The functions of a media asset
manager (MAM) are designed for this.
Metadata is a MAM’s heart and soul. MAMs
span a wide range of functionality. A mini-
MAM supports search, browse and creating
clip lists. A maxi-MAM covers many
aspects of the lifecycle of a piece of media.
Metadata is linked to its media in three
Media and metadata are embedded in
the same file wrapper. For example, the
file movie123.mxf contains both media
and select metadata.
Media file1 and metadata file2 are located
in the same directory or linked using
identical file names or similar. For example,
movie123.mpg and movie123.xml
share same name so are linked. Some call
this metadata a sidecar since it rides alongside
the media as a separate file.
Metadata, residing in a database of other
repository, is linked to its media by reference.
The database can be local or in a
distant cloud. The linking process can rely
on the SMPTE Unique Material Identifier
(SMPTE ST330) or similar.
In 2012, most media facilities have defined
an in-house schema for their metadata.
When exporting or importing metadata
from partners, often there is a format
mismatch. This is a pain point. There is
very little metadata interop except within
One bright point is the work from the
Advanced Media Workflow Association
(AMWA). They have developed several
MXF-based formats (Application Specification)
that include defined metadata for file
A good example of this is the new AS-12
used for commercial delivery. It contains
a “digital slate” with consistent metadata
METADATA INTEGRATION USING ETL
With the mining of program related
Twitter, Facebook and other sources of
Web information, metadata is becoming
richer in form and content. Importing and
exporting rich metadata often requires
specialized tools. True, most MAM products
offer some form of format conversion
and mapping. However, there are times
when the metadata needs to be processed
and managed independent of the MAM
system. This is especially true when a facility
supports multiple schemas.
The problem of manipulating data records
is an old one. The basic need is to
import a record(s), transform it, and export
it. This functionality could be called
ITE, but it’s not. It’s called ETL—extract,
transform and load. ETL operations are
widely used with databases (see the upper
portion of Fig. 1).
Let’s look at a test case. You are given
1,000 media files to import into your program
library. There is a short XML metadata
file associated with each media file. The
metadata format is completely different
from your house standard. Use a commercial
ETL product to transform the metadata
ready to import into your MAM.
There is a wide selection of ETL programs
to choose from (see Fig. 1). Most
can leverage the cloud to scale for dataheavy
projects. If you are not familiar with
these, download one for a trial run. Most
have intuitive, graphical UIs with built-in
templates to do just about any data transform
you can imagine. I am impressed with
these tools and see a long life ahead for the
category. In a future column I will discuss
the world of “Big Data in the cloud” and
how this will impact workflows of media
Al Kovalick is the founder of Media
Systems Consulting in Silicon Valley. He
is the author of “Video Systems in an IT
Environment (2nd ed).” He is a frequent
speaker at industry events and a SMPTE
Fellow. For a complete bio and contact
info, visit www.theAVITbook.com.