Shortly before the start of the SMPTE 2017 Technical Conference & Exhibition, TV Technology spoke with Michelle Munson, co-inventor of the Aspera FASP transport technology and CEO of Aspera until May 2017, and Yvonne Thomas, product manager of New Media for arvato Systems, about the SMPTE 2017 Symposium.

TV TECHNOLOGY: You co-chaired the SMPTE 2017 Symposium, and the topic is artificial intelligence and machine learning—relatively new concepts for the M&E industry. What are the top two or three areas where you expect to see AI and machine learning helping those in the industry? Why?

Michelle Munson

MICHELLE MUNSON: Currently, the most impactful area of innovation in AI and ML for the M&E industry is perhaps automatic video, image and audio indexing, which allows features of content to be recognized and tagged at scale, and in real time (live). This broad area has a host of applications that can dramatically help automate and scale the industry in the short/medium term:

- Automation of compliance and quality control in multi-territory distribution

- Automatic (live) editing and creation of screeners/previews and sub-clips

- “Tagging”/metadata enrichment of key elements like particular brands/faces/activities/scenes for the purposes of advertising placement or personalized recommendation; large-scale search for viewers; and content-archiving or repurposing

- Automatic scene change and keyframe detection for advanced compression techniques

- Automatic summarization of video content

- Speech-to-text translation for automating many heavy lifting tasks such as live captioning, live language translation

In benefit to the consumer, collaborative filtering techniques are widely used to providepersonalized content recommendation based on a user’s past preferences or the preferences of similar users. Deep learning neural networks are now beginning to expand these techniques to allow for dynamic content composition based on user sentiment to create dynamic, customized “choose your own story” experiences.

Additional major areas of applications are in enhancing content security and detecting piracy, e.g.:

- Anomaly detection in content security and access control systems allow for rapid discovery of potential deviant behavior in access to critical content systems based on identification of unusual events or usage patterns, and could flag breach attempts ahead of penetration.

- Deep learning techniques can be applied for highly accurate identification of low res or otherwise down-modified master content that has been pirated.

Advanced ML optimization techniques also have potential to dramatically optimize resource selection in content storage and distribution, such as compute, bandwidth, storage and internet routing to maximize user streaming and download experience and to reduce costs of content storage, exchange and distribution (see question 3).

Yvonne Thomas

YVONNE THOMAS: First of all AI is a general term, a summary for several kinds of technologies. Machine learning is part of AI, so are analytics, neural networks, cognitive computing, algorithms, deep learning, big data, linked data part of the family.

Metadata is very essential to the M&E industry and analytics services help to enrich metadata automatically and reliably, especially when it comes to huge amounts of data we humans are very inefficient in processing. However, it´s most essential that machine learning is part of those systems and evolves continously.

Second AI can be useful for predictions. Let me describe two examples here:

- A manufacturer (Thyssen Krupp) uses AI to predict when their elevators break and thus can repair them before they are out of service and avoid lengthy, costly and disappointing down-times. The same use case can be relevant to M&E technology, especially where we have 24/7.

- Another example seems pretty impressive. At the 2017 NAB Show, Banjo did a demonstration of their system in which they demonstrated all kinds of data souces (even police radio) and use this data for predictions. According to them they were the first ones in the world to report about the Paris attack. Especially for news, this is extremly essential to be the first.

TVT: Should SMPTE members be concerned they will be losing their jobs to AI and machine learning or happy that these technologies will make their work easier and them more productive?

MUNSON: I don’t believe the AI/ML era is significantly different in this respect than other big leaps in technological innovation (and that the risks are more about the ethical ramifications, see question 4). A few points:

- The automation and AI/ML can replace the “undifferentiated heavy lifting” (quoting our speaker from AWS Konstantin Williams)—in areas such as object and scene detection, facial analysis, facial recognition, face comparison, celebrity identification, image moderation.

- The sheer inventive possibility AI opens throughout the media creation and supply chain drives huge new revenue opportunities and consumer experiences.

- Success relies on new multidisciplinary expertise crossing devops, software, data science and media expertise generating new job opportunities of higher value.

- And, research has shown the most accurate models combine human analysis with machine-learned recommendations—meaning that what we are creating can largely improve the work of the human media professional rather than replace it.

THOMAS: No, I don´t think they must be afraid of AI. As said in question 1, we humans are very inefficient in processing huge amounts of data and AI can do this for us in a faster, more raliable and efficient way. It won´t take our jobs, but we will concentrate on what we are good at and we might have a shift in tasks. For example, we need new skill sets to train those systems (machine learning).

TVT: How do AI and machine learning play into more efficient resource use in media? I was thinking of tasks like closed captioning and subtitle creation, managing network traffic and storage for CDNs and also helping media companies deal with new requirements, such as delivering personalized content or channels to viewers based on their individual interests, preferences, habits, etc.? But maybe I am way off base here.

MUNSON: Many broadcasters’ media management and distribution systems still rely on brute force techniques to meet the demands of scale, which are no longer practical today—because they cost too much, fail to maintain uniform high quality of service in live and dynamic events or simply aren’t built for this era of direct-to-consumer, personalized media. Consequently, they are suffering in the face of extreme competition for OTT audiences dominated by the large Internet infrastructure companies. In content storage and distribution for example, the direct-to-consumer services today are relying largely on cloud storage and CDN architectures that use the same brute force edge storage and caching models introduced 10-20 years ago, which are often expensive and even at top dollar don't always provide deterministic, high qualities of service across all users or for live events. New machine-learning-based techniques can help make possible highly efficient storage and routing solutions that can reduce the costs dramatically, optimize high bandwidth distribution and adapt in real time.

THOMAS: It´s analytics services that are relevant for speech-to-text, face recognition, scene and concept recognition or network traffic management, etc. This metadata enrichment is extremely important to M&E to work as efficient as possible. In regard to recommendation engines I believe that what we currently use (no AI) is already pretty good, but is purely based on past data—because I watched movie X, I might be interested in movie Y. Analytics for recommendation engines will become more interesting to predict what‘s the next movie a customer will select.

TVT: What do you see as the top ethical concerns raised by the adoption of AI and machine learning in media?

MUNSON: Alan Turing, often credited as the father of modern computing, famously made the point that because it was not possible to measure the intelligence of a computing program/machine, saying that computers can be made “intelligent,” is nonsense. Another way of putting this is that much of today’s deep learning, “learns” in a way that imitates human behavior, but still heuristically, meaning that no one can explain exactly how these programs do what they do. In turn, many experts point out that the most advanced learning algorithms can neither be made fail safe or immune to manipulation by bad actors.

In media and entertainment the consequences of this are obviously not as grave as a self-driving car that “decides” to collide head on into another car, but are still concerning. For example, the very neural networks that drive high precision face and image recognition or live video editing could be manipulated by bad actors (or simply have software bugs) that cause the display of grotesque or otherwise offensive content taking advantage of viewers, advertisers or content owners. Additionally, the ability of today’s vast GPU/TPU compute power and scalable AI models could be used to manipulate news and create propaganda at scale that is undetectable or moves too fast for human detection; consider “fake news” at scale.

And finally, as an engineer I worry about the “commoditization” of AI tools such as ML programs that themselves create learning programs. These are already available to potentially unskilled users who may over-estimate their capacity or misuse their training accuracy, and could be a source of abuse—particularly when they are used to make decisions that can have clear bias or privacy consequences. Examples in media could be video surveillance and person matching that wrongly targets individuals based on profile, and security exploits that dwarf today’s worst exploits of IT negligence/ignorance.

THOMAS: I believe we must define rules and clear goals for what we would like to achieve with AI and where to use it. Think about the internet—the technology came into our lives pretty fast without any discussion about rules, and it has changed our lives drastically (positive and negative). As always technology can be misused, but let´s make something positive out of it.

HOLLYWOOD, CALIF.—If the central theme of last year’s SMPTE Technical Conference and Exhibition was about celebrating the centennial of the Society of Motion Picture and Television Engineers, this year’s has to be about positioning the society—in terms of its technical focus and the people it seeks to attract—for a future of rapid and constant change in media technology.

Organizers of the SMPTE 2017 Annual and Exhibition, Oct. 23-26 in Hollywood, Calif., have made a concerted effort to create an event that offers a new level of diversity, not simply in the age, race and gender of its presenters, conference chairs and attendees, but also in the exhibitors its attracting and the subjects it will tackle, according to SMPTE Fellow Michelle Munson, who along with Yvonne Thomas of Arvato Systems, is co-chairing the SMPTE 2017 Symposium, which kicks off Oct. 23.

Matthew Goldman, senior vice president of technology at Ericsson and SMPTE president, echoes Munson’s observation.

“This is our pre-eminent conference for imparting knowledge,” he says. “One of the things we have tried to do—especially with the symposium—is to make it a much more open topic area, the hot topic of the day.”

FOCUS ON AI

Michelle Munson

At this year’s symposium that hot topic will be artificial and machine learning in media. “If you’ve been wondering about the technical underpinnings and the state of the art in machine learning and AI, the symposium represents the capstone of some of the best work in terms of what services you can exploit,” says Munson.

The day will begin with a keynote from Jeffrey Kember, technical director of media in the Office of the CTO at Google, and will be followed by papers exploring AI and machine learning applications for media workflows. The afternoon will examine what machine learning is, characteristics of ML algorithms and applications in media, she says.

Munson herself will present on work she has been doing in ML aimed at creating new optimized algorithms for resource location in media storage and location distribution.

Martin Wahl, principal program manager for Microsoft’s Azure Media Services, will examine a new video indexing service from Azure, and Konstantin Wilms, principal solutions architect at Amazon Web Services will discuss content intelligence on AWS.

Yvonne Thomas

The day will wrap up with a panel on the ethics of and policy issues involved with the rapid advances taking place in AI and ML, says Munson.

NEW IP TRANSPORT STANDARD & BEYOND

The SMPTE Technical Conference, which kicks off Oct. 24, also will address emerging technical issues, but ones that are a little closer to present day concerns, such as the newly approved SMPTE ST 2110-10, -20 and -30 standard for professional media.

“There will be some great papers about 2110 and IP in media,” said Thomas Edwards, vice president of engineering and development at Fox, and one of three conference co-chairs.

Those include a report from the North American Broadcaster Association on media over IP; a look at the “superpower” of the new SMPTE ST 2110 standard by Leigh Whitcomb, principal engineer at Imagine Communications; a paper on synchronization and timing issues related to the new standard by Evertz solutions architect Paul Briscoe; and an examination of control, throughput and latency in multicast-based IP routing system by John Mailhot, Imagine Communications system architect for IP convergence, according to Edwards.

There will be much more to the conference than IP, however, says Sara Kudrle, product marketing manager for playout at Imagine Communications and conference co-chair.

“We are covering topics from mentoring and change management [to] AI, cinema, VR, AR, UHD, IP, 8K, ATSC 3.0, cloud, content management [and] quality monitoring,” she said.

Pete Putman, president of ROAM Consulting, will discuss what he calls the UHD “speed bump,” which describes the challenge of interfacing higher and higher data rates with consumer displays.

Rodney Grubbs of NASA’s Marshall Space Flight Center and Sandy George of Science Applications International Corp. (SAIC) will discuss the challenges and solutions to broadcasting live UHD content from the International Space Station.

Next-gen TV will come under the tech conference microscope with papers on acquisition and processing, HDR production using Hybrid Log-Gamma (HLG), HDR encoding parameters for streaming, 8K camera systems, and captioning and subtitling for social media.

However, there will be more to the conference this year than technology, according to Kudrle.

“There is a whole section dedicated to people, managing people, transition and change and managing diversity in the industry,” she said.

The focus on diversity is “really important,” especially because in broadcast engineering and digital cinema it’s “often been a challenge,” said Edwards.

A paper to be presented by Kylee Pena, workflow supervisor at Bling Digital and president of Blue Collar Post Collective in Los Angeles, will examine why diversity programs often fail and how to fix them.

EXHIBITORS AND MORE

There’s more to the SMPTE event than technical papers, however. “This is not just a papers’ conference,” says Goldman. “It’s also about the exhibitors.” As of Sept. 25, 100 companies have agreed to exhibit at the event, a high for the past decade, he adds. “We are at maximum capacity,” Goldman said.

Beyond the technical presentations and exhibition, this year’s conference will offer multiple opportunities to socialize and network.

HPA Women in Post and SMPTE will present the Women in Technology Luncheon, Oct. 23. The theme will be changing the status qu gender bias in media and entertainment. The luncheon will feature a speech by Madeline Di Nonno, executive director of the Geena Davis Institute.

That night, budding student filmmakers will present their work at the SMPTE-HPA Student Film Festival. The screening will take place at the TCL Chinese 6 Theatres.

The SMPTE 2017 Fellows Luncheon is scheduled for Oct. 24 and the SMPTE Annual Membership Meeting will also take place Oct. 24 at 5:30 - 6:30 p.m. in the Ray Dolby Exhibit Hall. The Oktoberfest Reception will be in the Centennial Exhibit Hall upstairs from the from 6 to 8 p.m.

The following evening SMPTE will hold its Trick-or-Treat Spooktacular Cocktail Reception from 6 to 8 p.m. in the Ray Dolby Exhibit Hall.

The conference social calendar will wrap up Oct. 26 at 6:30 p.m. with the SMPTE Annual Awards Gala followed by an after party.

As the evening celebration begins to fade away, SMPTE will say goodbye to the Loews Hollywood Hotel as the venue for its annual event. The 2018 SMPTE Annual Technical Conference and Exhibition will be held at the Westin Bonaventure in downtown Los Angeles.

“We are busting at the seams, growing and growing,” Goldman said. “As much as we like this venue, we are turning away too many people who would like to exhibit. That’s why we will be moving.”

For more information, visit smpte.org.