CAMBRIDGE, MASS.—Humans have been watching and enjoying moving images have been a part of the world since the late 19th century. Now, as 2017 comes to a close, moving images may also be something a machine can watch and comprehend, according to a blog post from the MIT Technology Review.

The post shares that Google has recently launched a tool that is capable of allowing AI systems to recognize objects in a video as part of its Cloud Platform. Previously, AI was only really capable of recognizing objects from a static frame. An MIT program is looking to now see if AI can determine not just what a video contains, but what’s happening in the footage as well.

To read the full story, visit the MIT Technology Review.

Want to learn more about AI and the future of broadcast? Attend TV Technology’s webinar, The Future of Artificial Intelligence in Broadcasting, Wednesday, Dec. 13 at 2 p.m., EST.

InputOutput

InputOutput

InputOutput

CAMBRIDGE, MASS.— Scientists at MIT have used machine learning to create video from a single still shot.

“In our generation experiments, we show that our model can generate scenes with plausible motions,” Carl Vondrick, Hamed Pirsiavash and Antonio Torralba said in a paper to be presented at Conference on Neural Information Processing Systems in Barcelona next week. “We conducted a psychophysical study where we asked over a hundred people to compare generated videos, and people preferred videos from our full model more often.”

The team started by setting up an algorithm to “watch” 2 million random videos—about two years worth—to learn scene dynamics, and use that knowledge to generate video.

“We use a large amount of unlabeled video to train our model. We downloaded over 2 million videos from Flickr by querying for popular Flickr tags as well as querying for common English words,” they said.

These videos were divided into two data sets; one unfiltered, and the other filtered for scene categories, of which four were used—golf course, babies, beaches and train stations. The videos were motion stabilized so static backgrounds could be more easily differentiated from foreground objects in motion.

This allowed researchers to set up a two-stream video generation architecture (illustrated below) that would produce a “foreground or [a] background model for each pixel location and timestamp,” a methodology reflective of the way video compression codecs “reuse” pixels in static scene elements.

The video generator produced 32-frame videos a little more than one second in length, at 64x64 resolution. These were run by a discriminator network programmed to discern “realistic scenes from synthetically generated scenes.” This served to further instruct the algorithm to create “plausible” motion, described by Motherboard as “far surpass[ing] previous work in the field.”

Vondrock, a Ph.D. student at MIT’s Computer Science and Artificial Intelligence Laboratory, wrote the paper with Torralba, and MIT professor, and Pirsiavash, a former CSAIL post-doctoral student who is now a professor at the University of Maryland Baltimore County, according to CSAIL.

See “Generating Videos With Scene Dynamics,” by Carl Vonrick, Hamed Pirsiavash and Antonio Torralba.