AI Edit Technique Tracks Viewer Reaction to Determine Story Angles

LOS ANGELES — What if viewers – not editors — determined the trajectory of a story? Filmmaker Armen Perian intends to find out with the May 18 release of his new short film, “The Angry River.”

While not a tech guru himself, Perian, along with tech partner Crossbeat New York, have created a film with the interactivity of a video game but without a game controller.

The idea came to Perian during a long editing session. “Someone said, ‘Man, I wish we could just edit this thing with our minds,’” he says. “It was totally off-the-cuff, but the idea stuck with me.”

His new short film, starring Jim Beaver ("Deadwood") and Brooke Smith ("Bates Motel") employs eye-tracking technology to determine what a viewer is watching – where the viewer’s eye lingers — and then edits itself into one of five possible storylines, each matched to the viewer’s interest.

While enamored with the idea of game theory, Perian “wanted to preserve a cinematic experience, even though you’re watching it on your computer,” he says.

Like any other film, a viewer watches the screen, but when the viewer pays attention to something on screen, that drives the action, and the movie changes based on what is being viewed. 

[Read: The Next Big Step For AI? Understanding Video]

Crossbeat New York developed the eye-gaze detection and machine learning that power the story’s algorithm. Perian directed the film, which was shot in Oregon over four days. Alex Hall edited the film into five distinct narrative tracks. A custom-built algorithm turns the five different perspectives into a story that looks and plays like a traditional movie.

Perian’s production company Pomegranate Films has seen interest in acquiring the technology from studios and tech companies.

“I resist the idea that this project puts me in the tech world. I’m still a filmmaker, a story person. I just wouldn’t be able to tell this story without the tech,” he says. “It’s a tool. Even though it’s this sexy provocative new thing, it’s use is to serve the story.” 

"The Angry River" will premiere online May 18. 

Phil Kurz

Phil Kurz is a contributing editor to TV Tech. He has written about TV and video technology for more than 30 years and served as editor of three leading industry magazines. He earned a Bachelor of Journalism and a Master’s Degree in Journalism from the University of Missouri-Columbia School of Journalism.