Artificial intelligence, and its implications for the film and TV industry, were among the main topics of discussion at the SMPTE DC Chapter’s “Bits by the Bay” conference held in Chesapeake Beach, Maryland, from May 21-22.

The conference—organized by the late Peter Wharton, former chief strategy officer for TAG Video Systems—returned after a long hiatus, and the event is dedicated to his memory.

Rounding Up
In a presentation devoted to fractional film frame rates, James Snyder, formerly with the Library of Congress’ National Audio Visual Conservation Center (NAVCC) and now an industry consultant, discussed the evolution of frame rates in tandem with the development of film and television and the expectation that with the film industry transitioning to digital, that frame rates can be modified to address the streaming market.

The battle waged over interlace vs. progressive image standards during the transition from analog to DTV several decades ago illustrated the coming battles between broadcasters—most of whom favored interlace—and the computer industry, who favored progressive.

In a world where 24 fps in traditional film is really 23.98 frames per second (fps) for TV and digital video, and 60 fps is technically 59.94 fps, Snyder said it’s time to round up to the next number (an even integer) because advances in technology have made fractionals less relevant in today’s media distribution ecosystem.

As streaming and gaming have taken hold over the past several decades, progressive has won that battle, and streamers simply don’t care about frame rates, Snyder added.

“Interlace and fractional are two sides of the same coin—they're both problems we need to get rid of,” he said. “[Interlace is] the easiest to get rid of right now, because all displays are now natively progressive; we have no CRTs anymore and progressive is baked into our technical distribution now, or at least the end viewing. And frankly, getting rid of fractional is also baked into it.”

Snyder blamed the current U.S. broadcast TV standard.

“The only reason we can't do even integer right now is because ATSC 1.0 is in the way; ATSC 3.0 can accommodate it, the displays can accommodate it,” he said. “Every vendor I've talked to who has equipment that does production or post production can switch to even integer. The only thing that's standing in the way right now seems to be ATSC. 1.0 is a technical blocker.”

Snyder said he came to this conclusion in his work at the Library of Congress, where he worked in film digitization and restoration. Adherence to global media standards and the demands of storing and managing millions of hours of film and video content prompted him to think of new ways to approach the problems of cost and efficiency.

“We had to deal with the problems of interlace, we had to deal with the problems of fractional frame rates, and we had to deal with the technical issues that came with creating new versions for reuse, because people could license the content, and they would ask for various versions,” Snyder said. “And so I got to see firsthand the extra cost. And the reality is, there are increased costs just in producing new content, there are increased production costs when it comes to reusing old content.

“And so that's where a number of us got together and asked ourselves, ‘do we really need to be doing this?,’” Snyder added. “A genuine technical reason why we're still doing this, and the only one we can come up with is ATSC 1.0. If you take ATSC 1.0 out of the mix, ATSC 3.0 doesn't have that problem, and streaming certainly doesn't have that problem. Streaming doesn't care what your framework is. Your computer just adjusts it to present the content.”

Snyder added that this approach needs to be modified when considering live broadcasts.

“The one area that still needs some discussion and some work is, ‘how do you go from live feeds that are fractional to live feeds that are integer?’ We do need to talk about that. There are converters out there that do the work we can do… but they have a strange cadence to them. We need to figure out how to not have that strange cadence, and work it out in the software and the hardware.”

AI and Democratization of Media
John Footen, managing director with Deloitte, discussed the democratization of media production over the past four decades and how AI is impacting its future, pointing out that for as little as $500, today’s mobile device can produce professional content that cost thousands of dollars to produce in the 1980s. (Footen also pens TV Tech’s Media Matrix column.)

“You don't even have to have an education to do it, people are just doing it … and that's an overall trend of democratization in content production and distribution,“ he said. “Are we done democratizing media? Is it all democratized out? The answer is no.”

The ability for the average Joe to produce Disney-level content powered by AI is coming sooner than we thank, Footen added.

“That level of production is going to be theoretically possible for good or for bad, it's going to happen,” he said. “And it’s going to further democratize the content types that can be created by anyone. You don't have to be good at using a camera because you don't have to touch a camera to make an image, right? You don't have to get a lot of training and framing and all that stuff, because a machine will frame it for you.”

Footen elaborated on agentic AI, the concept of using AI “agents” to do specific tasks that could eventually replace the “app” environment of today.

“You can think of it interchangeably with the word 'apps' that you've been using today,” he said. “An agent is a system … it could be a simple system or it can be a complex system that performs a function. It actually does something like you would expect out of many kinds of applications.”

As these agents become more adept at each individual’s media tastes, this trend will lead to a breakdown of brands that still act as a sort of “walled garden.”

“A curator agent is an agent that works on your behalf to get your content for you today,” Footen said. Using the Plex video playback app as an example, Footen said the more information a user adds to it, the more intelligent it gets, the less important the origination.

“All you need to add to that is keeping track of who you are and what your preferences are, and giving it access to your data and giving it this AI capability to curate for your content,” he said. “Eventually, that curation disintermediates the media company, because why do I care what channel it's on, what app it's on… if it's just coming to you through my agent, my agent is my app. I don't care where the content came from.”

Content Validation
Matt Galek, Footen’s colleague at Deloitte, discussed the issue of authenticated content in the age of AI, elaborating on the latest developments around the Content Authenticity Initiative (CAI), a six-year-old project designed to to promote transparency and authenticity in digital content; and the C2PA, a consortium of tech companies tasked with developing the technical standards for content credentials, among other things.

C2PA is the “leading standard” to confirming and maintaining trust in digital content and is on the path to becoming an ISO standard this year, according to Galek, who pointed to the three pillars of technology that drive the standards developed by the CAI: metadata, fingerprinting and watermarking.

“All these together provide a solid foundation for authenticity,” Galek said.

While C2PA’s standards don’t deal with detection of AI-altered media, it starts with the assumption that most of it is now, Galek said. “C2PA kind of assumes that users are going to be changing a piece of media, and [in] some of the research I was looking at, most studies were saying that over 50% of people posting content to social media platforms have manipulated in some way, whether it's due to editing that content or adding a filter,” he said. “So that's a significant amount that has been altered in some way, and kind of unknowingly, you're consuming that media.”

Galek pointed to the importance of metadata in tracking a piece of content’s evolution through the complete production chain and how that vital information is getting lost when content migrates to various platforms. Adopting C2PA will help protect that information, he added.

“There's a ton of tech that’s used today where metadata gets stripped inherently out when media is transferred from one system to another,” he said. “For vendors adopting this standard, you will have that methodology to retain that metadata through an entire supply chain.”

Working groups with the C2PA are focused on areas like audio and video watermarking, but Galek said there has to be intra-industry cooperation.

“There needs to be an additional spotlight on the implementation side of things, to provide a forum for broadcast-based and other industries to collaborate,” he said. “We know how increasingly fragmented the broadcast vendor landscape is, so it’s that much more critical for us to collaborate together around content authenticity.”

Detecting Deep Fakes
Anyone who has dabbled in generative AI knows how better it is getting with time but altering video and images (aka “deep fakes”) has been done for decades, something Ed Grogan, a motion imagery researcher and former Defense Department official, discussed during the session “The Dark Side of AI.”

(Grogan discussed his work in video forensics with TV Tech in 2018).

“When we started this, it took a little Hollywood studio to really make good deep fakes, and they couldn't do much,” Grogan said. “We’re now at the point that some of these levels where that little Hollywood studio was are [now] being done by middle schoolers.”

Using a historic picture of Winston Churchill, Franklin Roosevelt and Josef Stalin at the Yalta conference in 1945—with Groucho Marx in place of the Soviet dictator—Grogan emphasized the importance of spotting discrepancies as early as possible, since a widely distributed AI-altered image could rapidly alter public perception. In other words, it could allow a lie to travel halfway around the world before the truth has its boots on, a line often attributed to Churchill himself.

“If you think it's fake, put it in your documentation, because someone 20 years from now might lose the reference material,” he said. “And if you don't say you think it’s fake and the audience didn’t have the historical reference, they might not know that really should've been Stalin and that’s the wrong Marx. So sometimes, because they don’t have the historical understanding, you can get something past somebody to tell your story.”

Although no one sees media companies losing their gatekeeper status anytime soon, Footen summarized the promise and perils of AI in removing media companies as the “middleman” for content consumption.

“If everyone can do what we're doing today, I think our industry is going to be disintermediated,” Footen concluded. “I'm not sure how much of a media industry we have left when everyone is a media company. Every person, every company, is a media company. Our job, historically, has been to create the content, to gather the audiences and to monetize that content. But what if we’re not needed to do that? That’s kind of happening already.”

COLUMBIA, MD.—For two decades, Ed Grogan has been hunkered over video monitors and computer terminals performing a very unusual type of video “quality control.” This particular QC work, however, doesn’t result in less-than-perfect footage being sent back to the colorist or returned to the production company to be reshot. In Grogan’s case, what he sees or doesn’t see may make the difference between American lives being lost or unnecessary deployment of big-ticket military resources.

Grogan, a four-decade civilian employee of the U.S. Department of Defense, has spent half of his career directly or indirectly involved in the relatively new field of video forensics—a painstaking and careful analysis of recordings produced by enemies of the state for any evidence of less-than-honest presentation of events, and any and all clues that could spell fakery.

So, why the interest in such video?

Grogan spelled out the answer very bluntly.

“American lives may be at stake,” he said. “Terrorists don’t get paid unless they can show that they killed or injured Americans,” adding that terrorists are not always inspired by idealism or religious fervor, but rather as a means of financial support for themselves and their nefarious activities.

“Sometimes they manipulate the video they’re submitting for payment to show things that didn’t really happen,” Grogan added. “In other cases, recordings are prepared for propaganda purposes—to show that they fought with the ‘infidels,’ to show that they are in tune with the mission, and to try and get others to follow along in what they are doing.”

Grogan offered a recording of a motorized improvised explosive being remotely piloted to explode under a military vehicle as an example of this attempted video legerdemain.

“In some of the videos it was obvious that the bomb went off early by several frames,” he said. “This probably damaged the radiator rather than the main parts of the vehicle. The bad guys drop a few frames and this moves the explosion so that it looks like they made a direct hit on the vehicle or whatever when they really didn’t. That’s quite creative—just offsetting things a few frames. They haven’t necessarily been good at this sort of manipulation, but have gotten better over time—at least some of them.”

Grogan speculates that the quality improvements may be the result of terrorists now relying on professional videographers and post-production specialists.

“We’re seeing really good skill sets in some of the more recent stuff,” he said. “They do make mistakes from time to time, though. Sometimes they’re not good at greenscreen work, or they do a garbage matte and someone will put his hand right through the matte. Maybe this is just for a frame or two, so you have to look quickly.”

TIME IS OF THE ESSENCE

In the case of the military video forensics, decisions as to whether it’s real or manipulated have to be rendered very quickly to avoid either placing lives at stake or unnecessarily—and very expensively—deploying military resources that could be better utilized elsewhere.

“We have to look at all the videos and make decisions based on this evaluation, because if the videos are real, someone has to be alerted as to where American or allied forces might have been killed or injured,” Grogan said. “I get calls every now and then with someone saying ‘we’ve got some footage and they want a report all the way up to the President—what do you think?’ They’ll tell us ‘you’ve got 20 minutes; please send us your thoughts.’”

MANY CLUES IF YOU KNOW WHERE TO LOOK

During his career as a video forensics analyst, Grogan has compiled a checklist of some 90 “fingerprints” that could suggest less than truthful presentation of events.

“Any time I saw what appeared to be a mistake, I wrote it down.” Over time he was able to reduce the original list down to about 50 items. These include:

● Stuck pixels
● Errors in focus and lighting in various parts of images
● Gamma errors
● Reflections that don’t match the object supposedly casting the reflections
● Histograms that don’t match across certain scenes
● Errors seen in shadows and reflections of objects
● Inconsistent black levels across a frame
● Incorrect intermixing of even and odd video fields
● Mixing of 8-bit and 10-bit video

TOOLS OF THE [BOGUS VIDEO] TRADE

Grogan said terrorists have been known to use any tools that can be pirated to doctor video; however, some of the bad guys do stay up-to-date with post-production software and may even secure legitimate copies.

“Over time, terrorists have gotten fairly good at creating stuff that was faked to make their point,” said Grogan, adding that there are exceptions. In one case, implementation of Adobe software by a less-than-fully-knowledgeable group resulted in much consternation by forensicists.

“This was back in the analog NTSC and PAL days. We were seeing a lot of stuff that was really, really weird,” he said, referring to motion artifacts being observed. “It was just plain stupid and we couldn’t explain why.”

Eventually the answer came from an individual who had knowledge of video production as practiced in the Middle East.

“Through luck we got to talk to this guy,” he said. “And he told us what was happening, saying, ‘Oh, I can explain that. We opened the box from Adobe, loaded it and ran it. We took the video in and processed it and outputted it. I knew that it was set for American standards and everything we were doing was PAL, but we were directed not to change any settings, so the software converted the (frame rate, and line count) going in and coming out. This was the philosophy. Don’t touch the software settings. If it came from the company that way it must be right!’”

This quickly explained the dropped/added frames and other previously inexplicable artifacts observed in the video.

HOW DO YOU GET TO BE A VIDEO FORENSICIST?

Asked about his career track, Grogan was quick to state that scrutiny of video was not really part of his educational track, as his formal training was in electrical engineering. Shortly after taking his BS degree from Drexel University, he first took a job at the Philadelphia Navy Shipyard while earning his MSEE. Grogan then accepted a position in Washington, D.C., with his initial duties involving military communications and the development of engineering solutions for problems being reported in the field.

“I never worked in broadcasting,” said Grogan, “But I was the AV guy in high school, and while I was in college, I did do some work in video production for a cable TV company, but I liked the hardware side of things better.”

His eventual springboard into video detective work stemmed from a conversation with a co-worker.

“We had a secure phone and one of the guys said that he couldn’t wait until we had a picture phone, as then he wouldn’t have to take the steps necessary to verify who was calling. I said ‘yeah, provided you could trust the video, because I could fake it.’ This was around 1996 and I knew enough about Hollywood then to know things like this could be faked.

“I wrote a technical paper on this and rather quickly the question arose that if I could create fake video, could I detect it? My paper got circulated through the DoD and I was asked to do some briefings. Later, I was asked to teach classes on the effects of video deception. This is how I entered the world of video forensics.”

Grogan continued to screen recordings for tampering, eventually doing “post-graduate” work of a sorts with a DoD-sponsored four-month stay in Hollywood, where he had free access to most all of the post-production houses there and got to observe that community’s professionals at work in creating visual effects for the big screen.

“That was extremely helpful,” he said.

THE NEXT GENERATION

Although not now directly involved in video forensics, Grogan does step in to assist from time to time, and observed that sometimes identifying video prestidigitation can depend on one’s perspective. He recalled a particular incident in which he was asked to evaluate a scene involving movement of a vehicle at dusk.

“I said that I didn’t think it was possible for the camera to capture such a lousy image of the truck while at the same time producing such gorgeous images of the stars above,” he said. “Sometime later, one of the new hires—a young kid really—looked at the same video and recognized the buildings in the scene, saying ‘Oh that’s from the so-and-so video game.’ He correctly identified footage the bad guys had appropriated from a game.”

Grogan recalled another case involving propaganda footage of an airliner being shot down. He calculated the required elevation of the camera to capture the images shown, and quickly determined that it couldn’t have been on the ground.

To further “ice the cake,” Grogan said one of the younger teammates was able to identify the explosion in the propaganda video as having been excerpted from a video game. “We were able to match it pixel-by-pixel,” he said, noting that “perhaps there just may be a future for people who are hung up on playing video games.”