Adobe has expanded its work with Speechmatics to deliver cloud-grade speech recognition on-device for its Premiere editing software that the company says can run accurate on-device transcription locally and is powerful enough for professional work.

Speechmatics has been Adobe's partner since 2021, when Adobe became the first non-linear editing platform to include speech-to-text (STT) in Premiere. The new feature deepens with a new on-device STT model in Premiere that delivers near-cloud accuracy while keeping all audio local to the device.

When Adobe launched STT for Premiere, large enterprises couldn't always use cloud-based services due to privacy concerns. Speechmatics was one of the few providers with on-device models—a key reason for the partnership.

Five years later, those privacy requirements haven't changed. With the rise of LLMs and data sovereignty concerns, the need for secure deployments has, in fact, increased. What has changed is the performance gap: Speechmatics' new on-device model brings local transcription on par with cloud accuracy with optimisations to run efficiently.

This means, Adobe explained, that studios, agencies, and production companies handling content before it goes public can now work seamlessly from anywhere: on a film set, between client meetings, on a flight—at full accuracy, with no dependency on a connection and no interruption to the work.

The new Speechmatics on-device model has been trained on millions of hours of speech to deliver high accuracy for accented speech, non-native speakers, and noisy environments like field reporting or film sets. As a result, Adobe said that the new on-device model in Premiere:

• Is within 5% relative to cloud accuracy, evaluated across nearly 10 million words of diverse real-world data
• Processes 1 hour of audio in about 55 seconds
• Leads the way against the closest competitor, with a 12-16% improvement against Whisper-powered creative solutions
• Runs on Windows & Mac, making use of the latest AI acceleration techniques to ensure efficient processing across a range of hardware, including broad hardware support for the latest Mac M5, NVIDIA RTX, AMD GPUs and older hardware such as Intel Macs.

Speechmatics on-device joins Speechmatics cloud and Speechmatics on-prem as a purpose-built option for ISVs and OEMs where data residency, offline capability, or predictable costs make local execution the right architectural call. It integrates as a C/C++ library on macOS and Windows.

SANTA CLARA, Calif.—Agora has launched a Real-Time Transcription solution. The offering provides developers with the fast, accurate, and affordable automated transcription, and subtitling capabilities that can be easily integrated into any app or service.

"The launch of our new Real-Time Transcription solution provides instant audio transcription to developers and brands, allowing them to boost accessibility and discoverability for their end users,” said Tony Zhao, CEO and co-founder of Agora. “This powerful cloud-based technology is designed to seamlessly integrate with any app or service.”

Key features include:

• Live Transcription: Transcribe real-time audio and video to include captions to improve audience reach and experience.
• Speaker Labeling: Easily label who said what with a speaker dropdown to ensure accurate speaker information with the transcription.
• Searchable Transcripts: Search for words, phrases, and themes across all transcripts. Leverage transcript as input to ChatGPT or other generative AI solutions.
• Recording Captioning: Play back audio or video recordings with closed captions (CC).
• Channel-Based Transcription: Transcribe with multiple active hosts in a channel and deliver a transcript to all participants while only paying for the channel duration—not the number of users or speakers.

While closed captioning makes any audio or video content more accessible, the real-time nature of Agora’s solution extends that accessibility to live audio and video, the company reported. 

The transcription service provides instant subtitles for any live meeting, conference, event, cast, or video stream—while enabling other essential services such as content moderation. The solution scales from one-to-one video calls to many-to-many streaming, and can support transcription with up to 100 people in an audio chat group. In addition to increasing accessibility for people with hearing impairments or difficulty understanding a spoken language, Real-Time Transcription improves the discoverability of audio and video content without the need for replay.

"Any industry that is reaching their audience or customers through audio or video will benefit from Real-Time Transcription technology,” said Zhao. "We're excited to see the impact our solution will have on improving engagement with audiences everywhere."

Agora’s Real-Time Transcription solution is currently in public beta, with general availability planned for Q3 2023.

For more information about Real-Time Transcription, visit: https://www.agora.io/en/products/real-time-transcription/.

MELBOURNE, Australia—Ai-Media Technologies Limited, a global provider of technology-driven captioning, transcription and  translation services, has announced that it has renewed its exclusive  agreement with Australian broadcaster Seven Network (Seven) for an additional five year term. 

As part of the new five year contract, Seven will extend its use of Ai-Media’s offerings from traditional premium  Services to technology products iCap Encode, Lexi and Smart Lexi, and an  enhanced use of SubSilo 

In addition, Seven will be the first free-to-air Australian broadcaster to transition to Ai Media’s iCap network, adopting the full suite of technology products, targeting a  Q3 FY23 roll-out, the companies said. 

“We are extremely pleased to have signed this renewal which showcases the strength of our  new iCap product offering as we continue to deliver the best possible captioning quality with  the highest published uptime in the industry,” explained Ai-Media co-founder and CEO Tony Abrahams. “Since 2016 Ai-Media has been the exclusive  provider of captioning services to Seven across all markets. This renewal includes a  significant technology uplift delivering improvements in scalability and cost for Seven.”

“Through the pioneering use of our SubSilo product, team members at Seven can now use embedded captioning to search extended video archives, he added. “The full suite of Ai-Media’s products and services will be available to Seven once we  complete the transition to the iCap network, which is expected to occur in Q3 FY23.” 

“The agreement further accelerates our technology-driven growth through recurring revenue and speeds up the transition of our top customers to the iCap network and SaaS products Lexi  and Smart Lexi,” Abrahams also explained. “This will also deliver viewers much more choice of captioned content, helping  to deliver further on our vision of making the world’s content accessible for everyone.” 

An element of the Seven renewal was Seven’s organization-wide implementation of SubSilo,  which uses Ai-Media’s captioning to power automated search across Seven’s extensive media  archive. 

“We have partnered with Ai-Media since 2016 and have ensured that our captioning is the  best possible quality for all of our viewers who rely on this important service,” said Seven Network director of broadcast operations. “Our focus is on  tight technical integration between our systems, Ai-Media and our playout operations. This  renewal, further enhances the ability to deploy accurate automated captions across more  content as well as improved media search for Seven’s internal teams by utilizing technology from Ai-Media’s recent EEG acquisition.”

Ai-Media also reported that under the five-year renewal agreement, Ai-Media will deliver all captioning across the Seven network at lower costs to Seven, driven by the rollout of the iCap network through the iCap  Encode products and the automated Lexi and Smart Lexi SaaS products. 

In addition, Seven will purchase  hardware from Ai-Media and will pay a monthly captioning fee (for an agreed amount of  captioning) and a monthly equipment support fee. Seven will continue to benefit from the  integrated SubSilo product which fully indexes archived content across the network with  “search by caption” functionality available 24 hours after broadcast, the companies said. 

FREMONT, Calif.—Digital Nirvana has unveiled an upgrade for its Trance self-service SaaS application, a tool that supports transcription, closed captioning and subtitle generation and automatic translations to localize content within media companies’ workflows.

Enhancements offered in Trance 4.0 include elaborate account management, modular rights for users and advance pro-captioning window for enterprises and individual users, Digital Nirvana said.

With its new modular application approach, Trance 4.0 can be used as a combined or individual tool for transcription, captioning, text localization or conformance of existing captions, it said. 

The latest version’s new and improved transcription window can be used as a standalone app rather than being part of a three-step transcription, captioning and subtitling workflow that only outputs after each step is completed.

The company has made major changes to the pro-captioning window to make it more user-friendly and efficient. Machine-generated timecodes can be adjusted using spectrogram and manual inputs and the window now enables user to upload media, generate automatic captions and display using desired presets defined by the user or organization, it said.

An extension of the pro-captioning window, text localization can now be used as a Trance module to focus on localizing captions. Users can now import existing captions and quickly use the machine translation feature to generate highly accurate localized text specifically designed for subtitling purposes, the company said.

The caption-conformance app allows users to import text with new presets to conform with various publishing platforms. It automatically reviews text and highlights any nonconformities with selected guidelines, it said.

Digital Nirvana also has upgraded Trance’s management functions. The dashboard now allows better real-time visibility into which users are online and the progress of jobs.

The new version offers enhanced user control over processing jobs with the addition of the ability to choose from available jobs, import a transcript and create captions with existing or new presets, it said.

Trance 4.0 now allows enterprises to assign work items to internal staff and to share work items with Digital Nirvana’s service team to receive fully qualified, human-curated output using the same application, it said.

More information is available on the company’s website.

SAN FRANCISCO—Digital Anarchy has launched Transcriptive.com, a speech-to-text web service that uses AI and natural speech processing to perform transcriptions.

Transcriptive.com is part of Digital Anarchy’s family of intelligent transcript, search and collaboration tools for video production. Through the website, users can upload media and transcribe or sync existing transcripts; users can share transcripts with others for collaboration; users are able to export captions and subtitles for use in any video editing application, as well as plain and rich text files for paper edits; words can be assigned a timecode for user to jump to the matching time in a video; there is a simplified text editor; and speaker management features allow for the addition or removal of speakers from the transcripts and assigning of names.

Transcriptive.com’s AI technology enables linguistic support for more than 70 unique languages and natural language processing. The web service is rooted with Digital Anarchy’s Transcriptive plug-in with Adobe Premiere Pro integration.

Digital Anarchy is providing multiple payment plans for Transcriptive.com, including a monthly ($19/month) or annual ($160/year) subscriptions that allow for transcribing, editing, exporting captions or text files and the ability to share transcripts. There is also a free transcribe-only plan for Transcriptive-Premiere Pro users and a free, entry-level review-only plan for users who need to look at a file sent to them by a Transcriptive-Premiere Pro user.

An updated Transcriptive plug-in, v2.5, has also been announced by Digital Anarchy.

For more information, visit www.digitalanarchy.com.  

LOS ANGELES—axle ai, a software management company, has announced the launch of the ascribe.ai AI-driven video transcription service that is able to work with Adobe Premiere Pro.

Ascribe.ai is available through the axle ai website and the Adobe Exchange Application Marketplace at no charge. When registering, users receive a free hour of transcription of video content; after that, users can sign up for subscriptions that start at $4.95 per month for 2.5 hours, with lower prices in larger quantities. The service can provide transcriptions in English, Spanish, French, Italian, Portugese, Russian and Korean.

The service creates an audio-only file that is transcribed in faster than real-time, featuring the transcript and relevant markers that are posted to the timeline in Adobe Premiere Pro. Users can also export an industry-standard SRT or VTT file with the transcript.

The same axle Speech services are used in ascribe.ai as in the company’s 2019 media management system for small and midsize video teams, but ascribe.ai is free and does not require a centralized database or shared network storage. The system can connect with Speech’s cloud service via nearly any connection, per axle.

The ascribe.ai system is available immediately from axle’s website, and will soon be available through the Adobe Exchange Application Marketplace.

Axle ai debuted the service at the Adobe MAX Conference.