Adobe has expanded its work with Speechmatics to deliver cloud-grade speech recognition on-device for its Premiere editing software that the company says can run accurate on-device transcription locally and is powerful enough for professional work.

Speechmatics has been Adobe's partner since 2021, when Adobe became the first non-linear editing platform to include speech-to-text (STT) in Premiere. The new feature deepens with a new on-device STT model in Premiere that delivers near-cloud accuracy while keeping all audio local to the device.

When Adobe launched STT for Premiere, large enterprises couldn't always use cloud-based services due to privacy concerns. Speechmatics was one of the few providers with on-device models—a key reason for the partnership.

Five years later, those privacy requirements haven't changed. With the rise of LLMs and data sovereignty concerns, the need for secure deployments has, in fact, increased. What has changed is the performance gap: Speechmatics' new on-device model brings local transcription on par with cloud accuracy with optimisations to run efficiently.

This means, Adobe explained, that studios, agencies, and production companies handling content before it goes public can now work seamlessly from anywhere: on a film set, between client meetings, on a flight—at full accuracy, with no dependency on a connection and no interruption to the work.

The new Speechmatics on-device model has been trained on millions of hours of speech to deliver high accuracy for accented speech, non-native speakers, and noisy environments like field reporting or film sets. As a result, Adobe said that the new on-device model in Premiere:

• Is within 5% relative to cloud accuracy, evaluated across nearly 10 million words of diverse real-world data
• Processes 1 hour of audio in about 55 seconds
• Leads the way against the closest competitor, with a 12-16% improvement against Whisper-powered creative solutions
• Runs on Windows & Mac, making use of the latest AI acceleration techniques to ensure efficient processing across a range of hardware, including broad hardware support for the latest Mac M5, NVIDIA RTX, AMD GPUs and older hardware such as Intel Macs.

Speechmatics on-device joins Speechmatics cloud and Speechmatics on-prem as a purpose-built option for ISVs and OEMs where data residency, offline capability, or predictable costs make local execution the right architectural call. It integrates as a C/C++ library on macOS and Windows.

CAMBRIDGE, U.K. & NOVI, Mi.—Speechmatics has announced that ENCO has chosen its Autonomous Speech Recognition (ASR) technology for ENCO’s new enCaption5 automated captioning and transcription solution. 

This partnership will enable AV professionals, content producers, and broadcasters to create highly accurate transcripts, adding closed or open captions to both live and pre-recorded AV content regardless of the speaker’s demographic, age, gender, accent, dialect or location, the companies said. 

“Speechmatics’ core technology is without a doubt the most accurate of the AI speech-to-text engines available,” said Ken Frommert, president of ENCO. “More than just fantastic technology, however, Speechmatics has also been a great partner. We worked with them previously on enCaption4 and they have been very responsive to our feedback, helping us drive the enCaption platform forward to meet the unique needs of our markets and customers. We’re excited to use their latest technology for enCaption5.”

Speechmatics are global experts in deep learning and speech recognition working with solution and service providers to integrate into their stack regardless of their industry or use case. Speechmatics’ technology is trained on huge amounts of unlabeled data without the need for human intervention, delivering a far more comprehensive understanding of all voices and dramatically reducing both AI Bias and errors in speech recognition.

The fifth generation enCaption5 harnesses Speechmatics’ technology in a cloud-native architecture – which can also be deployed in hybrid or on-premises workflows – and an intuitive user interface optimized for professional AV deployments, broadcast environments, and conversational AI applications. 

In addition, enCaption5 offers such features as integration with external vocabulary sources; AV signal input and output; media indexing and search; confidence scoring; and the ability to review and edit transcripts – with manual refinements automatically updating the system’s word filters to improve future accuracy. enCaption5 also offers an open, advanced API for third-party developers, and is backed by world-class, U.S.-based support.

ENCO will be leveraging the entire suite of Speechmatics’ market-leading features for enCaption5. These include:

• Coverage of 34 languages
• Advanced Punctuation
• Custom Dictionary (the ability to add context specific words to the Speechmatics dictionary in real-time) and
• Entity Formatting (consistently and more accurately interpret how entities such as numbers, currencies, and addresses should appear in written form)

For more information about ENCO and enCaption5 visit www.enco.com/encaption. For more information about Speechmatics visit www.speechmatics.com.

CAMBRIDGE, ENGLAND—Speechmatics has incorporated its automatic speech recognition (ASR) technology with Tedial’s live sport solution SMARTLIVE.

The technology partnership between the ASR company and the MAM specialist allows operators to search for comments made during a sporting match or live event, automatically create file locators and manually generate speech-to-text from a commentary track.

This eliminates the need for production teams to scale up, freeing them to create personalized stories while maximizing efficiency, output and profitability for broadcasters.

“We’re delighted to be partnered with Tedial in what is an exciting time for both companies,” said Ian Firth, VP products at Speechmatics. “Given the incredibly competitive nature of the marketplace in which they operate, both Tedial and its customers require the most accurate and efficient solutions, and we see this partnership as further testament to the quality of our product.”

Jérôme Wauthoz, VP products at Tedial, added: “The integration of Speechmatics’ technology reaffirms our commitment to deliver market-leading, cutting-edge MAM solutions. It’s a natural fit and we’re thrilled to welcome them to the fold.

“The beauty of technology like this is that this initial integration enables us to continue to benefit from further improvements in years to come. This is what machine learning is all about, and we look forward to continuing to work closely with Speechmatics to further enhance our solutions for an enriched customer experience in the weeks, months and years to come.”