Egyptian and Nigerian English Now Featured on the Open ASR Leaderboard

Introduction to Private Data in ASR

In the rapidly evolving field of Automatic Speech Recognition (ASR), the accuracy and reliability of evaluations hinge crucially on the quality and diversity of the datasets used. A few months back, the introduction of private data to the Open ASR Leaderboard made waves in the community. This included 11 splits from Appen and DataOceanAI, encompassing various accents and styles of speech. The focus was on both scripted and conversational speech from five unique accents, an effort that was warmly welcomed by users and researchers alike.

The Importance of Held-Out Sets

The community response to our previous updates was overwhelmingly positive, with many individuals emphasizing the significance of held-out sets. These specially curated datasets serve as vital components for executing trustworthy evaluations in the realm of ASR. By ensuring that models are assessed against data that they have never encountered during training, we can genuinely evaluate their efficacy and reliability. Subsequently, we augmented our offerings with an additional private split focused on Indian-accented English, which further enhanced our dataset diversity.

Exciting New Collaborations: Egyptian and Nigerian Accents

We are thrilled to announce our latest collaboration with Liva AI, a forward-thinking entity in the field, to further expand our private data offerings. This partnership introduces two new accents into the fold: Egyptian 🇪🇬 and Nigerian 🇳🇬.

Detailed Accent Data

As part of this initiative, we have prepared specific datasets aimed at capturing the essence of these accents in a conversational context. Here’s a quick look at the details:

These new datasets are crucial for broadening representation within ASR evaluations, ensuring that accents and languages from various cultural backgrounds are adequately represented.

Enhanced Accent Breakdown in Open ASR Leaderboard

With the integration of these new accents, we can offer a more nuanced breakdown of accents based on Kachru’s Three Circles model. This framework is instrumental in categorizing languages and accents into specific groups, helping us understand the diversity of speech patterns globally. Moving forward, the Open ASR Leaderboard will include:

• US vs. Non-US: This split highlights accents from the United States alongside those from other nations, including the UK, Australia, Canada, India, Nigeria, and Egypt.

• Inner Circle vs. Outer & Expanding Circles: The Inner Circle includes accents from the US, UK, Australia, and Canada, while the Outer and Expanding Circles will comprise accents from India, Nigeria, and Egypt. As more accents are added, we will continue to refine this classification further.

Overview of Liva AI

Liva AI stands out as an innovative player in the tech industry, especially in the realm of ASR and multimodal interactions. By collecting real, high-context data through consumer products, Liva AI is uniquely positioned to address the existing model gaps. This is achieved by leveraging natural user behaviors that emerge from genuine product usage rather than relying solely on artificial or task-based interactions. This focus not only enriches the datasets but also ensures that the interactions captured are realistic and representative of true human communication.

Contributing to ASR Trust and Performance

The ongoing efforts to enhance the representation of diverse accents and languages are a testament to the commitment of various stakeholders in pushing the envelope of ASR performance. The introduction of new private data, along with collaboration with pioneering data providers like Liva AI, reflects our dedication to minimalizing “benchmaxxing” and fostering a more inclusive evaluation environment. As we continue this journey, we are excited to witness the growth and evolution of ASR technology, enabling it to cater to a broader, more diverse audience.