What is ASR and how does automatic speech recognition work?

In 1952, the world’s most advanced speech recognition system could understand exactly 9 words. 

Fast forward by around 75 years, and automatic speech recognition (ASR) transcribes dozens of languages in real-time, powering everything from the voice assistants on your phone to the captions that materialize on live videos.

This guide explores the technology that bridged the gap between the 1952 and modern versions, covering what ASR is, how it converts speech into text, and how ASR is still evolving, even today.

Summary:

• ASR stands for automatic speech recognition, the technology that converts spoken audio into written text.
• The earliest form of ASR emerged in 1952, with deep learning systems matching human benchmarks in the late 2010s.
• Modern ASR uses end-to-end neural networks trained on millions of hours of audio.
• Word error rate (WER) is the standard accuracy metric, but latency, diarization quality, and contextual understanding also play a role in production ASR.
• ElevenAPI gives developers production-grade ASR, independently benchmarked by Artificial Analysis at a 2.2% word error rate, the lowest on its index (July 2026).

What does ASR stand for and what is automatic speech recognition?

Automatic speech recognition, often known solely by its acronym ASR, describes software that listens to human speech and turns it into accurate, machine-readable text. It’s also commonly referred to as speech to text and speech recognition, or even sometimes computer speech recognition.

ASR has become so ubiquitous that you may interact with it every single day without even realizing. Every time you dictate a message, ask a voice assistant a question, read subtitles during a live video, or navigate through an interactive voice response phone system, you’re engaging with ASR.

While speech to text and ASR are often used as synonyms (and are closely related), they’re not exactly the same thing. ASR is the technology that identifies what is said, while STT is the application of that technology as a tool. Voice recognition software layers on top of ASR and identifies who said a certain word, which forms the foundation of speaker diarization capabilities.

These are small distinctions, but worth understanding if you’re looking to integrate ASR/STT/voice recognition systems into your apps or website.

A brief history of automatic speech recognition technology

Automatic speech recognition technology is far older than most people assume, with early iterations dating back to the 1950s. In the over 70 years since its inception, ASR has moved through several important stages which have largely formed the foundation for its historical improvements.

Here are the main eras that ASR technology has moved through:

• 1952 and the first ASR: In 1952, the Automatic Digit Recognizer, known as AUDREY, was built by Bell Laboratories to understand digits from one to nine. The tool was only around 90% accurate and only worked if used by its inventor, making it extremely limited. Despite being a world away from what we now know as modern day capabilities, even simply being able to recognize 1-9 was an impressive feat.
• 1960s to 70s and the growing ASR vocabulary: Across the decades that followed, laboratories around the world, including at IBM, University College London, and NEC in Japan, were able to produce similar models to AUDREY in different contexts. Raj Reddy, a graduate student from India at Stanford, built a vowel recognizer for his PhD project, laying the foundations for continuous speech recognition without the need to pause between each word. DARPA (Defense Advanced Research Projects Agency) funded research in this period which led to Beam Search, a sentence construction and decoding method that was fundamental in recognizing over 1,000 total words by 1976.
• 1980s to early 2010s with statistical advances: In 1987, a graduate student of Raj Reddy’s (now working as a professor) combined Hidden Markov Models with Beam Search, enabling ASRs that didn’t require training on one specific speaker. This breakthrough radically decreased training time for speech recognition systems. Dragon Systems released the first commercially available ASR in 1997, named NaturallySpeaking Preferred. It could recognize 100 words per minute and sold well. 
• The modern era and deep learning: In the 2010s, researchers demonstrated that a single neural network trained end-to-end on audio and transcripts outperformed a purely statistical pipeline. With the arrival of deep neural networks, ASRs could reach near human-level performance between 5% and 10% error rate. At this time, voice-based assistants like Alexa and Siri hit the market. 

Since then, ASR has only become more accurate and more widely used. As of July 2026, Artificial Analysis’ independent research identified ElevenLabs Scribe v2 as having an industry-leading word error rate (WER) of only 2.2%. 

How does ASR work? The basics of speech to text technology

ASR works by capturing an audio sample, converting it into a numerical representation, and then using a trained model to predict the most probable sequence of words that those figures represent. Modern ASR condenses this system into real time, completing the whole end-to-end in a matter of under a second.

There are five main stages to an ASR pipeline:

1. Audio capture and preprocessing: The system records the audio signal, removes background noise, reduces echo, and normalizes volume levels to improve consistency. Depending on the exact model, it may use voice activity detection (VAD) to distinguish speech from silence or background sounds before transcription begins.
2. Feature extraction: The audio is converted into a numerical representation, typically a spectrogram or mel-frequency features, which precisely highlights the frequencies and patterns human speech contains. This step essentially transforms a raw waveform into data that a machine learning model can process effectively.
3. Acoustic modeling: A neural network analyzes the features from the previous step and predicts phonemes or other speech units, learning the relationship between acoustic patterns and spoken language. Modern end-to-end models often perform this step jointly with language understanding, rather than as an entirely separate stage.
4. Language modeling and decoding: The system then weights which word sequences are most likely based on rules like grammar, context, and mathematical probability. The latter two are vital here, as the IPA transcription of two sentences can be similar while their context is completely different. For example, “Recognize Speech” and “Wreck a nice peach” may sound the same, but have completely different meanings. Context will help the system decide which is correct if its precision can’t tell for sure. 
5. Output formatting: After deciding on what the speech contains, the final text is transcribed, with punctuation and capitalization. Additional metadata, like word-level timestamps and speaker labels, will create more detailed transcripts.

Across these main stages, the model transforms incoming audio data into a written transcript.

Traditional hybrid systems vs. end-to-end deep learning ASR

While the pipeline above describes how ASR works, there are actually two technological pathways that you can take for the middle part of that pipeline. 

Legacy systems chain together several components: a lexicon model that encodes how words are pronounced, an acoustic model that maps audio to phonemes, and a language model that predicts likely word sequences. Each of these components needs human expertise to craft, from linguists who hand-craft pronunciation dictionaries to annotators who align every word to its exact position in the audio.

End-to-end deep learning collapses all of these components into a singular neural network. The network maps audio directly onto text, understanding pronunciation, acoustics, and language probability statistics implicitly from millions of hours of paired audio and transcripts.

Let’s break down the differences between traditional and modern approaches to ASR technology. 

How ASR accuracy is measured: Word error rate (WER) benchmarks

For all speech to text and ASR workflows, word error rate (WER) is the main accuracy metric businesses use. It compares a machine transcript generated with ASR against a human-verified version. There are three main errors to account for here, including substitutions, deletions, and insertions.

The formula for WER divides total errors by the number of words in the reference transcript. A WER of 5% would mean that the system was able to transcribe 95% of the text accurately, with most leading models now pushing far below 5% toward perfection.

While WER is a useful benchmark, other factors should also come into consideration when determining how effective an ASR tool is:

• Formatting accuracy: Can the system properly format non-standard strings? For example, is a monetary amount like $1,225 displayed as such, or spelled out as “one thousand two hundred twenty-five dollars”?
• Latency: While accuracy is important, if it takes a few minutes to generate a simple transcript, a tool is rendered unusable. Even if it’s extremely accurate, it needs to balance that with low latency to improve utility.
• Robustness: Even use cases like strong accents or noisy backgrounds shouldn't significantly decrease model accuracy. 
• Diarization quality: Multi-speaker use cases depend on correctly attributing every word to the right speaker. Being able to identify different speakers and correctly tag them contributes to ASR, particularly in heavy multi-speaker industries like legal courtrooms.

For more information, read our comprehensive guide to word error rate.

Key benefits of ASR for modern businesses

The global speech and voice recognition market is expected to grow to over $23.11 billion by 2030. The market's 19.1% CAGR reflects how widespread this technology has become within commercial devices. Every modern mobile phone ships with a dictation keyboard and a voice assistant, most new cars respond to voice commands, and smart speakers or assistants now live in homes around the world.

Interacting with technology via voice is now a default pathway that customers can use across the spectrum. For businesses, ASR provides everything from customer call transcription to voice agent assistance, integrating into processes and driving productivity.

Here are a few of the main benefits of ASR for modern businesses:

• Faster input and documentation: Even over 10 years ago, Stanford published a paper demonstrating that speech recognition technology was almost three times faster than typing on a keyboard, all while maintaining a lower error rate. For the vast majority of people, ASR will enable faster documentation for transcription workflows and voice-first note-taking.
• Lower operational costs: In many use cases that once relied on hours of manual note-taking, ASR technology radically reduces the cost of high-quality transcription. Court cases, contact centers, healthcare clinics, and business meetings can now all have accurate ASR systems that create detailed notes and full diarized transcripts of conversations. 
• Greater accessibility: The World Health Organization predicts that 2.5 billion people will live with some form of hearing loss by 2050. Real-time captions provided by advanced ASR tools can make digital meetings and media accessible to users.
• Searchable voice data: When you automatically transcribe speech with ASR, every single customer-business interaction becomes fully logged. Every sales call, support ticket, prospect call, or meeting becomes a text that is searchable and auditable. Especially in the AI age, being able to surface context for a machine-readable format can help build extensive knowledge bases. Whether for functionality or compliance, this helps to keep information visible. 
• Always-on customer experiences: ASR provides one of the foundational technologies for voice agents, helping to enable automated support use cases that resolve customer calls 24/7/365.

Automatic speech recognition is so predominant in technology due to how many benefits there are of incorporating it and how extensive the use cases that it supports are. Whether you’re in the medical field or are looking to transcribe customer calls for sentiment analysis, ASR is a foundational technology that you’ll incorporate and use. 

Popular applications of ASR across industries

Automatic speech recognition is a central technology in countless workflows and products. It’s reached a level of ubiquity that often means users don’t even think about how it embeds into the tech they use.

Here is a quick rundown of some popular use cases of ASR across the world.

Customer service and contact centers

Contact centers were early adopters of ASR, using it to transcribe calls for quality assurance and compliance. Today, ASR can run in real time to surface suggestions to agents mid-conversation and turn thousands of customer interactions into data that teams can analyze. 

Media and entertainment

Broadcasters and streaming platforms can use ASR to generate subtitles and captions of human conversations at a scale that human transcribers could never match. It enables real-time transcription while also making video and audio libraries completely searchable.

Healthcare

Clinical staff spend a surprising share of their day on documentation and charting. ASR helps win that time back, giving doctors a medical STT platform that they can dictate their notes to in real time.

Virtual meetings

Meeting platforms often provide some form of digital note-taking to transcribe conversations as they happen, meaning that nobody needs to choose between participating and taking notes. Services like Google Meet even send those transcripts with action items highlighted after each meeting, helping to create a searchable index of information. 

Legal and compliance

In legal settings, precision in recording what was said and by whom is a central requirement in court. Law firms will use ASR platforms to transcribe their meetings, hearings, client calls, and court sessions with precise time-stamps for future reference. 

Education

University professors can provide a recorded transcript of their lectures to help students build up a record of the classes they’ve attended. When it comes to understanding and memorizing information, students will have a comprehensive resource available to them to start from.

Where ASR fits in a voice agent pipeline

ASR is a standard part of a range of tech deployments. In voice agent technology, it’s the first of three broader stages that run in a continuous loop.

• Listen (ASR): The agent captures incoming audio streams and converts speech into text in real time. Modern ASR continuously processes audio as it arrives, filtering out background noise, handling interruptions, producing partial transcripts, and identifying when each person is speaking. 
• Think (language model): A large language model interprets the transcript, understanding the user’s intent, retrieving information from connected tools and knowledge bases, maintaining a conversational context, and determining the most appropriate response or action. 
• Speak (text to speech): A text to speech model converts the generated LLM response into natural, expressive audio. Modern TTS systems can then adjust pacing, intonation, emotion, and emphasis when streaming audio back to the caller as it’s generated, rather than having to wait for the full response. 

Conversational latency accumulates across every one of these stages. Streaming ASR begins to emit words as soon as they’re spoken, rather than waiting for an utterance to finish, to reduce latency. ElevenAgents uses this as a standard for real-time voice agents, helping to create a high-efficiency agent experience.

Challenges in ASR and how technology is evolving

While ASR technology has come a long way since its 1952 origins, a range of challenges that may reduce accuracy still remain.

Below are some of the problems that ASR tools still encounter today:

• Accents and dialects: Available training data typically converges into a few languages and accents, making less common accents or languages spoken by fewer people more challenging for ASR tools. Rich and diverse training data sets are the solution here.
• Background noise and overlapping speakers: Environments that have variable volumes or high-volume background noise make individual words from a recording harder to pinpoint. Overlapping speakers may have a similar effect, reducing the accuracy of an ASR transcription.
• Domain-specific vocabulary: Fields with specific jargon or acronyms will need domain-level dictionaries and references to improve accuracy. Medicine and law are two fields where ASR tools will need extra assistance or go through specialized training.
• Privacy and security: In many jurisdictions, voice data is counted as personal data. Enterprises need clear retention policies in place and guardrails to protect the handling of voice data and ensure compliance with wider regulations.

A broader consideration to make when working with ASR technology is latency versus accuracy. Certain use cases need to strike a balance between the two, while domains like medicine would likely prioritize accuracy above all else. 

Get started with ElevenAPI for production-grade ASR integration

ElevenAPI brings production-grade automatic speech recognition to your product through Scribe v2, the ElevenLabs Speech to Text model. Scribe v2 delivers word-level timestamps, speaker diarization, audio event tagging, and the accuracy and reliability needed for real-time applications.

Explore the ElevenLabs Speech to Text page for more information or create a free account to get an API up and running in minutes. 

FAQs about ASR