AssemblyAI Speech to Text: Models, Benchmarks, Pricing and When to Use It

AssemblyAI Speech to Text: Models, Benchmarks, Pricing and When to Use It

AssemblyAI speech to text is an ASR platform built around the Universal model family, spanning async transcription, real-time streaming and a bundled Voice Agent API. It suits teams building voice agents that need high transcription accuracy, entity-level precision on names and account numbers, and Hinglish code-switching. It is a poor fit for teams optimising purely for lowest turn-taking latency, or for Indian languages beyond Hindi.

TL;DR

• Best for: Voice agents where getting the words right matters more than answering 300ms sooner — collections, KYC verification, medical intake, anything capturing account numbers or emails.
• Models: Universal-3.5 Pro (async), Universal-3.5 Pro Realtime, Universal-Streaming, Universal-Streaming Multilingual, Universal-2, Sync API.
• Latency: 4.1% WER at 0.44s after end of speech in Max Accuracy mode (Artificial Analysis, July 2026 — independent). Accurate, but slow relative to rivals.
• Price: Universal-Streaming at $0.15/hr, Universal-3.5 Pro Realtime at $0.45/hr, async Universal-3.5 Pro at $0.21/hr (AssemblyAI pricing page, verified July 2026 — check the vendor's pricing page before relying on this).
• Honest limitation: Streaming is billed on session duration, not audio processed. An unterminated WebSocket auto-closes after three hours and you are billed for all three. This is the most common cause of surprise invoices with AssemblyAI, and it is documented in their own FAQ.

What is AssemblyAI speech to text?

AssemblyAI speech to text is a commercial ASR API that trains its own models rather than serving Whisper. The product splits into three transcription surfaces — pre-recorded (async), real-time streaming, and a synchronous single-call API — plus a Speech Understanding layer for sentiment, summarisation and entity detection.

The strategic bet differs from Deepgram's. Where Deepgram optimised for turn-taking latency, AssemblyAI optimised for correctness on the tokens that carry meaning: names, emails, account IDs, medication names. Their Universal-3 Pro release cut missed entities by 27.1% against Universal-2, with the largest gains on emails, organisation names and medical terms (AssemblyAI, 2026).

They also publish an unusually self-critical essay arguing that WER is a broken metric. It includes an admission that customers reported Universal-3 Pro performing worse than Universal-2 — and that on inspection, most of those "errors" were the model correctly transcribing words the human ground truth had missed. Read it before you benchmark anything.

Which AssemblyAI STT models are available?

Six current surfaces. Model naming has churned fast: Universal-3 Pro shipped 3 February 2026, and Universal-3.5 Pro superseded it within months.

Universal-3.5 Pro (async)

The flagship pre-recorded model at $0.21/hr. AssemblyAI's pricing page describes it as working across 18 languages with native code-switching and their most accurate speaker diarization to date.

Use it for call recordings, meeting archives, and any batch pipeline where accuracy beats turnaround.

Universal-3.5 Pro Realtime

The flagship streaming model at $0.45/hr, covering 18 languages with mid-sentence code-switching. Two features distinguish it. Context Carryover lets you pass the agent's last question via agent_context at connection and refresh it after each turn without reconnecting — AssemblyAI reports this cuts WER by 10.2% on voice agent calls. It also keeps a rolling conversation memory on by default.

The practical effect: prime it with "What's your email address?" and it returns [email protected] rather than "user at gmail dot com".

Three modes ship by default — Balanced, Max Accuracy and Min Latency.

Universal-Streaming

English-only streaming at $0.15/hr. AssemblyAI reports 305ms median emission latency. This is the cost-effective production option when your agent is English-only.

Universal-Streaming Multilingual

Same $0.15/hr, covering English, Spanish, German, French, Portuguese and Italian. No Hindi, no Indic coverage at all.

Universal-2

The previous generation at $0.15/hr async, supporting 99 languages, trained on over 12.5 million hours of audio. It remains the widest-coverage option in the lineup, and keyterm prompting is included rather than charged.

Sync API

$0.45/hr. Send up to two minutes of audio, get a finished transcript back in the same response at roughly 134ms p50 — no polling, no WebSocket, no job to manage. Covers 18 languages with word-level timing.

Use it for short-utterance capture: IVR responses, voice search, single-field dictation.

What features does AssemblyAI STT actually give you?

Accuracy and formatting. Keyterm prompting takes up to 1,000 words or phrases on async, 100 on streaming. Natural-language Prompting is separate — pass a plain-English domain description for a reported 5–20% accuracy lift on difficult audio, with detailed prompts cutting name errors by 48.6% and place-name errors by 43.5% (AssemblyAI, 2026). Both are vendor figures.

Languages and code-switching. 99 languages on Universal-2 async. The 18 streaming languages are English, Spanish, French, German, Italian, Portuguese, Arabic, Danish, Dutch, Finnish, Hebrew, Hindi, Japanese, Mandarin, Norwegian, Swedish, Turkish and Vietnamese. Hinglish code-switching is explicitly supported and needs no configuration.

Real-time behaviour. Server-side turn detection with configurable thresholds, interruption handling, and self-correcting speaker labels that revise attribution as more context arrives. Concurrency auto-scales: Pay-As-You-Go starts at 100 new sessions per minute and increases 10% whenever you exceed 70% utilisation, with no published ceiling and no concurrency fee.

Compliance and data residency. HIPAA with BAAs, Medical Mode as a $0.15/hr add-on on both async and streaming, PII redaction on audio and text, content moderation, and zero data retention on Enterprise. Self-hosted deployment and a managed Voice AI Cloud are both offered, plus AWS Marketplace for consolidated billing. No published India data residency.

Developer experience. LLM Gateway exposes GPT-5.5, Claude Opus 4.8, Gemini 3.5 Flash and others behind the same key, so transcript-to-reasoning stays in one billing relationship. Requests targeting universal-3-pro fall back to Universal-2 automatically on failure, surfaced in metadata.warnings.

Benchmarks — how accurate is AssemblyAI really?

Unusually, the independent numbers here are better than the vendor's own headline figures — because AssemblyAI publishes conservative multi-dataset averages rather than one flattering number.

The Artificial Analysis head-to-head is the most useful table in this post, and it does not flatter AssemblyAI on speed:

Source: Artificial Analysis, July 2026 — independent.

Read that carefully. AssemblyAI is roughly 22× slower to first final than Deepgram Flux, at close to half the error rate. Two providers — ElevenLabs and Cartesia — currently beat it on both axes at once. If someone tells you AssemblyAI is the fastest streaming STT, they have not looked at this table.

What these numbers do not tell you. Telephony audio is 8kHz narrowband with compression artefacts and packet loss. Most public benchmarks are 16kHz clean speech. AssemblyAI's own engineers make this argument better than we can: their "Your Ground Truth Is Wrong" workshop found a large share of measured model errors were actually errors in the human reference transcripts.

There is also a code-switching cliff no headline number captures. A model posting 5% WER on monolingual benchmarks routinely posts 15–20% on Hindi-English code-switched calls (Coval, April 2026). If your traffic is Hinglish, benchmark Hinglish.

WER on your own call recordings is the only number that decides anything.

How much does AssemblyAI cost?

All rates from AssemblyAI's pricing page, verified July 2026 — check the vendor's pricing page before relying on this.

The headline rate is not the bill. Add-ons price separately, and the streaming diarization rate is the one that catches people:

Run Universal-Streaming with diarization and you go from $0.15/hr to $0.27/hr — an 80% increase. Multichannel audio bills per channel, so a stereo call-centre recording costs double.

The session-duration trap. Streaming bills for the entire time the WebSocket stays open, including silence and idle time. Sessions never terminated auto-close after three hours and bill for the full three hours regardless of audio sent. Always send an explicit termination message. AssemblyAI names this as the most common cause of unexpected charges in their own documentation.

The free tier is genuinely generous: 185 hours of pre-recorded and 333 hours of streaming.

What is AssemblyAI best used for?

• KYC and account-verification calls for an Indian fintech — Hinglish code-switching plus Context Carryover, where a misheard account number is a failed transaction rather than a cosmetic error.
• Medical intake and patient-access calls for a US telehealth provider — Medical Mode, HIPAA BAAs and PII redaction across both live and recorded audio.
• Post-call QA for an insurance contact centre — async Universal-3.5 Pro at $0.21/hr with diarization at $0.02/hr, then Speech Understanding for sentiment and topic scoring in the same API call.
• Sales-call intelligence for a B2B SaaS team — entity-level accuracy on company names, plus LLM Gateway to score calls without a second vendor relationship.
• Live agent-assist for a support desk — Universal-Streaming at $0.15/hr feeding suggested responses to a human agent, where 400ms of latency is invisible because a person is doing the talking.
• Voice search inside a consumer app — Sync API at ~134ms p50 for short utterances, with no WebSocket lifecycle to manage.

When should you use AssemblyAI?

• Transcription accuracy on entities — names, emails, account numbers, medications — determines whether your agent completes the task or fails it.
• Your agent handles Hindi-English code-switching and you need it in real time, not batch.
• You want transcription, speech understanding and an LLM gateway behind a single key and one invoice.
• Your traffic is spiky and you cannot predict concurrency. Auto-scaling streaming concurrency with no ceiling and no concurrency fee is genuinely uncommon.

When should you not use AssemblyAI?

• Turn-taking latency is your bottleneck. At 0.44s after end of speech, AssemblyAI is materially slower than the alternatives. Use Deepgram — Flux posts 0.02s with model-integrated end-of-turn detection, at the cost of nearly double the WER.
• You need both speed and accuracy. Artificial Analysis currently shows two models beating AssemblyAI on both axes simultaneously. Use ElevenLabs Scribe v2 Realtime (3.6% @ 0.14s, 90+ languages) or Cartesia Ink-2 (3.7% @ 0.09s).
• You need Indian languages beyond Hindi. Tamil, Telugu, Bengali, Marathi, Gujarati, Kannada, Malayalam and Punjabi are absent from the 18 streaming languages entirely. Use Sarvam — Saaras v3 covers 22 Indian languages natively.
• You are transcribing large archives and cost dominates. $0.15/hr is competitive but not cheap. Use Groq at $0.04/hr for whisper-large-v3-turbo, or Mistral Voxtral Mini Transcribe V2 at $0.18/hr with Apache-2.0 weights if you would rather self-host.

How does AssemblyAI compare to the alternatives?

WER and latency from Artificial Analysis, July 2026 (independent). Prices verified July 2026 from vendor pages, except ElevenLabs.

How do you use AssemblyAI on Telenow?

AssemblyAI is live on Telenow as a selectable STT component. In the agent builder, open the Speech-to-Text step and select AssemblyAI from the AssemblyAI STT integration page. Pick the model deliberately: Universal-Streaming at $0.15/hr for English-only agents, Universal-3.5 Pro Realtime at $0.45/hr where Hinglish or entity accuracy matters. Enable diarization only if you need it — at $0.12/hr on streaming it costs more than the base English model.

Telenow manages the WebSocket session lifecycle, so sessions terminate when the call ends rather than idling toward the three-hour auto-close. That removes the exposure that produces AssemblyAI's most common billing surprise.

Your cost line itemises properly. A one-hour Hinglish agent call shows AssemblyAI STT at its own rate, diarization as its own line if enabled, your LLM at its own rate, TTS separately, and carrier minutes separately. No blended per-minute figure concealing a markup. Compare components on the full component pricing breakdown.

Swapping later does not mean rebuilding. If independent benchmarks convince you 0.44s is too slow for your callers, change the STT component to Deepgram from the same dropdown. If you expand from Hindi into Tamil and Telugu, switch to Sarvam. Prompts, tools, call logic and carrier configuration all stay put. Browse the full STT provider comparison or the voice AI platform overview.

Build on AssemblyAI, then run your own Hinglish call recordings through it before committing. That is the only benchmark that counts.

3. JSON-LD

[
  {
    "@context": "https://schema.org",
    "@type": "Article",
    "headline": "AssemblyAI Speech to Text: Models, Benchmarks, Pricing and When to Use It",
    "description": "AssemblyAI speech to text explained: Universal-3.5 Pro, Realtime and Streaming models, verified pricing, independent WER benchmarks and honest limitations.",
    "url": "https://telenow.ai/blog/assemblyai-speech-to-text",
    "datePublished": "2026-07-18",
    "dateModified": "2026-07-18",
    "author": { "@type": "Organization", "name": "Telenow", "url": "https://telenow.ai" },
    "publisher": {
      "@type": "Organization",
      "name": "Telenow",
      "url": "https://telenow.ai",
      "logo": { "@type": "ImageObject", "url": "https://telenow.ai/logo.png" }
    },
    "mainEntityOfPage": { "@type": "WebPage", "@id": "https://telenow.ai/blog/assemblyai-speech-to-text" },
    "about": [
      { "@type": "SoftwareApplication", "name": "AssemblyAI" },
      { "@type": "Thing", "name": "Speech recognition" }
    ]
  },
  {
    "@context": "https://schema.org",
    "@type": "SoftwareApplication",
    "name": "AssemblyAI Speech-to-Text API",
    "applicationCategory": "DeveloperApplication",
    "applicationSubCategory": "Speech Recognition API",
    "operatingSystem": "Cloud, Self-hosted",
    "url": "https://www.assemblyai.com",
    "softwareVersion": "Universal-3.5 Pro, Universal-3.5 Pro Realtime",
    "featureList": [
      "Real-time WebSocket streaming transcription",
      "Context Carryover with turn-by-turn agent context",
      "Mid-sentence code-switching across 18 languages including Hindi",
      "Speaker diarization with self-correcting labels",
      "Medical Mode with HIPAA BAA",
      "PII redaction on audio and transcripts",
      "LLM Gateway",
      "Self-hosted and Voice AI Cloud deployment"
    ],
    "offers": [
      {
        "@type": "Offer",
        "price": "0.15",
        "priceCurrency": "USD",
        "description": "Universal-Streaming, per hour of streaming session, verified July 2026",
        "url": "https://www.assemblyai.com/pricing"
      },
      {
        "@type": "Offer",
        "price": "0.45",
        "priceCurrency": "USD",
        "description": "Universal-3.5 Pro Realtime, per hour of streaming session, verified July 2026",
        "url": "https://www.assemblyai.com/pricing"
      }
    ],
    "provider": { "@type": "Organization", "name": "AssemblyAI, Inc." }
  },
  {
    "@context": "https://schema.org",
    "@type": "FAQPage",
    "mainEntity": [
      {
        "@type": "Question",
        "name": "How much does AssemblyAI speech to text cost per hour?",
        "acceptedAnswer": { "@type": "Answer", "text": "Universal-Streaming costs $0.15 per hour for English realtime transcription. Universal-3.5 Pro Realtime costs $0.45 per hour across 18 languages. Async Universal-3.5 Pro is $0.21 per hour and Universal-2 is $0.15 per hour, verified July 2026." }
      },
      {
        "@type": "Question",
        "name": "Does AssemblyAI support Hindi and Hinglish?",
        "acceptedAnswer": { "@type": "Answer", "text": "Yes. Universal-3.5 Pro Realtime supports Hindi among its 18 streaming languages, with mid-sentence code-switching that handles Hinglish without configuration. Other Indian languages including Tamil, Telugu, Bengali and Marathi are not covered by the streaming models." }
      },
      {
        "@type": "Question",
        "name": "Why is my AssemblyAI streaming bill higher than expected?",
        "acceptedAnswer": { "@type": "Answer", "text": "Streaming bills on total WebSocket session duration, not audio processed. Idle time on an open session costs the same as active audio. Sessions never terminated auto-close after three hours and bill for the full three hours. Always send an explicit session termination message." }
      },
      {
        "@type": "Question",
        "name": "Is AssemblyAI faster than Deepgram?",
        "acceptedAnswer": { "@type": "Answer", "text": "No. Artificial Analysis measured AssemblyAI Universal-3.5 Pro Realtime at 0.44 seconds after end of speech versus Deepgram Flux at 0.02 seconds, in July 2026. AssemblyAI is substantially more accurate, at 4.1% WER against Flux's 7.4%." }
      },
      {
        "@type": "Question",
        "name": "What is Context Carryover in AssemblyAI?",
        "acceptedAnswer": { "@type": "Answer", "text": "Context Carryover lets you pass your agent's previous question into the streaming model via the agent_context parameter and refresh it after each turn without reconnecting. AssemblyAI reports it reduces word error rate by 10.2% on voice agent calls, particularly on short replies and spelled-out identifiers." }
      },
      {
        "@type": "Question",
        "name": "Can AssemblyAI be self-hosted?",
        "acceptedAnswer": { "@type": "Answer", "text": "Yes. AssemblyAI offers self-hosted deployment alongside its managed Voice AI Cloud, plus availability through AWS Marketplace for consolidated billing. Enterprise plans add zero data retention. There is no published India data residency option." }
      },
      {
        "@type": "Question",
        "name": "Does AssemblyAI charge extra for speaker diarization?",
        "acceptedAnswer": { "@type": "Answer", "text": "Yes. Diarization costs $0.02 per hour on async transcription and $0.12 per hour on streaming. The streaming rate nearly doubles the cost of Universal-Streaming, which has a $0.15 per hour base rate. Speaker Identification by name is a separate $0.02 per hour." }
      },
      {
        "@type": "Question",
        "name": "What is the difference between Universal-3.5 Pro and Universal-2?",
        "acceptedAnswer": { "@type": "Answer", "text": "Universal-3.5 Pro is the newer flagship with higher accuracy, native code-switching and better diarization, at $0.21 per hour across 18 languages. Universal-2 costs $0.15 per hour and supports 99 languages, making it the wider-coverage option for less common languages." }
      }
    ]
  }
]

---

4. Internal links used

External links: assemblyai.com/pricing · assemblyai.com/benchmarks · assemblyai.com/docs/streaming/benchmarks · assemblyai.com/changelog · assemblyai.com/blog/word-error-rate-is-broken · artificialanalysis.ai/speech-to-text

---

5. Image concepts

1. Accuracy-versus-latency scatter plot. Five points from the Artificial Analysis July 2026 data — ElevenLabs, Cartesia, AssemblyAI, Deepgram Nova-3 RT, Deepgram Flux — WER on the Y axis, time-after-end-of-speech on the X axis. Bottom-left is best; AssemblyAI sits far right and low. Alt: "AssemblyAI speech to text accuracy versus latency compared with Deepgram, ElevenLabs and Cartesia on Artificial Analysis benchmarks"

2. Context Carryover before and after. Two transcript panels for identical audio: without context, "user at gmail dot com"; with agent_context set to "What's your email address?", [email protected]. Alt: "AssemblyAI speech to text Context Carryover improving email and account number transcription in a voice agent call"

3. Session-duration billing diagram. A timeline showing 12 minutes of actual speech inside a WebSocket left open for three hours, with the billed region shaded across the full three hours. Alt: "AssemblyAI speech to text streaming billed on session duration rather than audio processed, showing idle time charges"