Back home

Baseten Voice Agent Stack / Baseten Realtime Voice

sitemap+x+web 0.92
Visit source

Intro

Baseten Voice Agent Stack and Baseten Realtime Voice are the voice-agent infrastructure and reference implementation Baseten formally launched on 16 November 2026. @baseten posted the introduction at 2026-11-16T19:15:21Z (03:15:21 Beijing time on 17 November 2026). The post begins: “Introducing the Baseten Voice Agent Stack: optimized infrastructure to run and customize voice agents.” It says every model and the orchestrator are hosted for low end-to-end latency, each component scales on its own, and you can swap or post-train any model. The rest of the post also mentions that the company built Baseten Realtime.

The official blog at baseten.co/blog/introducing-the-baseten-voice-agent-stack is dated November 16, 2026, titled “Introducing the Baseten Voice Agent Stack: own and optimize voice agents end-to-end,” and attributed to Ian Carrasco and Tianshu Cheng. It says: “Today we’re launching the Baseten Voice Agent Stack and Baseten Realtime Voice.” The Voice Agent Stack is infrastructure for production voice agents on Baseten Dedicated Inference: it places the orchestrator and speech models in one cloud region over a tight in-cluster network, and supplies inference-runtime optimization, inter-model networking, traffic autoscaling, cross-cloud failover, and model post-training, with the option to pick or swap models at each stage of the cascade, including a mix of third-party APIs. The post also discusses post-training STT on vertical jargon, post-training LLMs for real calling scenarios, and supervised fine-tuning of TTS on a company’s own voice data.

Baseten Realtime Voice is a reference voice-agent API built on that stack. The company says it runs a cascade of Baseten-optimized open-weight speech models (the measured setup in the footnotes is STT: Qwen3-ASR, LLM: Gemma-4-31B, TTS: Qwen3-TTS, plus a GLM-5.3 Fast delegate on Baseten Model APIs) with a custom Pipecat-based orchestrator, and can optionally keep a fast, co-located talker on the conversation while a stronger thinker handles reasoning and tool calls. The reported figures include 93.01% on the Artificial Analysis split of Full-Duplex-Bench, the highest among listed open-source full-duplex systems; a self-reported 73.5% pass^1 on Sierra’s τ²-Bench Voice harness (English split, including 93.9% on telecom), which the post says was not measured by Artificial Analysis and is not directly comparable to that leaderboard, versus a 31.2% cascaded baseline; and $0.012 per audio minute at 32 fully utilized concurrent streams, or an estimated $0.008 per call-minute at 56 concurrent sessions under the utilization assumptions shown, including model-API delegation. On latency, the company says median turn latency stays around one second from 1 to 56 concurrent sessions with context up to 16K tokens, while median time to first audio rose from 0.94 to 1.14 seconds. The API is described as matching the core OpenAI Realtime event protocol, with audio-in/audio-out and text-in/audio-out, plus keyword boosting, configurable acknowledgments, and voice cloning. Clients must accept a full WebSocket URL; the post also says the OpenAI Python SDK appends /realtime to its base URL and therefore cannot connect directly, and that browser clients need a backend to hold the API key. Both offerings are in private preview, with a waitlist.

Key points:

Advertisement

Explain

Why it matters

These are new product names Baseten confirmed on 16 November 2026 with an official account post and a blog post, not a rename of the existing inference platform or of the already stored term Baseten Grid. The introduction post went out at 19:15:21Z (03:15:21 Beijing time on 17 November), inside this 00:32–04:32 scan window, and the same blog URL also appeared as a new sitemap entry, so the names are filed separately.

This page only restates the official introduction post and the official blog. The 93% Full-Duplex-Bench score, the 73.5% τ²-Bench Voice score, and the roughly $0.01-per-minute cost are company statements on the reference setup; the τ² result is explicitly self-reported and not measured by Artificial Analysis. This page does not independently reproduce those figures.

Limits

Both offerings are in private preview — do not write them as generally available to all customers. This page is not a guarantee of latency, cost, or benchmark scores, and it does not describe the stack as a replacement for Baseten’s general inference platform or for the already stored term Baseten Grid. Limits such as the OpenAI Python SDK being unable to connect directly, and browsers needing a backend to hold the API key, should be taken from official docs.

Usage

  1. Read the official introduction at https://x.com/baseten/status/2122408198531363053 and cross-check the official blog at https://www.baseten.co/blog/introducing-the-baseten-voice-agent-stack ;
  2. Follow the official instructions: both offerings are in private preview, so join the official waitlist; protocol details are in the Baseten Realtime Voice docs;
  3. When citing the 93% Full-Duplex-Bench score, the 73.5% τ²-Bench Voice score, or the roughly $0.01-per-audio-minute cost, attribute them to the 16 November 2026 official blog, note that the τ² result is self-reported, and recheck the upstream source;
  4. Do not describe Baseten Voice Agent Stack / Baseten Realtime Voice as the existing Baseten inference platform, or as the already stored term Baseten Grid; this page only restates official materials — recheck the upstream sources before citing them externally.