GPT-Live Explained: Inside OpenAI's Full-Duplex Voice Architecture

On July 8, 2026, OpenAI quietly killed the walkie-talkie feel of talking to ChatGPT [1]. The company replaced Advanced Voice Mode with GPT-Live, a new generation of voice models built on a genuinely different architecture: full-duplex, meaning the model can listen and speak at the same time instead of waiting its turn [1][2]. Two variants shipped simultaneously, GPT-Live-1 and the smaller GPT-Live-1 mini, rolling out to ChatGPT on iOS, Android, and the web the same day [1][3].

That one architectural change, from turn-based to full-duplex, sounds like a small usability tweak. It is not. It requires rethinking what a "response" even means for a voice model, replacing the familiar prompt-in, completion-out loop with something closer to a real-time control system that has to decide, many times a second, whether to talk, stay quiet, or get out of the way [3][4]. It also introduces a genuinely new production pattern worth understanding on its own terms: GPT-Live handles the live audio stream itself, and quietly hands anything that needs real thinking, a web search, a multi-step calculation, an agentic task, to GPT-5.5 running in the background, stitching the answer back into the conversation once it lands [1][5].

This post is a full architecture walkthrough of GPT-Live: the pipeline problem it actually solves, the decision-loop mechanism that replaces silence-detection turn-taking, the small-model/big-model delegation pattern and how it compares to speculative decoding, mixture-of-experts routing, and other dual-model systems already in production, how GPT-Live stacks up against Gemini Live, Grok Voice, and Alexa+, and what a real production voice-agent build looks like against this kind of model, interruption handling, latency budgets, and failure modes included.

The Problem With Turn-Based Voice AI

Almost every voice assistant shipped before this generation, including OpenAI's own original Advanced Voice Mode era and the ChatGPT voice feature that preceded it, was built on the same three-stage pipeline: speech-to-text transcribes what you said, a language model reasons over the transcript and writes a reply, and text-to-speech synthesizes that reply back into audio [6][2]. It is a clean, modular design, and it is also the reason talking to most voice assistants still feels like using a two-way radio instead of having a conversation.

The Three-Stage Pipeline, and Where the Delay Actually Comes From

Each stage boundary in that pipeline adds latency, and the delays compound rather than overlap. The speech-to-text stage typically cannot finalize a transcript until it has enough confidence that you have stopped talking, which means it is already waiting on silence before it hands anything to the language model. The language model then has to generate a full turn, or at least enough of one to start streaming, before text-to-speech can begin synthesizing audio, and many production deployments buffer more than one sentence at a time to keep prosody sounding natural, which adds still more delay before the user hears anything. None of these stages can start meaningfully before the one before it finishes, so the pipeline's total latency is closer to the sum of its stages than the max of any single one.

The VAD Problem

The mechanism that decides when you have finished talking is voice activity detection, VAD for short, and it is the single biggest source of the awkwardness in turn-based voice AI. A VAD system watches for a gap in your speech, some threshold of silence, typically a few hundred milliseconds to a second or more, and treats that gap as your turn ending. That threshold is a genuine tradeoff with no clean answer. Set it too short and the system cuts you off mid-sentence the moment you pause to think or breathe. Set it too long and every exchange gets padded with dead air while the system waits to be sure you are really done, which is the single biggest complaint users have about older voice assistants and one of the specific failure modes call-center and voice-agent teams spend enormous engineering effort tuning around. Worse, VAD has no actual model of the conversation. It cannot tell the difference between "I paused because I'm thinking of the next word" and "I'm finished, please respond," because it is reasoning over silence, not meaning.

Full-duplex speech-to-speech models exist specifically to remove this bottleneck, not by tuning the VAD threshold better, but by getting rid of the discrete pipeline stages and the silence-based turn boundary entirely. That reframing is the technical core of what OpenAI shipped in GPT-Live, and it is worth understanding mechanically before getting to the delegation pattern that makes it practical.

What OpenAI Actually Shipped on July 8, 2026

GPT-Live is not one model, it is a small model family plus an architecture. GPT-Live-1 is the larger, more capable variant, and GPT-Live-1 mini is a smaller, faster variant tuned to run as the free-tier default [1][3]. GPT-Live-1 mini replaced Advanced Voice Mode as the default voice experience for every ChatGPT user, free tier included, and GPT-Live-1 itself is available to Go, Plus, and Pro subscribers, who also get access to deeper reasoning tiers not available on the free plan [1][3][6].

Both models are full-duplex: OpenAI's own description is that the model can "speak and listen at the same time," enabling natural interruption, barge-in, and features like live translation that are structurally difficult to build on a turn-based pipeline, since translating live speech requires processing incoming audio and producing outgoing audio concurrently rather than waiting for a full sentence to complete first [1][2]. The rollout is global, live in the ChatGPT apps for iOS and Android as well as chatgpt.com, tap the Voice button and it is the model you are talking to by default [3].

Independent developer Simon Willison, who had preview access before the public launch, documented one useful reminder that a full-duplex model is still a model with real failure modes underneath the smoother interaction layer: an early preview build would occasionally interrupt with inappropriate laughter mid-conversation, a bug OpenAI fixed before the wider rollout [7]. It shipped with real, disclosed limits worth naming plainly rather than skipping past. At launch GPT-Live is not available in ChatGPT for Business, Enterprise, or Education workspaces, Temporary Chats, the desktop app, Codex, or custom GPTs, and there is no developer API yet, OpenAI has said one is coming but has not published a date or pricing [6][8]. Video and screen-sharing integration are also absent at launch, described as "in development" [3]. If you are building a production voice product today, the honest starting point is the existing Realtime API, covered later in this post, with an eye toward GPT-Live's eventual API surface rather than a dependency on it shipping on any particular date.

OpenAI's own internal preference testing, five-to-ten-minute matched conversations rated on turn-taking, interruption handling, conversational flow, and overall naturalness, found GPT-Live-1 preferred over Advanced Voice Mode 75.7 percent of the time, and GPT-Live-1 mini preferred 69.2 percent of the time under the same conditions [6][9]. Those are OpenAI's own numbers from its own evaluation, worth treating the way any self-reported benchmark should be treated, a real signal, not an independently audited one. More interesting for a technical read is what happened to the model's task-completion scores once the delegation pattern kicked in: GPT-Live-1 at its highest reasoning tier scored 84.2 percent on GPQA, a graduate-level scientific reasoning benchmark, up from Advanced Voice Mode's 45.3 percent, and on BrowseComp, an agentic web-search benchmark, GPT-Live-1's high-reasoning tier reached 75.2 percent against Advanced Voice Mode's 0.7 percent [6][9]. That is not a voice-quality improvement. It is the direct, measurable effect of handing hard reasoning to GPT-5.5 instead of asking a real-time conversational model to do agentic web research itself, and it is the clearest evidence that the delegation architecture, not just the full-duplex audio path, is doing real work.

GPT-Live also ships with four selectable reasoning tiers that trade latency for depth, worth its own table before going further, since the tiers are central to how the delegation pattern actually behaves in practice.

Free-tier users get Instant only. Go, Plus, and Pro unlock Medium and High, and Pro adds an extended X-High tier for the hardest agentic and research tasks [6][4]. Picking a deeper tier does not make the conversation itself feel slower, which is a detail worth sitting with, because it is only true because of the architecture covered in the next two sections.

Full-Duplex, Mechanically: From Discrete Turns to a Continuous Stream

"Full-duplex" is a term borrowed from telecommunications, where it describes a channel that can carry signal in both directions simultaneously, the way a phone call works, as opposed to half-duplex, where only one party can transmit at a time, the way a walkie-talkie or push-to-talk radio works. Every turn-based voice assistant, including the pipeline described above, is architecturally half-duplex even when the underlying network connection is bidirectional, because the system's internal state machine only ever processes one direction of the conversation at a time: it listens, then it thinks, then it speaks, and it cannot meaningfully do more than one of those at once.

A native full-duplex speech-to-speech model removes that constraint at the model level, not just the network level. Instead of discrete request-response turns, the model treats both the user's audio and its own audio as continuous, always-on streams it is jointly modeling. OpenAI describes this directly: instead of processing a sequence of separate messages, GPT-Live continuously processes input while generating output [4][3]. There is no moment where the model is purely "listening" with reasoning paused, and no moment where it is purely "speaking" with listening paused. Both are always happening.

This is not a new idea invented for GPT-Live. Kyutai's Moshi, released as an open research model in 2024, was one of the first publicly documented systems to demonstrate this at meaningful scale: a multi-stream architecture that models the user's audio and the model's own audio as two parallel token streams processed by the same transformer, without ever explicitly modeling discrete speaker turns [10]. Moshi's own paper reports a theoretical latency floor of 160 milliseconds and a practical end-to-end latency around 200 milliseconds running on a single L4 GPU, low enough that the model can genuinely react mid-utterance rather than waiting for a turn boundary that, architecturally, no longer exists [10]. Alibaba's Qwen2.5-Omni took a related but distinct approach the same year with its Thinker-Talker design, where a Thinker component handles the actual language reasoning across text, audio, and video, and a separate Talker component, a dual-track autoregressive decoder, converts the Thinker's hidden representations directly into streaming speech tokens, letting speech generation begin before the underlying reasoning has even fully resolved into text [11].

GPT-Live's specific implementation details are not public, OpenAI has not released a technical paper the way Kyutai or the Qwen team did, but the description OpenAI has given, a model that jointly and continuously processes audio in and audio out rather than discrete turns, places it in the same family of native speech-to-speech architectures rather than the older pipeline design [1][4]. The practical difference this makes for a user is immediate: the model can register that you have started talking again mid-response and begin adjusting before you have finished a full sentence, the same way a human listener does, instead of waiting for a VAD system to declare your turn over.

The Decision Loop: Voice as a Control Problem, Not a Generation Problem

Standard autoregressive text generation is a straightforward inference pattern. The model receives a prompt, then generates output tokens one at a time, each conditioned on everything before it, until it decides to stop or hits a length limit. There is no ongoing environment to react to mid-generation. The prompt does not change while the model is writing its answer.

A full-duplex voice model cannot work that way, because the "prompt" never stops arriving. The user's audio is a continuous stream that keeps updating for as long as the conversation runs, and the model has to keep producing a coherent response to a target that is itself changing in real time. OpenAI's own framing of this is precise and worth quoting directly: the model can make interaction decisions many times per second, whether to speak, continue listening, pause, interrupt, or invoke a tool [4][3]. Every one of those verbs is a distinct action in a small, fixed action space, and the model has to select among them continuously based on the ongoing audio stream, not once per turn.

That is a genuinely different inference pattern from anything a standard chat model does, and it is closer to a real-time control-loop or reinforcement-learning-style policy problem than to open-ended text generation. At each timestep, the system effectively has:

• State: the running audio context, both what the user has said and what the model itself has been saying, plus whatever background task state exists if a delegation to GPT-5.5 is in flight.
• Action space: speak (continue or start a response), listen (stay silent and keep absorbing context), pause (a deliberate short silence, distinct from listening, used for natural conversational rhythm), acknowledge (a backchannel like "mhmm" or "got it" that signals attention without taking the floor), interrupt (break into the user's speech, or accept being broken into), and invoke a tool or delegate to the background reasoning model.
• Policy: whatever internal mechanism the model uses to select among those actions given the current state, evaluated many times per second rather than once per generated token or once per turn.

A simplified illustration of that decision loop, not GPT-Live's actual internals, which OpenAI has not published, but a structurally honest sketch of the kind of per-timestep policy evaluation the "many times per second" framing describes:

python
/code
# Simplified illustration of the "many times per second" decision loop
# framing OpenAI uses to describe GPT-Live's full-duplex architecture.
# Not GPT-Live's real internals (unpublished) -- a structurally honest
# sketch of a per-timestep policy over a small, fixed action space [1][4].
import random
from dataclasses import dataclass
from enum import Enum

class Action(Enum):
    SPEAK = "speak"
    LISTEN = "listen"
    PAUSE = "pause"
    ACKNOWLEDGE = "acknowledge"
    INTERRUPT = "interrupt"
    DELEGATE = "delegate_to_background_model"

@dataclass
class StreamState:
    user_is_speaking: bool
    user_finished_thought: float   # model's running confidence, 0-1
    pending_delegation: bool
    question_complexity: float     # 0-1, estimated from context so far

def policy_step(state: StreamState) -> Action:
    """One timestep of the interaction policy. In a real full-duplex model
    this runs continuously against streaming audio embeddings; here it is
    a rule-based stand-in to show the shape of the decision, not the
    learned mechanism itself."""
    if state.pending_delegation:
        return Action.ACKNOWLEDGE
    if state.user_is_speaking and state.user_finished_thought < 0.4:
        return Action.LISTEN
    if state.user_is_speaking and state.user_finished_thought >= 0.85:
        return Action.SPEAK
    if not state.user_is_speaking and state.question_complexity > 0.7:
        return Action.DELEGATE
    if not state.user_is_speaking:
        return Action.SPEAK
    return Action.PAUSE

# Simulate ~2 seconds of a conversation at 20 decisions/second
random.seed(7)
state = StreamState(user_is_speaking=True, user_finished_thought=0.1,
                     pending_delegation=False, question_complexity=0.2)
for tick in range(40):
    action = policy_step(state)
    print(f"t={tick:>2} ({tick*0.05:.2f}s)  state={state}  ->  action={action.value}")
    state.user_finished_thought = min(1.0, state.user_finished_thought + random.uniform(0.02, 0.08))
    if state.user_finished_thought > 0.9:
        state.user_is_speaking = False
        state.question_complexity = random.uniform(0.5, 0.95)
    if action == Action.DELEGATE:
        state.pending_delegation = True
    if action == Action.ACKNOWLEDGE and random.random() < 0.3:
        state.pending_delegation = False  # background model returned

The mechanical distinction worth internalizing is this: a standard chat model's only real decision per token is which word comes next. A full-duplex voice model's per-timestep decision is first whether to produce output at all, and only then, if it decides to speak, what to say. That first decision, speak versus don't, is the part that has no clean analogue in text generation, and it is the part that has to run continuously and cheaply enough to evaluate many times a second without itself becoming a latency bottleneck. This is also exactly why the model doing this job has to be architecturally small and fast relative to whatever handles the hard reasoning, since a slow policy loop defeats the entire point of full duplex.

The Delegation Pattern: A Fast Model for the Stream, a Big Model for the Hard Problem

This is the part of GPT-Live's architecture that is the most interesting engineering story, and it is worth being precise about what OpenAI actually built rather than flattening it into "it uses two models." GPT-Live is structured as two layers: a continuous interaction layer, the full-duplex voice model itself, that owns the live audio stream and the decision loop described above, and a delegation layer that hands anything requiring real reasoning, a web search, a multi-step calculation, deep domain knowledge, agentic tool use, to GPT-5.5, OpenAI's larger frontier text model, running in the background [3][1]. The voice layer keeps the conversation alive, acknowledging, back-channeling, answering easy follow-ups, while GPT-5.5 works, then folds the result back into the conversation naturally once it lands, rather than making the user sit through dead air while the harder model thinks [1][5].

DeepLearning.AI's The Batch put the design rationale well: decoupling the conversational layer from the reasoning layer means GPT-Live can inherit every future frontier-model improvement without ever having to trade responsiveness for depth, since the voice model does not need to get smarter to answer a hard question, it just needs to know when to ask a smarter model for help [12]. That is a genuinely different tradeoff than making one model do both jobs, and it explains why the benchmark jump in the previous section, 45.3 to 84.2 on GPQA, 0.7 to 75.2 on BrowseComp, happened without the voice model itself getting bigger. The voice layer did not get more capable at reasoning. It got better at recognizing when to stop trying and hand off.

A Family of "Fast Layer, Slow Layer" Patterns

GPT-Live's delegation split is not a novel idea in isolation, it is the voice-specific instance of a pattern that keeps reappearing across AI systems whenever a fast, cheap component needs to stay responsive while a slower, more capable component does the actual hard work. Three comparisons are worth walking through directly, because each one clarifies a different part of why this pattern works.

Speculative decoding is the closest structural cousin, even though it solves a completely different problem. In speculative decoding, a small, fast draft model proposes several tokens ahead, and a large target model verifies all of them in a single parallel forward pass, accepting the ones that match what it would have generated itself and discarding the rest [13][14]. The draft model never has final authority, it only proposes, and the big model's verification is what makes the technique exact rather than approximate. GPT-Live's delegation is structurally inverted from that, the small, fast layer has full authority over the live conversation, and the big model is invoked selectively, only when the fast layer decides a question exceeds what it should try to answer alone, rather than on every single response. But the underlying bet is the same one: most of the workload does not need the expensive model, so keep the expensive model in reserve and pay for it only when it earns its cost. Our full walkthrough of how speculative decoding works covers the draft-and-verify mechanism in depth if you want the token-generation version of this same idea.

Mixture-of-experts routing is the second useful comparison, and it is closer in spirit than it first appears. An MoE model like Tencent's 770B Hy4 preview or NVIDIA's Nemotron 3 Ultra activates only a small subset of its total parameters per token, routing each token to a handful of specialized experts out of a much larger pool, rather than running every parameter on every input. The router's job is exactly the same shape of decision GPT-Live's voice layer makes: given the current input, decide which specialized capacity actually needs to get involved, and skip the rest. GPT-Live's routing decision is binary and coarser, handle it locally or delegate to GPT-5.5, rather than a soft selection over dozens of experts, but the underlying principle, most inputs do not need your full capacity, spend compute selectively, is identical.

Agentic model routers are the third and most directly analogous comparison, because they solve almost exactly GPT-Live's problem, just outside a real-time audio context. NVIDIA's NeMo Switchyard is an open-source router that sits between an application and a pool of models, deciding per request or per agent step which model should actually handle it, optimizing for cost, latency, or quality depending on task difficulty [15]. Runway's model router does something structurally similar for creative generation workloads, picking the right underlying model for a given generation request rather than sending everything through the largest available option. Miraflow's own explainer on routing AI requests like NeMo Switchyard and Runway's router covers both in depth, and the comparison to GPT-Live is genuinely direct: GPT-Live is effectively a real-time, latency-critical, two-tier version of the same router pattern, except the router itself, the voice layer, also has to carry the entire live conversation rather than sitting outside it as a stateless proxy. That extra constraint, the router has to be a full conversational participant, not just a dispatcher, is what makes voice delegation a harder systems problem than text-based model routing, and it is why GPT-Live's fast layer cannot simply be a lightweight classifier the way many text routers can. It has to be a real, capable dialogue model in its own right, one that just happens to also know when to get out of its own way.

A simplified illustration of the shape of that delegation decision, again not GPT-Live's real internals, but a structurally honest sketch of the pattern:

python
/code
# Simplified illustration of GPT-Live's delegation pattern: the fast
# conversational layer keeps the audio stream alive with acknowledgments
# while a background reasoning model (GPT-5.5 in GPT-Live's case) works
# on the actual hard question [1][5][12]. Not GPT-Live's real internals.
import asyncio
import random

ACKNOWLEDGMENTS = ["Let me check that.", "Good question, one sec.", "Mhmm, looking into it."]

async def background_reasoning_call(question: str) -> str:
    """Stand-in for a call to a large background model like GPT-5.5.
    Real latency for a genuinely hard, tool-using question can run into
    several seconds, which is exactly why the conversational layer keeps
    talking instead of going silent while this runs."""
    await asyncio.sleep(random.uniform(2.0, 4.5))
    return f"Background model's answer to: {question}"

async def conversational_layer(question: str, needs_delegation: bool):
    if not needs_delegation:
        print("[voice layer] answering directly, no delegation needed")
        return

    print(f"[voice layer] {random.choice(ACKNOWLEDGMENTS)}")
    task = asyncio.create_task(background_reasoning_call(question))

    # Keep the stream alive with small talk / backchannels while waiting,
    # instead of silence -- this is the acknowledge-now, answer-later
    # pattern Agora's measurement documented in GPT-Live itself.
    while not task.done():
        print("[voice layer] ...(still with you, one moment)")
        await asyncio.sleep(1.0)

    result = await task
    print(f"[voice layer] incorporating result: {result}")

asyncio.run(conversational_layer(
    "What's the current forecast for launch demand next quarter, cross-referenced with our supplier lead times?",
    needs_delegation=True,
))

How Fast Is It Really? What OpenAI Didn't Publish

Latency is the single most load-bearing metric for a full-duplex voice model, and it is also the number OpenAI's own launch materials are conspicuously light on. The company describes GPT-Live as feeling like a real conversation and highlights the decision-loop and delegation architecture at length, but it does not publish a hard, specific latency figure the way, for instance, Google publishes model-processing latency ranges for Gemini Live [1][16]. That gap is exactly why it is worth treating any specific "sub-300ms" number circulating in secondary coverage with real caution rather than repeating it as an OpenAI-confirmed figure, since it does not trace back to an official source or to independently reproduced measurement.

Agora, a real-time engagement infrastructure company, ran its own measurement rather than waiting for OpenAI to publish one, and the methodology is worth taking seriously: prerecorded speech played through an artificial mouth apparatus into a real iPhone 13, dual-track waveform recording, thirty samples per condition [17]. The results are notably slower than the sub-300ms figure that circulates informally, and more interesting for what they reveal about the architecture than the raw numbers alone. GPT-Live-1's response latency, from the last frame of user speech to the first frame of AI audio, measured around 1.1 seconds median, with a 90th-percentile figure around 1.2 seconds, actually a touch behind Advanced Voice Mode's roughly 1.3 second median in Agora's own test [17]. Where GPT-Live pulled clearly ahead was interruption handling and resilience: under a simulated 10 percent uplink packet loss, GPT-Live-1's latency degraded by roughly 314 milliseconds, against a roughly 2.4 second degradation for Advanced Voice Mode under the same loss [17].

That packet-loss result is the more technically meaningful finding, and it follows directly from the architecture covered above. Agora's own explanation is that GPT-Live uses an incremental architecture that consumes audio continuously, so a lost packet costs the model some context rather than stalling an entire pipeline stage waiting for a complete, uncorrupted chunk to arrive [17]. A discrete pipeline has a hard dependency at every stage boundary, if the STT stage never gets a clean enough signal to finalize a transcript, everything downstream simply waits. A continuous stream-processing model has no equivalent hard stall point, it just has slightly degraded input for one moment in an ongoing stream it keeps processing regardless. Agora also documented a three-clock structure worth understanding for anyone building against this kind of model: acknowledgment onset, when the model first makes a sound at all, often a quick "let me check that"; substantive-answer onset, when the actual informative content starts; and answer completion [17]. Separating those three clocks is exactly what lets a delegation call to GPT-5.5 feel fast even when it is not, the acknowledgment clock can fire in well under a second while the substantive-answer clock waits on the background model, and a user perceives responsiveness from the first clock even though the real answer is still cooking.

The honest read here is qualitative rather than a single headline number: GPT-Live is not obviously faster than its predecessor on raw first-response latency by Agora's independent measurement, but it degrades far more gracefully under real-world network conditions, and its architecture is what makes the acknowledge-now, answer-later pattern from the delegation section actually work end to end. If your own product cites a sub-300ms figure for GPT-Live, ask where it traces back to, because as of this writing that specific number has not been corroborated by OpenAI's own materials or by any independent measurement this post could verify. For more on how much raw inference latency shapes a product's feel, our breakdowns of OpenAI's Jalapeno inference chip and GPT-5.6 Sol Ultrafast on Cerebras wafer-scale silicon cover the hardware side of the same latency problem GPT-Live is solving at the model-architecture level.

On the qualitative side, several independent reviewers, not just OpenAI's own materials, describe GPT-Live as noticeably more responsive to vocal tone, pacing, and mood than Advanced Voice Mode, including the ability to whisper, adjust pacing around detected hesitation, and shift register for playful or serious moments on the higher-tier model, though none of these are attached to a hard, independently reproducible number the way the latency figures above are, so they are worth treating as a described capability rather than a measured one [4][6].

How GPT-Live Compares to Gemini Live, Grok Voice, and Alexa+

GPT-Live shipped into a market where full-duplex, or at least full-duplex-adjacent, voice AI was already a live competitive front, not a green field. Understanding where it sits relative to the alternatives is more useful than reading its architecture in isolation.

Google's Gemini Live takes the most architecturally similar approach: native audio-to-audio processing that collapses the transcribe-reason-synthesize stack into a single model, over a persistent bidirectional WebSocket connection that keeps a continuous, full-duplex stream of audio, video frames, and transcripts flowing in both directions at once [16]. Google publishes concrete model-processing latency figures for its Flash Live models, in the 100 to 200 millisecond range on TPU-backed serving infrastructure, a hard number OpenAI has not matched with an equivalent public figure for GPT-Live [16]. Where Gemini Live's public documentation is comparatively thin is on the delegation side, Google has not described an equivalent explicit hand-off to a larger background reasoning model the way OpenAI has described GPT-Live handing off to GPT-5.5, which suggests Gemini Live's native audio model is doing more of its reasoning in-line rather than delegating it out.

xAI's Grok Voice takes a related but distinct bet: a single unified speech-to-speech model rather than a two-layer conversational-plus-delegation split, reasoning directly in the audio domain with no conversion to and from text as an intermediate step [18]. Its Think Fast 2 update specifically targets first-audio-response latency, reportedly cutting it to around 0.70 seconds from roughly 1.25 seconds in the prior version, a genuinely fast, single-model-focused optimization rather than an architectural delegation strategy [18]. The tradeoff is the one you would expect: a single model that has to be fast enough for real-time audio has a harder ceiling on how deep its own reasoning can go on a genuinely hard question, compared to a system with an explicit escape hatch to a much larger background model.

Amazon's Alexa+ approaches the same underlying turn-taking problem from a different angle, one grounded in years of production call-center-style deployment rather than a from-scratch full-duplex redesign. Amazon's own published turn-taking work fuses audio and visual signal through an on-device model to judge device-directedness, whether speech is actually meant for Alexa, distinguishing that from a person talking to someone else in the room, and separately handles what it calls contextual barge-in, where an interruption like "that one" needs to resolve against exactly how far Alexa had gotten through a list it was reading aloud [19]. Alexa+ itself is built around a new speech-to-speech model that produces output audio directly from input audio and adapts prosody to match the speaker, a real architectural shift for Amazon, though its published technical description leans more on natural, humanlike delivery and device-directedness detection in shared-room, ambient environments than on the explicit many-times-per-second decision loop OpenAI describes for GPT-Live [20][19].

The honest takeaway across all four systems is that "full-duplex" is no longer a single agreed-upon architecture, it is a design goal that at least four major labs are solving with real but meaningfully different tradeoffs: Google betting on a single native audio-to-audio model with hard TPU-backed latency numbers, xAI betting on raw single-model speed, Amazon betting on device-directedness and ambient, shared-room robustness, and OpenAI betting on an explicit two-layer split that trades a slightly less impressive raw latency number for the ability to plug in an arbitrarily capable background reasoning model without ever touching the conversational layer.

Building a Voice Agent Against This Kind of Model

GPT-Live itself has no public developer API at the time of this post, OpenAI has said an API is coming but has not published pricing or a date [8][3]. What does exist today, and what any team building toward GPT-Live-style delegation should build against right now, is OpenAI's Realtime API, the same websocket-based, persistent-session pattern that GPT-Live's own eventual API is expected to extend rather than replace [21][22]. Designing your integration layer around this pattern now means adopting GPT-Live later is a model swap, not a rebuild.

The pattern below is illustrative, written to be structurally realistic against the Realtime API's actual session and event model, session configuration, semantic voice-activity detection with an eagerness setting, and a persistent websocket carrying streamed audio in both directions, rather than pseudocode [21][22]. Treat the exact event and field names as representative of the pattern rather than a guaranteed-current API surface, and check OpenAI's current API reference before shipping.

python
/code
# Illustrative pattern for building a voice agent today against OpenAI's
# Realtime API, the websocket-based, persistent-session model that
# GPT-Live's own developer API is expected to extend once it ships
# [21][22]. Field and event names reflect the documented Realtime API
# pattern; check OpenAI's current API reference before shipping, since
# GPT-Live itself has no public API at the time of writing.
import asyncio
import base64
import json
import os

import websockets

REALTIME_URL = "wss://api.openai.com/v1/realtime?model=gpt-realtime"

async def run_voice_agent():
    headers = {
        "Authorization": f"Bearer {os.environ['OPENAI_API_KEY']}",
        "OpenAI-Beta": "realtime=v1",
    }

    async with websockets.connect(REALTIME_URL, extra_headers=headers) as ws:
        # Configure the session: voice, background delegation instructions,
        # and semantic VAD with a tuned eagerness level rather than a fixed
        # silence timeout -- the production analogue of the decision-loop
        # speak-vs-listen tradeoff covered earlier in this post.
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "modalities": ["audio", "text"],
                "voice": "verse",
                "instructions": (
                    "You are a live voice agent. Handle quick questions "
                    "yourself. For anything requiring real research, "
                    "multi-step reasoning, or a tool call, acknowledge "
                    "briefly and call the `delegate_to_reasoning_model` "
                    "tool instead of guessing."
                ),
                "turn_detection": {
                    "type": "semantic_vad",
                    "eagerness": "medium",   # low | medium | high | auto
                },
                "tools": [{
                    "type": "function",
                    "name": "delegate_to_reasoning_model",
                    "description": "Hand a complex question to a larger background model.",
                    "parameters": {
                        "type": "object",
                        "properties": {"question": {"type": "string"}},
                        "required": ["question"],
                    },
                }],
            },
        }))

        async def stream_microphone_audio():
            """Continuously stream raw PCM16 audio chunks from the mic.
            capture_audio_chunk() is a stand-in for a real audio capture
            loop (e.g. sounddevice or a browser getUserMedia bridge)."""
            while True:
                chunk = await capture_audio_chunk()
                await ws.send(json.dumps({
                    "type": "input_audio_buffer.append",
                    "audio": base64.b64encode(chunk).decode("utf-8"),
                }))

        async def handle_server_events():
            async for raw_event in ws:
                event = json.loads(raw_event)

                if event["type"] == "response.audio.delta":
                    play_audio_chunk(base64.b64decode(event["delta"]))

                elif event["type"] == "response.function_call_arguments.done":
                    # The model chose to delegate. Acknowledge immediately,
                    # then run the real background call and feed the
                    # result back into the same conversation once ready.
                    args = json.loads(event["arguments"])
                    asyncio.create_task(
                        handle_delegation(ws, event["call_id"], args["question"])
                    )

                elif event["type"] == "input_audio_buffer.speech_started":
                    # The user has started talking again -- this is the
                    # barge-in signal. Stop any audio already queued for
                    # playback immediately, do not just let it finish.
                    stop_audio_playback()

        await asyncio.gather(stream_microphone_audio(), handle_server_events())


async def handle_delegation(ws, call_id: str, question: str):
    """Runs the actual background reasoning call and reports the result
    back into the live session once it resolves."""
    answer = await call_background_reasoning_model(question)  # e.g. GPT-5.5
    await ws.send(json.dumps({
        "type": "conversation.item.create",
        "item": {
            "type": "function_call_output",
            "call_id": call_id,
            "output": answer,
        },
    }))
    await ws.send(json.dumps({"type": "response.create"}))


# Stand-ins for real audio I/O and the background model call, left
# unimplemented here since they depend on your specific client stack.
async def capture_audio_chunk() -> bytes: ...
def play_audio_chunk(pcm_bytes: bytes) -> None: ...
def stop_audio_playback() -> None: ...
async def call_background_reasoning_model(question: str) -> str: ...

asyncio.run(run_voice_agent())

A few details in that snippet are worth calling out because they map directly onto the architecture covered earlier in this post. The turn_detection block using semantic_vad with an eagerness setting is the production analogue of the decision-loop problem from the earlier section: eagerness is effectively a dial on how aggressively the model is willing to interrupt versus wait, the same speak-versus-listen tradeoff GPT-Live's own policy has to resolve many times a second, just exposed as a single tunable knob instead of a learned per-timestep decision [21]. And the pattern of streaming partial acknowledgments while a background function call runs is exactly the acknowledge-now, answer-later shape Agora's three-clock measurement documented in GPT-Live itself, applied at the API level rather than inside the model.

Production Notes: Latency Budgets, Interruption Handling, and Failure Modes

Shipping a real product against a full-duplex voice model is a genuinely different engineering discipline than shipping against a chat completion endpoint, and most of the hard-won lessons from turn-based voice AI only partially transfer.

Set an Explicit Latency Budget, Not Just a Vibe

Treat every conversational turn as having three distinct clocks, the same three Agora's measurement study used: time to first sound of any kind, time to substantive content, and time to full answer completion. Set a real target for each one independently, not one blended number. A reasonable starting budget for a consumer-facing voice product is under a second to first acknowledgment sound, a few seconds to substantive content for anything requiring delegation, and no hard cap on completion time as long as the user is being kept engaged by acknowledgment and partial content along the way. Measure against real network conditions, not just a clean local connection, since Agora's packet-loss result shows the gap between a full-duplex architecture and a legacy pipeline widens specifically under degraded conditions, which is exactly when your product is most likely to be judged harshly by a real user on a real mobile network [17].

Design Interruption Handling as a First-Class State, Not an Edge Case

In a half-duplex pipeline, interruption is an exception path bolted onto a system that was not designed for it. In a full-duplex system, being interrupted, and correctly interrupting back, is a core, constantly-active state, not an edge case, and your application logic needs to treat it that way from the start. Concretely, that means your client needs to handle three distinct interruption scenarios differently: the user interrupting the model mid-sentence and expecting the model to actually stop generating audio immediately, not just stop playing audio that was already buffered; the model itself deciding to interrupt the user, which should be rare and used sparingly, reserved for genuinely time-sensitive corrections rather than the model talking over the user out of habit; and contextual barge-in, where the interruption references something the model had only partially said, "wait, the second one," which requires the model to actually know how far it got, the same problem Amazon's Alexa team specifically named in its own turn-taking work [19].

Budget for the Delegation Path Separately From the Conversational Path

Every request that triggers a delegation to a background reasoning model, whether that is GPT-5.5 behind GPT-Live or your own equivalent background model, needs its own latency budget and its own fallback behavior, distinct from the conversational path's budget. Decide upfront what the voice layer says while it waits: a generic acknowledgment is safer than a specific promise, since promising "let me pull that up" and then failing to deliver reads as broken in a way that a vaguer "let me think about that for a second" does not. Decide what happens if the background model times out or errors, does the conversational layer apologize and retry, offer to answer with lower confidence itself, or ask the user to repeat the request, and make that decision deliberately rather than letting it default to silence, which is the single worst failure mode in a voice product, since a silent gap in an audio-only channel is much harder for a user to interpret than an empty state in a visual UI.

Handle Tool Calls Mid-Conversation Explicitly

If your voice agent invokes tools, function calls, retrieval, external API requests, treat every tool call as an interruption-shaped event with its own acknowledgment and its own timeout, exactly like a delegation to a background reasoning model. A tool call that takes four seconds with no verbal acknowledgment reads to a user as the system having frozen or disconnected, not as the system working on their request, precisely because a voice channel has no visual loading indicator to fall back on. Our deeper look at how LLM agents actually learn to use tools covers the underlying tool-calling mechanics this applies to, and it is worth pairing with a hard look at failure rates: our breakdown of why 88 percent of AI agents fail in production is written about text agents, but nearly every failure category it names, unhandled timeouts, silent errors, brittle state management, applies just as directly to a voice agent's background tool calls and delegation path.

python
/code
# A production latency-budget worksheet using the three-clock model
# Agora's independent GPT-Live measurement documented: acknowledgment
# onset, substantive-answer onset, and answer completion [17].
from dataclasses import dataclass

@dataclass
class LatencyBudget:
    ack_onset_ms: int          # time to first sound of any kind
    substantive_onset_ms: int  # time to first informative content
    completion_ms: int         # time to full answer completion

def evaluate_budget(measured: LatencyBudget, target: LatencyBudget) -> None:
    checks = [
        ("acknowledgment onset", measured.ack_onset_ms, target.ack_onset_ms),
        ("substantive-answer onset", measured.substantive_onset_ms, target.substantive_onset_ms),
        ("answer completion", measured.completion_ms, target.completion_ms),
    ]
    for label, measured_val, target_val in checks:
        status = "OK" if measured_val <= target_val else "OVER BUDGET"
        print(f"{label:<28} measured={measured_val:>5}ms  target<={target_val:>5}ms  [{status}]")

# A reasonable starting production target, separate budgets for the
# conversational path and the delegation path.
conversational_target = LatencyBudget(ack_onset_ms=800, substantive_onset_ms=1500, completion_ms=3000)
delegation_target = LatencyBudget(ack_onset_ms=800, substantive_onset_ms=4000, completion_ms=8000)

# Example measured values from a real test call
measured_easy_question = LatencyBudget(ack_onset_ms=650, substantive_onset_ms=1200, completion_ms=2100)
measured_delegated_question = LatencyBudget(ack_onset_ms=700, substantive_onset_ms=5200, completion_ms=9400)

print("Direct-answer path:")
evaluate_budget(measured_easy_question, conversational_target)
print("\nDelegated (background model) path:")
evaluate_budget(measured_delegated_question, delegation_target)

Known Failure Modes to Design Around

A few failure modes show up consistently across full-duplex voice deployments and are worth naming directly rather than discovering them in production.

Confident wrong answers delivered smoothly. Reviewers testing GPT-Live specifically flagged this: a genuinely more natural voice makes an incorrect answer sound more trustworthy than the same wrong answer delivered in a stilted, obviously synthetic voice would, which is a real UX risk independent of raw model accuracy [8]. Design your confidence signaling, hedging language, offers to verify, explicit citations when relevant, deliberately, rather than assuming a more natural voice is strictly an improvement with no downside.

Over-eager or under-eager interruption. A model tuned to interrupt too readily talks over users mid-thought, which reads as rude and erodes trust fast. A model tuned to interrupt too rarely fails to jump in when a user genuinely wants a quick correction or clarification, which reads as unresponsive. Both failure directions are real, they are opposite ends of the same eagerness dial covered in the code example above, and the right setting is genuinely workload-dependent, a customer-support agent handling frustrated callers wants a different eagerness setting than a language-tutoring product where thoughtful pauses from the user should not be interrupted.

Language and accent gaps outside the model's strongest languages. Coverage of GPT-Live's live translation feature specifically noted a heavy accent artifact in some target languages and an unnaturally formal, "bookish" register in others, a reminder that full-duplex capability and per-language fluency are two separate axes of quality, and a model can be architecturally excellent at turn-taking while still sounding noticeably non-native in a language it was not as heavily optimized for [2].

Packet loss and real network conditions degrading the experience unevenly. As Agora's measurement showed, this is where full-duplex architectures pull genuinely ahead of legacy pipelines, but "ahead" does not mean immune, and testing only on clean, low-latency office wifi will hide problems that show up immediately on a real user's mobile connection [17].

Common Mistakes Developers Make Integrating Full-Duplex Voice Models

A handful of specific mistakes show up repeatedly when teams move from a turn-based STT-LLM-TTS pipeline to a full-duplex model, worth naming explicitly since they are easy to make even with a genuinely capable model underneath.

• Treating it like a drop-in replacement for a turn-based pipeline. A full-duplex model expects a continuous audio stream and makes its own decisions about when to speak. Wrapping it in application logic that still waits for a discrete "user turn ended" signal before sending audio defeats the entire architecture and reduces it to a slower version of the pipeline it was meant to replace.
• Ignoring the latency budget for the delegation path. Building a demo against only the conversational layer's fast responses, then discovering in production that every real request that needs actual reasoning triggers a multi-second background call with no acknowledgment strategy in place, is one of the most common gaps between a prototype and a shippable product.
• Not handling barge-in state correctly. Failing to actually stop audio generation the instant a user interrupts, instead of just muting already-buffered audio client-side, produces the specific broken feeling of a voice agent that "hears" the interruption but keeps talking anyway for another second or two.
• Assuming a single eagerness or interruption setting works for every use case. A setting tuned for a fast-paced customer support flow will feel aggressive and rude in a reflective, tutoring-style product, and vice versa. Treat the eagerness dial as a real product decision, not a default to leave untouched.
• Skipping real network-condition testing. Full-duplex architectures earn much of their real advantage specifically under degraded conditions, packet loss, mobile handoffs, spotty wifi, and testing exclusively on a clean connection hides exactly the scenario where the architecture matters most.
• Citing unverified latency or capability numbers as OpenAI-confirmed. As covered above, several specific figures circulating about GPT-Live, a hard sub-300ms latency claim among them, do not trace back to an official source or a reproducible independent measurement. Cite what is actually verifiable, and describe the rest qualitatively.

Where Voice Delegation Meets Voice-Driven Content Creation

The small-model-handles-the-stream, big-model-handles-the-hard-part pattern is not unique to live conversational voice AI, it shows up anywhere a product needs to feel immediately responsive while a harder generation step happens underneath. Text2Shorts in Miraflow AI follows a version of the same shape end to end, generating a script from a topic, then scene visuals based on that script, then a voice narration track, then a finished vertical video, each stage handled by the model best suited to it rather than forcing one model to do the entire pipeline alone. The user experience goal is the same one GPT-Live is chasing in a real-time audio context: the workflow should feel immediate and continuous even though real, sometimes slow, generation work is happening underneath each stage.

You can browse the full pipeline, and every other tool mentioned in this post's product tie-ins, at miraflow.ai, and see more architecture explainers like this one on the Miraflow AI blog.

If you want to see this delegation idea visualized as motion rather than static architecture diagrams, here is a video generation prompt built around the same conversation-flow concept covered throughout this post, written for a Wan or Veo-style video model:

A tabletop scene shot from directly above on a dark wooden surface. Two intertwined ribbons of light, one warm amber and one cool blue, flow continuously across the frame representing two speakers' voices overlapping without ever fully stopping, occasionally one ribbon dims briefly while the other brightens, showing a natural back-and-forth with no harsh cuts or freezes. Midway through, a small brass clockwork mechanism at the edge of frame lights up and begins turning quietly in the background while the two ribbons of light keep flowing uninterrupted in the foreground, then the mechanism dims again once it settles, its result folding smoothly back into the flowing ribbons. Camera holds a slow, steady overhead drift throughout. Warm cinematic lighting, shallow depth of field at the edges, no readable text, no logos, no people, smooth continuous motion.

Frequently Asked Questions

Is GPT-Live available to everyone right now? Yes, GPT-Live-1 mini is the default voice experience for every ChatGPT user, including the free tier, and GPT-Live-1, the larger variant, is available to Go, Plus, and Pro subscribers, on iOS, Android, and the web [1][3]. It is not yet available in ChatGPT for Business, Enterprise, or Education workspaces, Temporary Chats, the desktop app, Codex, or custom GPTs [6].

Is there a developer API for GPT-Live? Not at launch. OpenAI has said an API is coming but has not published a date or pricing [8]. Developers building toward it today should build against the existing Realtime API, which uses the same persistent-websocket, streaming-audio pattern GPT-Live's own eventual API is expected to extend [21][22].

Does GPT-Live actually run at sub-300ms latency? This specific figure circulates widely in secondary coverage but is not confirmed in OpenAI's own materials, and it does not match the independent measurement this post could verify. Agora's own test measured a roughly 1.1 second median first-response latency for GPT-Live-1, on par with or slightly behind Advanced Voice Mode on that specific metric, with GPT-Live's real advantage showing up in graceful degradation under packet loss rather than raw first-response speed [17]. Treat the sub-300ms figure as unverified until traced to an official or independently reproducible source.

What does "delegates to GPT-5.5" actually mean mechanically? GPT-Live's continuous interaction layer, the full-duplex voice model, handles the live conversation and decides when a request needs real reasoning, web search, or agentic tool use it should not attempt itself. When it decides that, it hands the task to GPT-5.5, OpenAI's larger frontier text model, running in the background, keeps the conversation alive with acknowledgments and small talk while that model works, then incorporates the result once it is ready [1][3].

How is this different from just using a smaller model for everything? A single small model tuned only for real-time conversation has a hard ceiling on how deep it can reason, since the same architecture that makes it fast enough for live audio limits how much computation it can spend per response. The two-layer split lets the conversational layer stay fast and responsive permanently while the reasoning ceiling is set by whatever the background model, GPT-5.5 today, can do, and that ceiling can rise with every future frontier model release without ever touching the voice layer itself [12].

Are the four reasoning tiers available to free users? No. Free-tier users get the Instant tier only. Go, Plus, and Pro subscribers unlock Medium and High, and Pro adds an extended X-High tier for the most demanding agentic and research-style requests [6][4].

How does GPT-Live compare to Gemini Live and Grok Voice? All three are full-duplex or full-duplex-adjacent, but with different bets. Gemini Live uses a single native audio-to-audio model with published, comparatively fast model-processing latency figures on Google's TPU infrastructure [16]. Grok Voice uses a single unified speech-to-speech model optimized aggressively for raw first-response speed [18]. GPT-Live is the only one of the three with an explicit, disclosed two-layer delegation split to a separate, much larger background reasoning model [1][16][18].

Conclusion

GPT-Live is a genuinely interesting piece of production AI engineering wrapped in what reads, on the surface, like a routine ChatGPT feature update. The full-duplex shift from discrete pipeline stages to a continuous audio stream is a real architectural change, not a latency-tuning exercise, and the many-times-per-second decision loop it requires is a meaningfully different inference pattern from standard autoregressive generation, closer to a real-time control policy than to next-token prediction. The delegation split to GPT-5.5 is the more quietly important half of the design, a real, working instance of the same fast-layer, slow-layer pattern that shows up across speculative decoding, mixture-of-experts routing, and agentic model routers, applied for the first time at real consumer scale to a live audio conversation rather than to text generation or offline routing. The honest technical picture includes real gaps too, no confirmed developer API yet, latency numbers that OpenAI has not published and that independent measurement suggests are not as dramatic as some secondary coverage claims, and failure modes, confident wrong answers chief among them, that a smoother voice makes easier to miss rather than harder to hit. All of that is exactly what makes it worth a real technical read rather than a feature announcement skimmed once and forgotten.

References and Sources

[1] OpenAI. "Introducing GPT-Live."

[2] TechCrunch. "OpenAI releases new voice models for more natural live conversations."

[3] MacRumors. "OpenAI Introduces GPT-Live to Make ChatGPT Voice Feel Like a Real Conversation."

[4] Build Fast with AI. "GPT-Live Review: OpenAI's Full-Duplex Voice Model Explained (July 2026)."

[5] MarkTechPost. "OpenAI Releases GPT-Live and GPT-Live-1 mini: Full-Duplex Voice Models That Delegate Deeper Reasoning to GPT-5.5."

[6] eesel AI. "What is GPT-Live? OpenAI's real-time voice AI, explained."

[7] Simon Willison. "Introducing GPT-Live."

[8] eesel AI. "GPT-Live review: is OpenAI's new voice AI worth it?"

[9] MLQ.ai. "OpenAI Launches GPT-Live-1, a Full-Duplex Voice Model That Listens and Speaks Simultaneously."

[10] Kyutai. "Moshi: a speech-text foundation model for real-time dialogue."

[11] Xu et al.. "Qwen2.5-Omni Technical Report."

[12] DeepLearning.AI, The Batch. "One Model Talks, Another One Thinks."

[13] Leviathan, Kalman, Matias. "Fast Inference from Transformers via Speculative Decoding."

[14] Chen, Borgeaud, Irving, Lespiau, Sifre, Jumper. "Accelerating Large Language Model Decoding with Speculative Sampling."

[15] NVIDIA Technical Blog. "Route AI Agents Across Models with NVIDIA NeMo Switchyard."

[16] Google AI for Developers. "Live API capabilities guide."

[17] Agora. "OpenAI Didn't Publish GPT-Live's Latency. So We Measured It."

[18] xAI. "Voice Overview, Grok API Documentation."

[19] Amazon Science. "New Alexa features: Natural turn-taking."

[20] About Amazon. "Introducing Alexa+: skills, cost, and availability."

[21] OpenAI. "Voice activity detection (VAD), Realtime API guide."

[22] OpenAI. "Realtime conversations, Realtime API guide."