Conversation engines

A hosted Skysay AI agent answers the phone with one conversation engine. The default is Standard: speech-to-text, a model, then text-to-speech. You can instead choose an available live engine, which listens and speaks within one session. The catalog states which engines can currently be selected.

Telephony still uses Skysay numbers and LiveKit. GPT-Live 1 does not send the call to OpenAI SIP.

Choose the engine

In the workspace, open the AI agent and look at Voice. The first control is Conversation engine. The same choice is the first control on the create wizard's voice step.

• Standard keeps separate transcriber, model, and voice rows. Interruptions, acknowledgements, and tool-wait speech stay under your control.
• GPT-Live 1 replaces those three rows with a GPT-Live voice and a backend model for tools. The live model owns barge-in, backchannels, and waiting speech, so acknowledgements and turn-taking stay off for that engine.
• Eleven Realtime, when enabled, uses a native ElevenLabs session for speech, reasoning and tool requests. It requires a written greeting and an agent-first opening. See Eleven Realtime.

GET /v1/agent-catalog lists the engines under conversation_engines. Standard remains the default when you omit the field. Each row declares what the engine can do rather than leaving you to infer it from its name; today the properties include has_language_axis, supports_manual_replies and supports_deferred_tool_replies.

Engines and the language axis

A language axis is the group of three settings that only make sense for an engine that picks a transcriber and a voice per language: the accepted language set, the per-language overrides, and Follow the caller.

• Standard has the axis. has_language_axis is true.
• GPT-Live 1 does not. It has no language parameter, no separate transcriber, and one voice for the whole call, so has_language_axis is false.

Read the flag from the catalog rather than matching on the engine id. Any future voice-to-voice engine will declare false and follow exactly the same rule, so a client that reads the flag needs no change when one is added.

Browser testing

The test panel offers two targets. Current draft runs the engine, voice and backend as they stand on the page, before you publish -- switch a Standard agent to GPT-Live 1, pick a voice, and talk to it. Published runs the last published snapshot. The panel names what each target will run: for GPT-Live 1 the voice and engine ("Marin · GPT-Live 1"), the backend for tools, and that speech-to-text is built into the engine.

Two things to know:

• On an agent with published languages, a Current draft test takes the voice and engine from the draft and the prompt and greeting from the published languages. The panel says so.
• While Follow the caller is published for an agent, the panel offers only Published: the published Follow contract is bound to the published transcriber. Publish the voice change to test it.

If a test cannot start because of the agent's setup, the panel lists what needs fixing in the same words the Agents page uses, with a link to the section, rather than an error code.

On the homepage demo

The anonymous homepage demo offers active agents in the Skysay demo workspace whose engine is supported by its platform-funded billing path. Eleven Realtime is not yet offered in the anonymous demo; use its workspace browser test panel instead. A GPT-Live or Gemini Live agent appears once, in its own language, and follows the visitor's language inside the session, as it does on a phone call. It opens with its configured greeting, and the demo's session length and daily limits apply exactly as they do to a Standard agent.

API and MCP

The discriminator is voice_frontend: use an engine identifier from the catalog, such as cascade, gpt_live, gemini_live or eleven_realtime. Omit it and the agent stays Standard. An unavailable engine cannot be selected.

An engine whose catalog row says has_language_axis: false refuses languages, language_profiles and follow_caller. Both writers refuse them -- POST /v1/agents and PATCH /v1/agents/{agent_id}/voice-stack, over REST and over MCP -- with a 400 that names the field and the engine:

GPT-Live has no language set; omit languages for this conversation engine

The field is refused rather than accepted and ignored, so a client never believes it chose a set the engine cannot honour. PATCH /v1/agents/{agent_id} has never saved these three fields and still answers 400 pointing at the voice-stack route.

A language set the agent kept from its Standard days stays stored and is inert: its profiles are published engine-shaped, no per-language transcriber or voice override is applied, and Follow the caller publishes disabled. Nothing clears it on a later save, because it is what a switch back to Standard resumes -- send voice_frontend: "cascade" with a full Standard stack and the three fields are writable again in the same request.

A browser test's draft overlay (POST /v1/agents/{agent_id}/browser-sessions with snapshot: "draft") names its engine in draft.stack.voice_frontend and carries that engine's own fields: for GPT-Live the voice and model (backend), for Standard the transcriber, model and voice. See the request schema in the API reference.

{
  "voice_frontend": "gpt_live",
  "voice": {
    "provider": "openai",
    "model": "gpt-live-1",
    "voice_id": "marin"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-5.6-luna"
  }
}

POST /v1/agents and PATCH /v1/agents/{agent_id}/voice-stack accept the field. So do MCP create_agent and set_agent_voice_stack.

GET /v1/agents/{agent_id}/voice-stack always returns transcriber, model, and voice. For GPT-Live 1 those are an OpenAI compatibility projection: native captions as the transcriber, the Responses backend as the model, and gpt-live-1 with the selected voice id.

Patching a Standard transcriber onto a GPT-Live agent without switching the engine is refused. Switching back requires voice_frontend: "cascade" and a full Standard stack.

Voices, language, and opening

GPT-Live 1 offers fourteen voices: Marin, Cedar, Gleam, Meridian, Willow, Stone, Vesper, Quartz, Ripple, Delta, Cinder, Beacon, Bossa and Tempo. Browse, compare and audition them in the Voice Library under the OpenAI provider, or pick one directly on the engine card in the agent editor. Use in an AI agent from the library switches the draft to GPT-Live 1 and tells you so; nothing is published until you publish.

GPT-Live voices are independent of spoken language. Do not filter them through Standard text-to-speech tables. The engine follows the caller's language automatically and cannot be pinned to one; see Which languages the voice agent follows for what that means and how to test your language. Opening language is configurable and does not lock the rest of the call.

GET /v1/agent-catalog lists each voice under conversation_engines with OpenAI's own descriptive fields where the provider documents them: presentation, regional_influence, provider_language_label (OpenAI's per-voice Language label, a description rather than a capability), source (Natural or Generated) and default. Marin and Cedar carry a presentation only; OpenAI does not document the other fields for them, and Skysay does not infer them.

On outbound phone calls without a workflow, GPT-Live 1 and Gemini Live let the person speak first. They hear the whole pickup before Skysay releases the first response. A request or correction in the pickup should be answered first; a simple hello is answered with the configured opening. If nobody speaks for two seconds, Skysay requests the opening. See Who speaks first for the timing and call types.

The configured opening sets the initial language and script. A per-call opening from the Calls API replaces the agent's saved opening for that call. These are instructions to a voice-to-voice model and cannot guarantee exact delivery; the measured Sinhala limit in Which languages the voice agent follows still applies.

If caller input cannot be preserved or observed during startup, the call fails instead of treating missing audio as silence. Startup audio is retained only in memory, for at most eight seconds. A reconnect during the pickup also ends that attempt. This is a universal opening safeguard, with no new API field or agent-editor setting.

Calls where the agent speaks first

Inbound phone calls and browser tests keep their existing agent-first opening. The following input hold and quick-reply recovery apply to that opening path.

While the opening plays, the engine does not hear the line. A voice-to-voice engine answers whatever it hears, so hold music, an automated announcement or someone saying "hello?" at the moment the call connects could otherwise get a reply before the greeting was ever said. Skysay keeps that audio from the engine until the opening has been spoken: until the engine's own transcript of what it said covers the opening, and the line has then been quiet for about a second. Skysay itself keeps listening the whole time, so answer detection and the call recording are not affected.

On GPT-Live, a quick reply after the greeting stops can arrive while Skysay is still checking that quiet period. Skysay briefly retains this post-greeting audio and passes it to the engine once the opening check succeeds, before new live audio. Silent intervals are not replayed; a recovered reply includes at most 200 milliseconds of leading quiet. Retention is limited to two seconds and stays in memory. This does not recover words spoken over the audible greeting or establish what the person meant.

Recovery is skipped when opening delivery cannot be established, the connection restarts, observation is interrupted, delayed audio arrives in a burst, the buffer reaches its limit, or the call is ending. Normal listening still resumes under the existing opening rules. If the caller's reply was not understood, the agent may still need to ask them to repeat it.

Two consequences to plan for:

• Anything the person on the line says during the opening is not passed to the engine afterwards. If they speak over the greeting, the engine answers what they say next.
• If the engine does not finish the opening, it starts listening anyway: 12 seconds after the call is answered if it has said nothing, or at most 6 seconds plus a tenth of a second per character of the opening (never more than 30 seconds) after it started speaking. A call whose engine said something other than the configured opening is not counted as having delivered it.

Without an opening message, GPT-Live and Gemini Live write their own greeting. When the call carries per-call context, that greeting asks for the person by first name only, or who is speaking when the first name is unclear, and states no amount, reference or other account detail.

Eleven Realtime

When enabled, eleven_realtime selects elevenlabs and eleven_realtime_v1_max for both the model and voice. Choose an exact voice from the catalog, a supported language, a nonempty literal first message and opening.first_speaker: "agent". A draft browser test can carry this opening policy in draft.stack.opening; it does not change the published agent.

The model can write delivery directions in square brackets into its replies, such as [cheerful] or [slow]. They shape how the voice sounds and are not spoken, so Skysay removes them from captions, the call transcript and anything derived from it, such as summaries. Other short bracketed text in a reply is treated the same way, so do not instruct the agent to say bracketed text.

The native model decides normal turns and requests tools. Skysay executes those tools using the same call permissions as other engines. Tools that change business data require the independent confirmation listener configured for native engines; its transcription charges are separate from the native model charge. This integration does not support generated greetings, caller-first openings, phone-menu navigation, controlled workflows or manually forced replies. Eleven Realtime cannot use Transfer to human yet. Remove transfer_to_human and cancel_transfer before choosing this engine, or choose an engine that supports deferred tool replies. Skysay refuses conflicting changes, publication and call starts; it does not silently disable a configured transfer. Synchronous tools, including webhooks, end_call, send_sms and an explicitly configured request_callback, remain available subject to their ordinary permissions and confirmation rules. Silence can end a call without an extra check-in or farewell. Native connections are not retried after failure, so a reconnect cannot repeat a tool action.

With Eleven Realtime, while an action runs, the agent cannot hear or answer the caller. Speech during that wait is absent from the transcript; the caller must repeat it after the action finishes. Speaking cannot stop an action that is already running. Interruptions work during normal speech.

There are 21 initial catalog voices and 90 base language choices, derived from the native API's 91 accepted locale codes. These are provider declarations; not every combination has been tested. Sinhala is not an accepted native API language. Listening samples use validated recordings and may cover fewer voice/language combinations.

Eleven Realtime costs $0.15 per connected minute, charged by the second on the call's own connected time: the same duration as the platform charge, so a 2-minute call costs $0.30. Platform, telephone and applicable tool charges are additional. The wallet reservation is an estimate, not the final charge, and duplicate usage cannot charge the same call twice. A call whose ElevenLabs conversation never starts is charged no Eleven Realtime minutes.

An Eleven Realtime agent's instructions can be at most 8,000 bytes of text, including the instructions Skysay adds for its tools. That is about 8,000 characters of English and fewer in most other scripts. A longer prompt cannot be published on this engine, and the save or engine change names the limit and the prompt's current size.

A preview-only entry remains unavailable for calls until the deployed integration is enabled.

Workflows

A GPT-Live 1 agent can run a published workflow in any language. The live session speaks every workflow line in GPT's own voice and its own wording, not word for word; the workflow still decides which step comes next. The agent's Voice → Transcriber listener, shared with its workflow, hears the caller, so a GPT-Live workflow needs a listener chosen there like any spoken workflow, one that accepts every language of the call (Azure real-time transcription, for example, accepts English, Sinhala and Tamil). Phone menu navigation does not run on these calls.

OpenAI publishes no per-language list for GPT-Live 1, so Skysay accepts it in every language it runs calls in and blocks none up front. English is the language Skysay has tested on GPT-Live workflow calls, and it carries the Tested by Skysay badge; any other language is accepted untested. How well GPT-Live 1 holds a language varies (see What Skysay has tested for free-form calls), so test your own languages before you launch. See Spoken workflow calls for the checks and their messages. Gemini Live still cannot run workflow calls.

Call evidence

If GPT-Live reconnects while an initial opening is still waiting to start, Skysay can request that opening once more. Recovery requires the replacement session to accept the same initial context and the original request to have left the send queue. It shares the original ten-second generation-start budget; reconnecting does not restart that budget.

This recovery applies before any caller speech, generated response, or tool activity. An opening that already started is not repeated, and later replies and tool actions are not replayed. Caller input and call-ending controls stay active. Other engines retain their existing failure handling. No agent setting or API option is needed.

GPT-Live saves native captions and full-call recordings when recording is enabled. Captions can be imperfect, especially across languages, and for some languages (Sinhala most often) they can be missing entirely. When the other party spoke and the engine answered without a caption, the transcript in the Calls workspace marks it with a "Spoke — no transcript" line; see Read call evidence. Per-turn audio clips and some timing evidence can be incomplete even when the full recording is available. Check the call's evidence status before using those details to assess audio quality or response speed.

Billing

GPT-Live 1 is billed by call duration, from the moment the call is placed, at $0.05 per minute, including silence, plus tokens for the backend that handles tools. Standard speech-to-text is not charged on that path.

A GPT-Live workflow call is billed differently: GPT-Live session time at the same per-minute rate, plus the workflow listener's speech-to-text. It has no backend token line, because no backend model writes the words, and no text-to-speech line.

See the generated API reference for the exact request and response shapes.

When a live agent proposes to end a call

For a live voice model, Skysay waits for the model's goodbye to finish before closing the connection. Caller input stays open during that wait. If local speech detection says the caller is already speaking, the model's end_call request is refused. If the caller starts speaking while that close is pending, Skysay withdraws the close and leaves the conversation open. After the goodbye, there is up to 750 ms for the caller to take the floor, within the existing shutdown time limit.

This safeguard is automatic; there is no new agent-editor switch or API field. It detects speech activity rather than meaning: noise can keep a call open, and speech the detector misses cannot stop closure. It does not establish identity, confirm a promise, count clarification questions, undo a goodbye already heard, or end a call when the model forgets to invoke end_call. Balance limits and other platform-enforced endings still close the call.

On phone and workspace browser-test calls, the existing farewell fallback can also request closure after it recognizes the caller's closing words. If the caller takes the floor again, that pending request is withdrawn; a later goodbye can request a fresh close. A fallback close does not mean the model invoked end_call. Model tool use and actual call disconnection are separate evidence.

Workflow listening

Ordinary live conversations use their engine's native listening. Spoken workflows use the agent's shared Voice → Transcriber setting independently of the output engine. A GPT-Live workflow can use any listener that accepts every language of the call: Deepgram Nova-3, Azure Realtime, Soniox STT-RT v5, or, in English only, OpenAI GPT-Live Transcribe. The latter is platform-managed, workflow-only, and provides no detected-language evidence. Timing belongs under Tuning → Turn-taking; workflow Publish shows the saved configuration and freezes it into a version. See workflow voice setup.