Audio stream protocol

An external_audio agent hears and speaks the call itself. Skysay answers or places the phone call, then connects the call's audio to your agent's audio_stream_url over a websocket. This page is that connection's protocol, skysay.audio.v1.

It covers phone calls on a Skysay number and on your own SIP trunk, incoming and outgoing. On your own trunk the call costs a flat $0.05 per minute instead of carrier minutes.

The connection

For every call, Skysay opens one wss:// connection to the agent's audio_stream_url.

• Only wss://. The URL must use TLS, and its host must resolve to public internet addresses. Skysay doesn't follow redirects.
• Opened before anyone answers. Skysay connects while an incoming call is still ringing, and before it dials an outgoing call: it places the outgoing call only once your agent has accepted the connection. If your agent can't be reached within 5 seconds, the call ends failed, with technical_failure.failure_reason_code external_audio_stream_unreachable (see Post-call results), and nothing is charged. An incoming call is hung up while it rings; an outgoing call is never dialled, so nobody's phone rings.
• Signed like a webhook. The upgrade request carries three headers: X-Skysay-Call-Id, and X-Agent-Telco-Timestamp and X-Agent-Telco-Signature signing it with the agent's signing key (agent_endpoint.webhook_secret, shown once when you create the agent; see Signing key to get a new one). You can check them before you accept the connection.

Verify the connection

Check the upgrade request before you accept it, the same way you check a webhook delivery, with the call id in place of the body:

1. Compute HMAC-SHA256 with your signing secret over the value of X-Agent-Telco-Timestamp, a ., and the value of X-Skysay-Call-Id.
2. Compare it with the hex digest after sha256= in X-Agent-Telco-Signature.
3. Reject a timestamp more than 5 minutes from your clock. That is the replay window: a captured upgrade is useless after it.
4. Accept each call id once.

Refuse the upgrade if any check fails. When start arrives, check that its call_id is the one the upgrade named.

Messages from Skysay

start is the first message, a JSON text message sent when the call is answered:

{
  "event": "start",
  "protocol": "skysay.audio.v1",
  "call_id": "call_123",
  "agent_id": "agent_456",
  "direction": "inbound",
  "from": "+15550101",
  "to": "+15550102",
  "audio": {"encoding": "pcm_s16le", "sample_rate": 16000, "channels": 1}
}

For an incoming call, from is the caller and to is your Skysay number. For an outgoing call, from is your Skysay number and to is the number dialled. Use call_id to match the call's webhook events and its record from GET /v1/calls/{call_id}.

Audio follows as binary messages: the caller's voice as 16-bit little-endian PCM, mono, 16 kHz. Each message is a short slice of audio; don't rely on a fixed message size.

stop is sent once, when the call ends for any reason, just before Skysay closes the connection:

{"event": "stop", "reason": "remote_party_disconnected"}

reason is the call's ending reason, the same value as ended_reason in Post-call results. It is empty when the call failed without one.

Messages from your agent

• Audio: binary messages in the same format, 16-bit little-endian PCM, mono, 16 kHz. A message can carry any length of audio up to one second (32,000 bytes); a larger message closes the connection. Skysay plays the audio in order as it arrives and holds at most 10 seconds that hasn't been played yet: past that, the oldest audio is dropped.
• {"event": "clear"}: drops the audio you've sent that hasn't been played yet. Send it when the caller starts talking over your agent.
• {"event": "hangup"}: ends the call. Sent after start, Skysay hangs up and the call ends with ended_reason agent_completed. Audio and clear sent before start are ignored.

Skysay ignores any other text message, so a newer agent never breaks an older call.

How the call ends

Skysay sends stop in every case where the connection is still open.

What else applies

• Recording. Skysay records both sides of the call, as for every call, and the call's webhook events fire as usual. Skysay doesn't transcribe the call: your agent owns the conversation.
• Billing. You pay the carrier minutes of the call: for an incoming call, your number's per-minute rate for incoming calls; for an outgoing call, Skysay's per-minute calling rate for the destination you dial. On your own SIP trunk your carrier bills the minutes, and the call costs a flat $0.05 per minute in either direction, in full minutes. See Billing. No AI components are billed, because your agent runs them.
• Not in this version. Tool calls, transcripts and keypad tones don't travel over the connection.

A minimal echo agent

This agent, using the Python websockets library, refuses an unsigned upgrade, then plays back whatever it hears:

import asyncio, hashlib, hmac, json, os, time
import websockets

SECRET = os.environ["SKYSAY_SIGNING_SECRET"].encode()

def verified(headers):
    call_id = headers.get("X-Skysay-Call-Id", "")
    timestamp = headers.get("X-Agent-Telco-Timestamp", "")
    given = headers.get("X-Agent-Telco-Signature", "").removeprefix("sha256=")
    expected = hmac.new(SECRET, f"{timestamp}.{call_id}".encode(), hashlib.sha256).hexdigest()
    return timestamp.isdigit() and abs(time.time() - int(timestamp)) <= 300 \
        and hmac.compare_digest(expected, given)

async def refuse_unsigned(connection, request):
    if not verified(request.headers):
        return connection.respond(401, "bad signature\n")

async def call(ws):
    start = json.loads(await ws.recv())
    if start.get("call_id") != ws.request.headers.get("X-Skysay-Call-Id"):
        await ws.close()
        return
    async for message in ws:
        if isinstance(message, bytes):
            await ws.send(message)          # echo the caller's audio back
        elif json.loads(message).get("event") == "stop":
            break

async def main():
    # Serve this behind TLS: Skysay connects to wss:// only.
    async with websockets.serve(call, "0.0.0.0", 8080, process_request=refuse_unsigned):
        await asyncio.Future()

asyncio.run(main())