Voicemail detection

The phone network does not tell the caller when a call goes to voicemail. A voicemail box answers the line exactly as a person does, so the only way to know is to listen to the first seconds after the answer. Skysay does that on every outbound call, for every AI agent and every conversation engine, and records what it concluded.

It runs around the AI agent, not through it: detection is not part of the agent's prompt or its workflow, and the agent never decides on its own that it reached a voicemail box.

Unsure means a person

A wrong "voicemail" verdict is worse than a missed one. Hanging up on a person is an abandoned call, and in some countries that is regulated. So the rule is lopsided on purpose:

• only evidence that is specific to a mailbox counts as proof;
• everything else only raises suspicion, and suspicion never acts alone;
• any sign of a person wins, and a call Skysay is not sure about is treated as a person.

How a mailbox is recognised

Proof: the only evidence that can make the agent act

There is one kind of proof, the same in every language, and nothing else counts:

• A record beep after a spoken greeting. A single, clean tone that follows a greeting which was recognised as speech with at least three words in it. Words are counted by the spaces and punctuation between them, so in Chinese, Japanese, Thai and other scripts written without spaces a short greeting is undercounted. A short greeting the language model described below reads as a voicemail greeting counts instead, when that reading arrives before the beep is judged, a few seconds after the beep. A tone that repeats, a keypad tone, a fax tone or a network information tone is not a record beep. Neither is a tone that overlaps the AI agent's own voice, or a tone after music or silence rather than speech.

A mailbox instruction such as "please leave a message after the tone" is not proof on its own, in any language: a person can say it too, and in the call records we checked it was sometimes said while a person was still on the line. It is shown in the call's evidence as suspicion.

Suspicion: signals that never act alone

• the call being answered unusually fast;
• the shape of the greeting: a long greeting that never pauses for a reply;
• music or tones with no speech in them, such as a caller tune;
• a beep after music or silence rather than after speech. A caller tune followed by network beeps sounds much like a mailbox, and often is not one;
• a mailbox instruction such as "leave a message after the tone". A person talking about a message ("did you leave me a message?", "please don't leave a message") is not even that;
• the far end's words read as a voicemail greeting. A language model reads what the far end says in the first moments of the call, in any language, and says whether it sounds like a voicemail greeting. A reading in a language the call's transcription is known to handle unreliably is ignored. On its own this is only suspicion, because a person can say the same words. Together with a record beep after it, it can be proof (above).

Recognising known mailbox recordings by their sound, so-called fingerprints, is planned and is not used today.

Phone menus, call screening services and "the number you have dialled is not available" announcements are recognised as their own kinds of answer. They are never treated as a mailbox by themselves.

Our own voice is excluded. When the AI agent's words come back down the line, for example from a speakerphone, they are recognised as echo and never count as evidence.

Languages

Detection works the same way in every language, including Sinhala and Tamil: the proof is the beep, and the language model that reads the far end's words is not limited to a list of languages.

The four settings

answer_detection.voicemail is set per agent, in the When a voicemail answers card of the agent editor or through PATCH /v1/agents/{agent_id}/voice-stack.

shadow is what an agent with no stored value gets. As with non-human audio, a value you send is stored even when it equals the default, so an agent you explicitly set to shadow keeps it if the default for new agents changes later.

curl -X PATCH https://api.skysay.ai/v1/agents/agt_123/voice-stack \
  -H "Authorization: Bearer $SKYSAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"answer_detection": {"non_human_audio": "shadow", "voicemail": "hang_up"}}'

The answer_detection object is replaced as a whole on save, so send every key you want to keep.

Messages for Leave a message, one per language

answer_detection.voicemail_messages holds the messages left when the setting is leave_message, one per language, keyed by language code:

{
  "answer_detection": {
    "voicemail": "leave_message",
    "voicemail_messages": {
      "en": "Hello, this is Example Lending. Please call us back at the number shown. Thank you.",
      "et": "Tere, helistab Example Lending. Palun helistage meile tagasi kuvatud numbril. Täname."
    }
  }
}

• The message played is the one for the language the call is actually in. A call in a language with no message is ended without one; it is never given a message in another language.
• leave_message needs a message for at least one language. In the editor there is one box per language the agent speaks.
• Each message is plain text of at most 400 characters. A locale such as de-DE is stored as de, and an empty text removes that language's message.
• Messages are kept when you switch to another setting, and when you remove a language from the agent, so switching back loses nothing.

A message is played exactly as approved, never improvised by the AI agent, and it is the same on every conversation engine. Each language's message is approved on its own. Keep them neutral: who is calling and how to call back, with no account, amount or other personal details, because a voicemail box can be heard by someone other than the person you meant to reach. A saved message is not played by any call today; see the note above.

What happens on a call today

With shadow, and with leave_message until it is released:

• the call runs exactly as it would without detection. The agent's opening, its conversation, its workflow and how the call ends are unchanged;
• nothing about detection appears in the call's answer_classification, ended_reason, webhooks or campaign results;
• the verdict and the evidence behind it are recorded with the call.

With hang_up, on proof of a mailbox:

• the call is ended without a message once the beep is confirmed as a single beep and the line has then stayed quiet, about six and a half seconds after the beep, on every conversation engine and on calls that run a workflow;
• it ends with ended_reason: no_human_after_answer and reason_detail: recorded_message_only: a recording and nothing else answered. The same pair is used when only recordings were heard on a call navigating a phone menu; the call's voicemail evidence says which;
• its answer_classification is unknown, not human: a mailbox greeting is words, but no person answered;
• a person heard before the proof, any sound on the line after the beep (a quiet reply included), audio after the beep that did not arrive, an answer the phone network did not confirm, or a call waiting in a phone menu or a queue keeps the call going;
• anything the agent was saying stops with the call, so a mailbox can record the start of the agent's opening before the call ends;
• a beep heard while the agent is speaking is never taken as proof, so a hang_up agent does not act when its opening covers the beep. Today the opening starts once the person or mailbox pauses for about 0.6 to 0.8 seconds, which on a mailbox whose greeting pauses, or beeps right after it ends, is often while the beep sounds; those calls continue as with shadow. Holding the opening until a suspected mailbox has beeped is planned;
• retries stay with your campaign's retry rule, and minutes up to the end of the call are billed as usual.

Reading the evidence

Watch only exists so that you can see, call by call, what detection would have done before it is allowed to do it. Open a call in Calls. An outbound call that detection ran on shows a Voicemail detection block:

• the verdict: Voicemail, Voicemail? (suspicion only), Person, Not sure, Phone menu or queue, Call screener, or Unavailable announcement;
• the setting you chose and the one that ran, for example "Hang up — on", or "Set to Leave a message — watching only until the accuracy check";
• what the agent did, or would have done, and when. A Hang up call that ended on the proof says "Hung up at … s: a voicemail answered"; everything that did not act is marked "not applied". For an agent on leave_message this is when the message would have started, measured from the end of the beep;
• whether a person was heard after that moment. This is the case the accuracy check exists to catch: a call where acting would have cut somebody off;
• each piece of evidence, marked as proof, suspicion, a sign of a person, or ignored, with when it was heard. A mailbox phrase that was set aside says why, for example that a phone menu was heard right around it.

Inbound calls, and calls from before this feature, show no block. That means nothing was checked, not that no voicemail was found.

What comes next

leave_message acts once it is released. A voicemail outcome of its own, for example as an event a workflow can branch on, is planned; until then a Hang up call is recognised by the pair above and its voicemail evidence.