Answer detection

A carrier reports a call answered the moment it starts carrying audio. That is true of a person picking up, and equally true of a ringback replacement tune, a hold-music loop, or a line that answers into silence. So "answered" on its own cannot tell you whether anybody was there.

Every answered call therefore carries two separate facts:

• its status, which says the transport answered, and
• its answer_classification, which says what was heard afterwards.

What the classification says

The last two are not the same thing, and the difference is worth keeping. unknown means detection ran and could not say. The empty string means nothing was reported: the call was placed before this field existed, the agent has collection switched off, or the call never carried media.

reason_detail qualifies the classification and nothing else:

It is not ended_reason. That field says why the call ended; this pair says what was heard when it started. A call can carry both.

Where the fields appear

Both fields are on the call resource returned by GET /v1/calls and GET /v1/calls/{call_id}, on each call in call group results, and in the call.completed webhook body. They are also on the call rows in your workspace.

{
  "call_id": "call_...",
  "status": "completed",
  "answer_classification": "non_human_audio",
  "reason_detail": "hold_music"
}

That is a call that ran its course against hold music. Whether a call like it is ended early is the subject of the next two sections, and it depends on one per-agent setting.

What counts as a person

A person is heard when speech starts and then stops. Concretely, either a burst of speech-shaped audio that is followed by a pause, or a recognised word whose audio has come to a stop. The pause is the part that matters: it is what separates somebody saying "hello" from a recording that never breaks.

The reverse verdict is deliberately harder to reach. Skysay only calls audio non-human when it runs continuously for several seconds with no pause in it, its shape lacks the rhythm of speech, and no word is recognised in that window. Anything short of all three is unknown.

That narrowness is on purpose, and it has a stated cost in both directions:

• A recorded announcement is treated as a person. Announcements and menu prompts have pauses in them, so they read as human or unknown and are left alone. Recognising a voicemail box is a separate detector with its own setting: see Voicemail detection.
• Somebody talking over music that never stops is missed. Their words never land in a pause, so they are not counted, and the call is classified on the audio's shape alone.

Detection runs on the audio itself, in every language. It does not depend on a transcript being available, and it never stores audio: only the verdict, its score and the timings are recorded.

The ending it is paired with

ended_reason has a value, no_human_after_answer, for a call the platform ended because only non-human audio was ever heard after pickup. A call that ends that way carries answer_classification non_human_audio and the detail naming which kind, so the reason says that nobody was there and the classification says what was heard instead.

An agent set to end_call ends a call this way in two situations and no others. An agent left on shadow, which is every agent that has not been changed, still ends nothing.

A flat, unbroken tone or tune. The audio runs continuously with no pause in it, its shape lacks the rhythm of speech, and no word is recognised in the window. That is the non_human_audio verdict described above, and it ends the call on the same forty-five-second clock as the rule below.

Nobody could have replied. The far end holds the line for twenty seconds without once leaving a gap of a second, so there has been no moment in which anybody could have said anything back. This rule does not ask what the audio sounds like, and that is the whole point of it: the caller tunes that prompted this work are songs. A singer's voice has the rhythm of speech because it is speech, and a recogniser turns the lyrics into words. What separates them from a conversation is not how they sound. It is that a conversation alternates and they never do.

A person stops the second rule by pausing, which anybody talking to somebody else does within a second or two. The pause is the release, so nothing has to recognise them first and the call simply carries on. They are not greeted again and the call is not restarted; the agent picks up from where it is.

Neither rule ends a call in its first forty-five seconds. Both silence the agent the moment they stand, and a pause before that point releases the rule and the silence together.

On the four caller tunes measured below, that grace period never came into it. A song stops between phrases and between tracks, so the far end keeps yielding the floor and starting again; each of those calls produces between twenty-two and thirty-three separate runs of audio. The first unbroken twenty seconds does not complete until between fifty-one and seventy-five seconds in, long after the forty-five is up, so each call goes quiet and ends on the same tick.

While either rule is standing, the agent stops speaking from its next turn onward. The sentence already playing is not cut off. If the far end turns out to be a person, the agent's voice comes back on the same pause that keeps the call alive.

Silence and the ending are the whole of what end_call does. It does not hold the call's own figures back from the agent. Withholding them until a person had been recognised was built alongside this and removed before release, because it did not work: a caller tune stops between phrases and between tracks, so the signal meant to say "somebody is there" is produced by the music itself within seconds. The figures are also part of the agent's setup instructions from before the phone is picked up, so holding back what it is handed mid-call was never a guarantee in any case.

Why twenty seconds, and what it catches

Measured on every completed call of the batch that prompted this work: seventeen calls, four of them answered by a caller tune, replayed through the same voice-activity detection a live call runs on.

Twenty seconds sits between the two, and nearer the tunes than the margin suggests: it is more than twice the longest hold any of the other thirteen calls produced. All four tunes are ended and none of the thirteen is touched.

The side of that gap the line sits on is a real choice, not a rounding. A person who holds the floor for twenty unbroken seconds, never once leaving a gap of a second, is silenced and then ended. Nobody in this batch came within eleven seconds of that, but seventeen calls in one language is not a promise about every caller. Run an agent on shadow first and read what it records for your own traffic before you move it to end_call.

Two of the thirteen are only seconds long and carry nothing that says what answered them, so they are labelled as neither; they are counted with the thirteen because the rule leaves them alone either way.

One more limit worth stating. A real person on a call that already triggered the rule is spoken to again as soon as they pause, because the pause is the release; the call is not restarted and they are not greeted a second time.

A call that ends this way is reported as not connected:

• it is excluded from a campaign's connect rate, and from talk time;
• its campaign disposition is no_answer, because no person was reached, so the contact is dialled again under your campaign's existing backoff rather than being treated as a conversation that happened;
• its termination_cause stays connected, because that field describes the transport and the transport genuinely did answer. The two are separate questions and the fields answer one each;
• carrier minutes up to the point it ends are real and are billed as usual.

Choose what an agent collects

answer_detection.non_human_audio is set per agent, in the Answer detection card of the agent editor or through PATCH /v1/agents/{agent_id}/voice-stack.

shadow is what an agent with no stored value gets, and it is the only value that can measure how often the verdict would have been wrong without being able to end anything. Move an agent to end_call once you have looked at what shadow recorded for it.

curl -X PATCH https://api.skysay.ai/v1/agents/agt_123/voice-stack \
  -H "Authorization: Bearer $SKYSAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"answer_detection": {"non_human_audio": "off"}}'

Choosing a value is not the same as leaving it alone

This setting deliberately behaves differently from the other per-agent tuning sections. Everywhere else, saving a value equal to the current default is the same as saving nothing. Here it is not:

• an absent answer_detection key means "follow the platform default", which is shadow today and is expected to change once the detector has been measured;
• an explicit value is stored as you sent it, including shadow, and the agent keeps it when the platform default moves.

So if you want an agent to observe and never do anything else, send shadow rather than leaving the section out. The editor makes the same distinction: the first choice is Platform default, and Shadow beside it is the explicit one.

The setting applies when a call starts; saving it never changes a call already in progress.

shadow is the platform default on purpose, and end_call is a choice you make per agent rather than something that arrives underneath you.

Phone menus and hold queues

An outbound call to a company often reaches an automated menu ("press 2 for English") and then a hold queue, not a person. Every AI agent has navigation on by default: on the calls it places, it answers the menu, waits in the queue, and talks when a person picks up. Inbound calls (the editor's phone test included), browser tests, Simulations Lab, chat and SMS never navigate. The whole feature is described in Phone menu navigation.

While a call is in a menu or on hold, the non-human-audio ending above does not end it, even on an AI agent set to end_call: hold music is what a queue sounds like. If an AI agent must hang up on hold music, turn navigation off for it.

You turn navigation off, or change its limits, per AI agent with the navigation section of PATCH /v1/agents/{agent_id}/voice-stack (MCP tool set_agent_voice_stack):

curl -X PATCH https://api.skysay.ai/v1/agents/agt_123/voice-stack \
  -H "Authorization: Bearer $SKYSAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"navigation": {"enabled": null, "max_hold_seconds": 900}}'

An empty object, "navigation": null, or a section without enabled is refused with 400 navigation_enabled_required. The section is replaced whole: a limit you leave out goes back to its default. A limit kept beside enabled: false is kept as it is. The section the GET below returns can be sent back as it is: its read-only fields effective, eligible and ineligible_reason are accepted and ignored on write, and any other unknown field is refused with 400.

The two limits belong to the agent and to the whole call: time in menus and time on hold are counted separately, never start over when the call goes back to a menu, and a single call cannot change them. The call also ends at its overall maximum length (turn_taking.max_call_seconds). GET /v1/agents/{agent_id}/voice-stack always returns navigation: the stored enabled, whether it is effective, both limits with their defaults applied, and whether the AI agent's outbound calls may navigate (eligible, with ineligible_reason when they may not; see Read what an AI agent will do). Like the other sections, it applies when a call starts; saving never changes a call in progress.

Telling the agent what the menu expects

If you know the menu, say so. Send navigation_hint in the call's call_context (or as a column of a campaign contact): plain text of at most 500 characters, for example Press 2 for English, then 1 for billing. It reaches the agent as the built-in variable call.navigation_hint (see Call variables). A longer hint, or one that is not text, refuses the call with call_variables_invalid. A hint is only a hint: the agent still answers the menu it actually hears.

Typing numbers into a menu

A menu sometimes asks for a number the call knows: an account number, a date of birth. Declare that value as a keypad variable: the agent can have it keyed on the menu, and it is never shown to the agent as text.

Languages

There is no separate list of menu languages: navigation works in each of an agent's languages that a transcriber Skysay can use for phone menus listens in.

To follow a menu or a queue the agent listens to the far end on its own speech-to-text connection for as long as the call is in a menu or on hold, using those transcribers. It uses at most two connections at once, chosen from your agent's selected languages:

• An agent with one language gets one connection.
• An agent with several languages gets one connection when a single transcriber can listen for the whole set at once, and otherwise one connection per language, up to two.
• If the set cannot be covered that way, nothing is refused: the AI agent's calls run without navigation, and GET /v1/agents/{agent_id}/voice-stack reports navigation.eligible false with ineligible_reason languages_uncovered (no transcriber hears one of its languages) or too_many_sockets (every language is covered, but not by two connections), and the languages in ineligible_languages. Remove a language to use navigation.

A transcriber measured to fail on phone menus in a language is never used for it, and an agent whose own transcriber failed in one of its languages does not navigate (ineligible_reason transcriber_unverified; today Deepgram Nova-3 in Estonian).

The agent answers menus in its own languages only: a menu in another language is not answered, by keys or by speech. Add the language to the agent if its calls should navigate such menus.

Numbers read out in words need a word list per language. Skysay has one for English, German, French, Spanish, Italian, Dutch, Portuguese, Russian, Estonian and Finnish, and gives keypad values in all of them except Estonian and Finnish (not yet tested end to end). An agent with any other language, such as Polish, or with Estonian or Finnish, navigates its menus but cannot enter keypad values yet: a value is refused with value_language_unsupported and the menus are still navigated. Digits are recognised in every language. For a language with no list Skysay does not guess: it treats spoken numbers it cannot place as present, and holds back the words instead of risking a number you keyed in.

How well the agent understands a menu is measured language by language on a synthetic corpus before it is claimed; a language that works on keys can still have a menu the agent misreads. Hold times and menu times are limited by the two settings above in every language.