Appearance

Event Types ​

Complete reference for all events in the SDK.

Overview ​

Events are triggered by the Flow service and handled by your assistant. All events include session information and are typed with TypeScript.

Base Event Structure ​

All events extend a base structure with session information:

typescript
interface SessionInfo {
  id: string;              // UUID of the session
  account_id: string;      // Account identifier
  phone_number: string;    // Phone number for this flow session
  direction?: "inbound" | "outbound";
  from_phone_number: string;
  to_phone_number: string;
}

Event Types ​

SessionStart Event ​

Triggered when a new call session begins — and when a failed transfer returns the caller to your assistant.

typescript
interface AiFlowEventSessionStart {
  type: "session_start";
  session: {
    id: string; // UUID of the session
    account_id: string; // Account identifier
    phone_number: string; // Phone number for this flow session
    direction?: "inbound" | "outbound"; // Call direction
    from_phone_number: string; // Phone number of the caller
    to_phone_number: string; // Phone number of the callee
  };
  /** Set only when a failed transfer returned the caller — absent on a new call. */
  transfer_failed_reason?: TransferFailedReason;
}

enum TransferFailedReason {
  BUSY = "busy",
  REJECTED = "rejected",
  NO_ANSWER = "no_answer",
  NUMBER_NOT_FOUND = "number_not_found",
  TECHNICAL_ERROR = "technical_error",
  UNKNOWN = "unknown",
}

Example:

typescript
onSessionStart: async (event) => {
  // The caller is back after a transfer that did not connect
  if (event.transfer_failed_reason) {
    return event.transfer_failed_reason === TransferFailedReason.BUSY
      ? "The line is busy. Shall I try a colleague?"
      : "Sorry, I could not put you through. Let me take a message.";
  }

  // Log session details
  console.log(
    `${event.session.direction} call from ${event.session.from_phone_number} to ${event.session.to_phone_number}`
  );

  // Return greeting
  return "Welcome to our service!";
};

transfer_failed_reason is only reported for transfers that set a timeout (transfer fallback) — see the Transfer action.

UserSpeechStarted Event ​

Triggered when the user's speech is first detected, before the full transcript is available. Uses Voice Activity Detection (VAD) and fires 20–120 ms after the user starts speaking.

WebSocket only — this event is not delivered to HTTP webhook handlers.

typescript
interface AiFlowEventUserSpeechStarted {
  type: "user_speech_started";
  session: SessionInfo;
}

Notes:

  • Fires at most once per speech turn; resets after user_speak is received
  • No return value is expected; returning an action has no effect

Example:

typescript
onUserSpeechStarted: async (event) => {
  console.log('User started speaking, session:', event.session.id);
  // No return value needed
},

UserSpeak Event ​

Triggered when the user speaks and speech-to-text completes.

typescript
interface AiFlowEventUserSpeak {
  type: "user_speak";
  text: string; // Recognized speech text
  session: SessionInfo;
}

Example:

typescript
onUserSpeak: async (event) => {
  const intent = analyzeIntent(event.text);

  if (intent === "help") {
    return "I can help you with billing, support, or sales.";
  }

  return processUserInput(event.text);
};

AssistantSpeak Event ​

Triggered after the assistant starts speaking. Event may be omitted for some text-to-speech models.

typescript
interface AiFlowEventAssistantSpeak {
  type: "assistant_speak";
  text?: string; // Text that was spoken
  ssml?: string; // SSML that was used (if applicable)
  duration_ms: number; // Duration of speech in milliseconds
  speech_started_at: number; // Unix timestamp (ms) when speech started
  session: SessionInfo;
}

Example:

typescript
onAssistantSpeak: async (event) => {
  console.log(`Spoke for ${event.duration_ms}ms`);

  // Track conversation metrics
  trackMetrics({
    sessionId: event.session.id,
    duration: event.duration_ms,
    text: event.text,
  });
};

AssistantSpeechEnded Event ​

Triggered after the assistant finishes speaking.

typescript
interface AiFlowEventAssistantSpeechEnded {
  type: "assistant_speech_ended";
  session: SessionInfo;
}

Example:

typescript
onAssistantSpeechEnded: async (event) => {
  console.log(`Finished speaking for session ${event.session.id}`);

  // Trigger next action if needed
  await triggerNextAction(event.session.id);
};

UserInputTimeout Event ​

Triggered when no user speech is detected within the configured timeout period after the assistant finishes speaking.

typescript
interface AiFlowEventUserInputTimeout {
  type: "user_input_timeout";
  session: SessionInfo;
}

When Triggered:

  1. A speak action includes a user_input_timeout_seconds field
  2. The assistant finishes speaking (assistant_speech_ended event fires)
  3. The specified timeout period elapses without any user speech detected

Example:

typescript
onUserInputTimeout: async (event) => {
  console.log(`No user input received for session ${event.session.id}`);

  // Retry the question
  return {
    type: "speak",
    session_id: event.session.id,
    text: "Are you still there? Please say yes or no.",
    user_input_timeout_seconds: 5
  };
};

Configuring Timeout:

Set user_input_timeout_seconds in the speak action:

typescript
onSessionStart: async (event) => {
  return {
    type: "speak",
    session_id: event.session.id,
    text: "What is your account number?",
    user_input_timeout_seconds: 5  // Wait 5 seconds for response
  };
};

Common Use Cases:

typescript
// Hangup after multiple timeouts
const timeoutCounts = new Map<string, number>();

onUserInputTimeout: async (event) => {
  const sessionId = event.session.id;
  const count = (timeoutCounts.get(sessionId) || 0) + 1;
  timeoutCounts.set(sessionId, count);

  if (count >= 3) {
    return {
      type: "hangup",
      session_id: sessionId
    };
  }

  return {
    type: "speak",
    session_id: sessionId,
    text: `I didn't hear anything. Please respond. Attempt ${count} of 3.`,
    user_input_timeout_seconds: 5
  };
};

// Transfer to agent after timeout
onUserInputTimeout: async (event) => {
  return {
    type: "speak",
    session_id: event.session.id,
    text: "Let me connect you with a live agent who can help you."
    // Follow with transfer action
  };
};

DtmfReceived Event ​

Triggered when the user presses a key on their phone keypad.

typescript
interface AiFlowEventDtmfReceived {
  type: "dtmf_received";
  digit: string; // The key pressed: "0"–"9", "*", or "#"
  session: SessionInfo;
}

Example:

typescript
onDtmfReceived: async (event) => {
  console.log(`User pressed: ${event.digit}`);

  if (event.digit === '1') {
    return {
      type: 'transfer',
      session_id: event.session.id,
      transfer_to: '+49211100200'
    };
  }

  return {
    type: 'speak',
    session_id: event.session.id,
    text: `You pressed ${event.digit}.`
  };
},

Notes:

  • All standard DTMF tones are supported: 0–9, *, #
  • Each key press triggers a separate event
  • DTMF events can occur at any point during the call

SessionEnd Event ​

Triggered when the call session ends.

typescript
interface AiFlowEventSessionEnd {
  type: "session_end";
  session: SessionInfo;
}

Example:

typescript
onSessionEnd: async (event) => {
  // Save conversation history
  await saveConversation(event.session.id);

  // Send analytics
  await trackSessionEnd(event.session);
};

Barge-In Detection ​

User interruptions are detected via the barged_in flag in user_speak events:

typescript
interface AiFlowEventUserSpeak {
  type: "user_speak";
  text: string;
  barged_in?: boolean; // true if user interrupted
  session: SessionInfo;
}

Example:

typescript
onUserBargeIn: async (event) => {
  // Called automatically when event.barged_in === true
  console.log(`User interrupted with: ${event.text}`);
  return "I'm listening, please continue.";
};

Event Flow ​

┌─────────────────┐
│  session_start  │──> Respond with speak/audio or do nothing
└─────────────────┘

┌─────────────────┐
│   user_speak    │──> Respond with speak/audio/transfer/hangup
│  (barged_in?)   │    Check barged_in flag for interruptions
└─────────────────┘

┌─────────────────┐
│ assistant_speak │──> Optional: track metrics, trigger next action
└─────────────────┘

┌─────────────────┐
│   session_end   │──> Cleanup only, no actions accepted
└─────────────────┘

Event Summary Table ​

Event TypeTransportDescriptionWhen TriggeredCan Return Action?
session_startHTTP + WebSocketCall session beginsWhen a new call is initiated — or when a failed transfer returns the caller✅ Yes
user_speech_startedWebSocket onlySpeech onset detectedWhen VAD detects the user starting to speak❌ No
user_speakHTTP + WebSocketUser speech detectedAfter speech-to-text completes (includes barged_in flag)✅ Yes
dtmf_receivedHTTP + WebSocketDTMF digit pressedWhen the user presses a phone key✅ Yes
assistant_speakHTTP + WebSocketAssistant finished speakingAfter TTS playback completes✅ Yes
assistant_speech_endedHTTP + WebSocketAssistant finished speakingAfter speech playback ends✅ Yes
user_input_timeoutHTTP + WebSocketUser input timeout reachedWhen no speech detected after timeout✅ Yes
session_endHTTP + WebSocketCall session endsWhen the call terminates❌ No

Type Safety ​

All events are fully typed. Import types from the SDK:

typescript
import type {
  AiFlowEventSessionStart,
  AiFlowEventUserSpeechStarted,
  AiFlowEventUserSpeak,
  AiFlowEventDtmfReceived,
  AiFlowEventAssistantSpeak,
  AiFlowEventAssistantSpeechEnded,
  AiFlowEventUserInputTimeout,
  AiFlowEventSessionEnd,
  AiFlowEventUserBargeIn,
} from "@sipgate/ai-flow-sdk";

onSessionStart: async (event: AiFlowEventSessionStart) => {
  // event is fully typed
  const sessionId: string = event.session.id;
  // ...
};

Next Steps ​