WebRTC Gateway API

Enable in-browser voice conversations with PolyAI agents using WebRTC and WebSocket signaling.

The WebRTC Gateway enables real-time voice communication between a web browser and a PolyAI voice agent, using WebSocket signaling and bidirectional WebRTC audio.

It provides two integration layers:

• WebSocket signaling for session setup, SDP exchange, and ICE candidate exchange
• WebRTC media for bidirectional audio once the connection is established

Prerequisites

• A WebRTC-capable browser
• Microphone permissions enabled
• A PolyAI authentication token

Quick start

1. Open a WebSocket connection to the signaling endpoint
2. Create a WebRTC peer connection and collect microphone audio
3. Send an offer message containing SDP and your auth token
4. Receive an answer message containing SDP and a session identifier
5. Exchange ICE candidates until the connection is established
6. Audio flows bidirectionally

Signaling endpoint

Signaling URL (WebSocket):

wss://webrtc-gateway.us-1.platform.polyai.app/api/v1/webrtc/signal

All signaling messages are JSON objects sent over the WebSocket connection.

Message structure

All signaling messages follow the same top-level structure.

Field	Required	Description
type	Yes	Message type: offer, answer, ice-candidate, error, close
sessionId	Yes	Empty string when creating a new session
data	No	Message-specific payload (SDP, ICE candidate, or error)
authToken	Offer	Authentication token for the voice agent. Required for every offer, including studio draft and preview calls
mode	No	Agent mode: end-to-end (default), traditional, or echo (debug only)
callSid	No	Unique call identifier (camelCase – distinct from the outbound REST API’s call_sid)
caller	No	Calling number
callee	No	Called number
accountId	No	Account identifier
projectId	No	Project identifier
variantId	No	Optional variant override
agentVersionOverride	No	Optional { artifactVersion, lambdaDeploymentVersion } to pin a specific agent build

Message types

Offer (client to server)

Starts a new session. Send with an empty sessionId.

Example message:

{
  "type": "offer",
  "sessionId": "",
  "data": {
    "type": "offer",
    "sdp": "v=0 o=- 4611731400430051336 2 IN IP4 127.0.0.1"
  },
  "authToken": "your-auth-token",
  "callSid": "call-unique-id",
  "caller": "[phone]",
  "callee": "[phone]"
}

Answer (server to client)

Sent in response to an offer. Contains the SDP answer and the assigned sessionId.

{
  "type": "answer",
  "sessionId": "550e8400-e29b-41d4-a716-446655440000",
  "data": {
    "type": "answer",
    "sdp": "v=0 o=- 4611731400430051336 2 IN IP4 192.168.1.1"
  }
}

Store the sessionId and use it for all subsequent messages.

ICE candidate (bidirectional)

Sent by both client and server to establish network connectivity.

{
  "type": "ice-candidate",
  "sessionId": "550e8400-e29b-41d4-a716-446655440000",
  "data": {
    "candidate": "candidate:1 1 UDP 2130706431 192.168.1.1 54321 typ host",
    "sdpMid": "0",
    "sdpMLineIndex": 0
  }
}

Close (client to server)

Terminates the session gracefully.

{
  "type": "close",
  "sessionId": "550e8400-e29b-41d4-a716-446655440000"
}

Error (server to client)

Sent when an error occurs.

{
  "type": "error",
  "sessionId": "550e8400-e29b-41d4-a716-446655440000",
  "data": {
    "code": "UNAUTHORIZED",
    "message": "Invalid authentication token"
  }
}

Error codes:

Code	Description
UNAUTHORIZED	Invalid or missing authentication token
INVALID_ARGUMENT	Request field has an invalid value (for example, an unknown mode)
INVALID_MESSAGE	Malformed or unsupported message format
HANDLER_ERROR	Error processing the signaling message
MEDIA_BRIDGE_FAILURE	Failed to establish media connection
AGENT_FAILURE	Error connecting to the PolyAI agent

WebRTC configuration

Audio codec

The gateway requires Opus audio.

• MIME type: audio/opus
• Sample rate: 48 kHz
• Channels: stereo

ICE servers

Configure your peer connection with STUN servers, and TURN servers if you expect clients on restrictive networks.

You can hardcode public STUN servers (for example, stun:stun.l.google.com:19302), but for TURN we recommend fetching credentials at connection time from the GET /api/v1/ice-servers endpoint. The gateway issues fresh ephemeral credentials on each request and returns turns: URLs on port 443 so clients behind corporate firewalls that block UDP can still relay media over TLS.

Fetch the configuration before creating the peer connection:

const res = await fetch(
  "https://webrtc-gateway.us-1.platform.polyai.app/api/v1/ice-servers",
  { headers: { Authorization: `Bearer ${authToken}` } },
);
const { iceServers } = await res.json();

const pc = new RTCPeerConnection({ iceServers });

Browser support

• Chrome 72 or newer
• Firefox 60 or newer
• Safari 14.1 or newer
• Edge 79 or newer

Troubleshooting

Unauthorized error

Ensure the authentication token is valid and included in the offer message. Every offer requires an authToken, including studio draft and preview calls that pin a specific build with agentVersionOverride. Offers without a token are rejected with an UNAUTHORIZED error and the message Auth token required.

No audio

• Confirm microphone permissions are granted
• Verify Opus is negotiated successfully

ICE connection fails

• Corporate firewalls may require TURN. Fetch the gateway’s TURN configuration from GET /api/v1/ice-servers and pass it to your RTCPeerConnection.
• Ensure UDP traffic is allowed where possible.
• The iceServers response includes a turns: URL on port 443 (TURN over TLS), which works on networks that block UDP and most non-HTTPS TCP traffic.

Useful links

• WebRTC overview — Getting started with WebRTC
• MDN WebRTC API — Browser API reference
• MDN WebSocket API — WebSocket reference
• RFC 8866 (SDP) — Session Description Protocol specification
• RFC 8445 (ICE) — Interactive Connectivity Establishment specification