> ## Documentation Index
> Fetch the complete documentation index at: https://docs.medisync.me/llms.txt
> Use this file to discover all available pages before exploring further.

# Live Speech API

> Real-time, low-latency speech-to-text over a single authenticated WebSocket, with optional speaker separation

## Overview

The **Live Speech API** turns a live audio stream into text in real time over a single, authenticated WebSocket. You open one connection, send a short JSON **config** handshake, then stream raw audio frames. The server replies with **transcript** frames as each segment of speech is finalized — each one carrying precise word-level timings and, optionally, per-speaker turns.

The API is designed for clinical dictation and conversation capture, so it is built around three guarantees: **medical-grade accuracy**, a **continuous absolute timeline** that survives reconnects, and **strict privacy** (audio is never written to disk and transcript text is never logged).

<Info>
  **One socket, one session.** Everything — config, audio, control, transcripts, acknowledgements and errors — flows over a single WebSocket. There is nothing else to poll.
</Info>

## How it works

<Steps>
  <Step title="Connect & authenticate">
    Open a WebSocket to the endpoint with a valid JWT passed as the `token` query parameter. The token is verified during the handshake.
  </Step>

  <Step title="Send a config handshake">
    Your **first** message is a single JSON `config` frame that declares your audio format, an optional language hint, and how you want speech chunked.
  </Step>

  <Step title="Receive session_ready">
    The server echoes the **effective** (validated and clamped) configuration and a `session_id`. You may now start streaming.
  </Step>

  <Step title="Stream audio">
    Send raw audio as **binary** frames of any length. Use JSON control frames to flush (`end_utterance`) or end (`close`) the session.
  </Step>

  <Step title="Receive transcripts">
    For each finalized segment the server sends a `transcript` frame with `text`, word timings, and a `processed_until` high-water mark.
  </Step>
</Steps>

## Key features

<CardGroup cols={2}>
  <Card title="Real-time captions" icon="bolt">
    Finalized segments arrive as you speak. Tune latency vs. density with a single `chunk.strategy` setting.
  </Card>

  <Card title="Word-level timings" icon="stopwatch">
    Every transcript carries per-word start/end times on one absolute timeline.
  </Card>

  <Card title="Speaker separation" icon="users">
    Enable diarization to receive per-speaker turns (for example clinician vs. patient).
  </Card>

  <Card title="Resilient reconnects" icon="rotate">
    `processed_until` and `timeline_offset_seconds` let a dropped connection resume with no lost transcript and one continuous timeline.
  </Card>

  <Card title="Multilingual" icon="language">
    Automatic language detection across a wide set of European languages, including mid-conversation switches.
  </Card>

  <Card title="JWT-secured" icon="shield-check">
    Every connection is authenticated at the handshake with your JWT.
  </Card>
</CardGroup>

## Connection at a glance

**Endpoint**

```
wss://speech.medisync.me/api/v2/ws/transcribe?token=<JWT>
```

| Property         | Value                                                                         |
| ---------------- | ----------------------------------------------------------------------------- |
| Transport        | WebSocket (WSS, TLS)                                                          |
| Authentication   | JWT in the `token` query parameter (verified at handshake)                    |
| First message    | JSON `config` frame                                                           |
| Audio            | Binary frames — raw 16-bit PCM (`pcm_s16le`) or `wav`                         |
| Server messages  | JSON: `session_ready`, `transcript`, `ack`, `backpressure`, `status`, `error` |
| Protocol version | `2.0`                                                                         |

## Message flow

```mermaid theme={null}
sequenceDiagram
    participant Client
    participant Server
    Client->>Server: Connect (wss ... ?token=<JWT>)
    Server-->>Client: (accepts, or closes 1008 on auth failure)
    Client->>Server: { "type": "config", ... }
    Server-->>Client: { "type": "session_ready", ... }
    Client->>Server: binary audio frames
    Server-->>Client: { "type": "transcript", ... }
    Client->>Server: { "type": "end_utterance" }
    Server-->>Client: { "type": "ack", "processed_until": ... }
    Client->>Server: { "type": "close" }
```

## One session per user

Each user may have **one** active live session at a time. If a second connection is opened while a session is still active, it is rejected with an `error` frame of `code: "session_conflict"` and the socket is closed. Close the previous socket before reconnecting.

<Note>
  This rule is what makes reconnection safe: because only one session can be live, a reconnecting client always resumes a single, unambiguous timeline. See [Reconnection](/websocket/reconnection).
</Note>

## Privacy

<Warning>
  Transcript text travels **only** over your authenticated socket. The service **never writes audio to disk** and **never logs transcript text or PII**. Nothing you stream is retained after the socket closes.
</Warning>

## Next steps

<CardGroup cols={2}>
  <Card title="Connection" icon="plug" href="/websocket/connection">
    Endpoint, JWT auth via query parameter, handshake sequence, and close codes.
  </Card>

  <Card title="Session config" icon="sliders" href="/websocket/session-config">
    Every config field, defaults, clamping, and the `session_ready` echo.
  </Card>

  <Card title="Messages" icon="messages" href="/websocket/messages">
    Audio frames, control frames, and the full transcript schema.
  </Card>

  <Card title="Reconnection" icon="rotate" href="/websocket/reconnection">
    Resume mid-recording with no lost transcript and one continuous timeline.
  </Card>

  <Card title="Audio requirements" icon="waveform-lines" href="/websocket/audio-requirements">
    Formats, sample rate, channels, and chunking guidance.
  </Card>

  <Card title="Errors" icon="triangle-exclamation" href="/websocket/errors">
    Error frame schema, the full code table, and fatal/close semantics.
  </Card>
</CardGroup>