> ## Documentation Index
> Fetch the complete documentation index at: https://docs.medisync.me/llms.txt
> Use this file to discover all available pages before exploring further.

# Audio Requirements

> Audio formats, sample rate, channels, and chunking guidance for the Live Speech API

## Overview

Send audio as **binary** WebSocket frames after `session_ready`. The format you stream must match what you declared in the `audio` object of your [config](/websocket/session-config#audio). Frames may be any length — the server re-frames the stream internally.

<Info>
  **Recommended:** raw 16-bit PCM (`pcm_s16le`), **16 kHz**, **mono**. This is the lowest-overhead, best-supported format.
</Info>

## Format

<CardGroup cols={2}>
  <Card title="Encoding" icon="file-audio">
    * `pcm_s16le` — raw 16-bit little-endian PCM (**preferred**)
    * `wav` — WAV container
  </Card>

  <Card title="Stream shape" icon="wave-square">
    * **Binary** frames, any length
    * Server re-frames internally
    * No client-side segment alignment needed
  </Card>
</CardGroup>

### Sample rate

<ParamField path="audio.sample_rate" type="integer" default="16000">
  Accepted range **8000–48000 Hz**. `16000` is recommended and is what most integrations should send.
</ParamField>

### Channels

<ParamField path="audio.channels" type="integer" default="1">
  `1` (mono, recommended) or `2`. Mono keeps bandwidth and processing minimal; use it unless you have a specific reason not to.
</ParamField>

| Setting     | Recommended | Accepted           |
| ----------- | ----------- | ------------------ |
| Encoding    | `pcm_s16le` | `pcm_s16le`, `wav` |
| Sample rate | `16000` Hz  | `8000`–`48000` Hz  |
| Channels    | `1` (mono)  | `1` or `2`         |
| Bit depth   | 16-bit      | 16-bit             |

<Note>
  For `pcm_s16le`, send interleaved signed 16-bit samples with **no** WAV header. For `wav`, the container header is read from the stream.
</Note>

## Chunking guidance

You do **not** chunk on segment boundaries — the server decides where segments end. Two independent things are worth getting right:

1. **How often you send binary frames** (transport pacing).
2. **Which `chunk.strategy` you choose** (how the server finalizes segments).

### Transport pacing

Send audio at roughly **real time**, in small frames. Frames of about **20–100 ms** of audio (for example \~640–3200 bytes at 16 kHz mono `pcm_s16le`) keep captions responsive without flooding the socket.

<Warning>
  Do not dump a long recording as fast as possible. Sending far ahead of real time can trigger a [`backpressure`](/websocket/messages#backpressure) signal and, if sustained, dropped segments. Pace your sends to the audio's own duration.
</Warning>

### Choosing a strategy

The `chunk.strategy` trades latency against accuracy and text density. Pick by behavior, not by frame size:

<CardGroup cols={3}>
  <Card title="low_latency" icon="gauge-high">
    Shortest segments, fastest captions. Best for live on-screen dictation.
  </Card>

  <Card title="balanced" icon="scale-balanced">
    The default. Strong accuracy while still feeling live.
  </Card>

  <Card title="high_throughput" icon="layer-group">
    Longest segments, highest accuracy and density. Best when a little extra delay is acceptable.
  </Card>
</CardGroup>

<Tip>
  Silence trimming and pause-based segmentation are automatic. You can send continuous audio (including silence) and let the server decide where each segment ends.
</Tip>

## Browser capture example

<CodeGroup>
  ```javascript JavaScript theme={null}
  // Capture mic audio at 16 kHz mono and stream it as pcm_s16le binary frames.
  async function startMicrophone(ws) {
    const stream = await navigator.mediaDevices.getUserMedia({
      audio: {
        sampleRate: 16000,
        channelCount: 1,
        echoCancellation: true,
        noiseSuppression: true,
      },
    });
    const ctx = new AudioContext({ sampleRate: 16000 });
    const source = ctx.createMediaStreamSource(stream);
    const node = ctx.createScriptProcessor(2048, 1, 1); // ~128 ms per callback

    node.onaudioprocess = (e) => {
      const f32 = e.inputBuffer.getChannelData(0);
      const pcm = new Int16Array(f32.length);
      for (let i = 0; i < f32.length; i++) {
        pcm[i] = Math.max(-1, Math.min(1, f32[i])) * 0x7fff; // float32 -> int16
      }
      if (ws.readyState === WebSocket.OPEN) ws.send(pcm.buffer);
    };

    source.connect(node);
    node.connect(ctx.destination);
  }
  ```

  ```python Python theme={null}
  import asyncio
  import numpy as np
  import sounddevice as sd

  async def stream_microphone(ws, sample_rate=16000, block_ms=40):
      """Capture 16 kHz mono audio and send it as pcm_s16le binary frames."""
      block = int(sample_rate * block_ms / 1000)
      loop = asyncio.get_running_loop()
      queue: asyncio.Queue = asyncio.Queue()

      def callback(indata, frames, time_info, status):
          pcm = (np.clip(indata[:, 0], -1.0, 1.0) * 32767).astype("<i2").tobytes()
          loop.call_soon_threadsafe(queue.put_nowait, pcm)

      with sd.InputStream(samplerate=sample_rate, channels=1,
                          dtype="float32", blocksize=block, callback=callback):
          while True:
              await ws.send(await queue.get())   # binary frame, ~40 ms of audio
  ```
</CodeGroup>

## Streaming a file

To transcribe a pre-recorded file, decode it to 16 kHz mono `pcm_s16le` and send it **paced to real time** (or slightly faster), watching for `backpressure`. For whole-file, non-real-time transcription, use the batch transcription flow instead of this live socket.

<AccordionGroup>
  <Accordion title="No transcripts arriving">
    * Confirm you waited for `session_ready` before sending audio.
    * Confirm the bytes match your declared `encoding`, `sample_rate`, and `channels`.
    * Confirm you are sending **binary** frames (not base64 or JSON).
    * Very short bursts of audio may not finalize until more speech (or an `end_utterance`) arrives.
  </Accordion>

  <Accordion title="Choppy or delayed captions">
    * Switch `chunk.strategy` to `low_latency` for faster segments.
    * Send smaller frames closer to real time.
    * Check for `backpressure` frames — if present, you are sending too fast.
  </Accordion>

  <Accordion title="Wrong or mixed language">
    * Provide a `language` hint in your config to bias detection.
    * Detection is automatic and per-segment; mid-conversation switches are expected and handled.
  </Accordion>
</AccordionGroup>

## Next steps

<CardGroup cols={2}>
  <Card title="Messages" icon="messages" href="/websocket/messages">
    Send audio and control frames; read transcripts.
  </Card>

  <Card title="Session config" icon="sliders" href="/websocket/session-config">
    Declare your audio format and chunk strategy.
  </Card>
</CardGroup>