A network blip mid-recording should cost no transcript and produce one continuous timeline — not a second recording that restarts at 00:00. The Live Speech API gives you two primitives that make this straightforward.
The full audio is the client’s responsibility to preserve. The server keeps no audio once a socket drops. Buffer the audio you send so you can resend the tail after a reconnect.
On every transcript frame (and via ack), the absolute second up to which the server has consumed your audio. It only ever advances — even across trimmed silence.
timeline_offset_seconds
A config field. On reconnect, set it to the last processed_until you saw. The server then emits absolute timestamps from that point, so post-reconnect segments never collide with 00:00.
As transcripts (and acks) arrive, keep the latestprocessed_until. This is the durable high-water mark — everything at or below it has been safely consumed.
2
Trim your send buffer
Buffer the audio you send, timestamped on the same absolute timeline. Whenever processed_until advances, drop buffered audio at or below it — the server already has it.
3
On disconnect, open a new session
A reconnect is a new session. Wait for the old socket to fully close (see one session per user), then connect again with the same config, plus timeline_offset_seconds set to the last processed_until.
4
Resend only the tail
Wait for session_ready, then resend your buffered audio after the last processed_until, in order. Nothing the server already consumed is resent, and nothing is lost.
Because the server applies timeline_offset_seconds to every emitted timestamp, the segment, words, and speaker times after the reconnect continue seamlessly from where you left off. No client-side re-anchoring is required.
class ResilientSession { constructor(token, config) { this.token = token; this.config = config; this.processedUntil = 0; // last durable high-water mark this.buffer = []; // [{ t: absoluteSeconds, pcm: ArrayBuffer }] } connect() { const offset = this.processedUntil; const url = `wss://speech.medisync.me/api/v2/ws/transcribe?token=${this.token}`; this.ws = new WebSocket(url); this.ws.binaryType = "arraybuffer"; this.ws.onopen = () => this.ws.send(JSON.stringify({ ...this.config, type: "config", timeline_offset_seconds: offset, })); this.ws.onmessage = (event) => { if (typeof event.data !== "string") return; const msg = JSON.parse(event.data); if (msg.type === "session_ready") { // resend only the tail the server has NOT consumed for (const frame of this.buffer) { if (frame.t > this.processedUntil) this.ws.send(frame.pcm); } this.startMic(); } else if (msg.type === "transcript" || msg.type === "ack") { this.processedUntil = Math.max(this.processedUntil, msg.processed_until); this.trimBuffer(); } }; // 1006/abnormal close, etc. -> reconnect after the socket is fully closed this.ws.onclose = () => setTimeout(() => this.connect(), 1000); } trimBuffer() { this.buffer = this.buffer.filter((f) => f.t > this.processedUntil); }}
class ResilientSession: def __init__(self, token, config): self.token = token self.config = config self.processed_until = 0.0 # last durable high-water mark self.buffer = [] # list of (abs_seconds, pcm_bytes) async def run(self): import json, websockets url = f"wss://speech.medisync.me/api/v2/ws/transcribe?token={self.token}" while True: try: async with websockets.connect(url, max_size=None) as ws: cfg = {**self.config, "type": "config", "timeline_offset_seconds": self.processed_until} await ws.send(json.dumps(cfg)) ready = json.loads(await ws.recv()) assert ready["type"] == "session_ready" # resend only the tail after the last processed_until for t, pcm in self.buffer: if t > self.processed_until: await ws.send(pcm) await self._pump(ws) except websockets.ConnectionClosed: continue # reconnect from the tracked high-water mark async def _pump(self, ws): import json async for message in ws: if isinstance(message, (bytes, bytearray)): continue msg = json.loads(message) if msg["type"] in ("transcript", "ack"): self.processed_until = max(self.processed_until, msg["processed_until"]) self.buffer = [(t, p) for (t, p) in self.buffer if t > self.processed_until]
A user may have one active live session at a time. A second concurrent connection is rejected with an error frame of code: "session_conflict", and its socket is closed.
Reconnect only after the previous socket has fully closed. If you open the new session too early, it is refused with session_conflict. On an abnormal drop, wait for the close event (or a short delay) before reconnecting.
A reconnect is a new session, so speaker labels are assigned per session. Labels for audio after a reconnect are not guaranteed to match the labels used before the drop. Text and word timings remain continuous; only the per-speaker labels may shift. If stable cross-reconnect speaker labels matter to your use case, contact support@medisync.me.
