Realtime Speech to Text

The realtime endpoint streams transcription over WebSocket instead of waiting for a complete file. Send audio frames as they arrive, get text back while the speaker is still talking, and flush the last few words on demand. It serves reson8/reson8-transcribe, which runs the Reson8 speech engine on our infrastructure, and is the streaming counterpart of the Speech to Text models.

Typical uses: voice agents, live captions, phone-line transcription. The endpoint accepts raw mu-law and A-law audio, so 8 kHz phone audio needs no conversion.

Prerequisites

• A Nebul API key, exported as NEBUL_API_KEY. The Quick Start covers key setup.
• A WebSocket client. The examples use Python's websockets (pip install websockets) and Node's ws (npm install ws).

Connect

wss://api.inference.nebul.io/v1/speech-to-text/realtime?model=reson8/reson8-transcribe

The model query parameter is required: the gateway routes and authorizes on it. Authenticate with your platform key on the handshake:

A successful handshake returns HTTP 101 and the session is open. A missing or invalid key is rejected before the session starts.

Query parameters

Set tuning parameters on the same handshake URL. They are forwarded to the speech engine; anything outside this list is dropped.

Timestamps (start_ms, duration_ms) are always included on transcripts and can't be turned off: the endpoint bills on them.

Send audio

Send audio as binary WebSocket messages. Keep chunks around one second; the engine accepts messages up to 64 KiB per message and transcribes about five to six times faster than real time. If you feed faster than the engine can process, the surplus is dropped (and not billed).

Send this message when the speaker finishes, or when you need the buffer finalized:

{ "type": "flush_request", "id": "turn-3" }

The optional id comes back on the matching confirmation, so you can pair requests with answers.

Receive results

Each text result is a JSON message:

{ "type": "transcript", "text": "can I book for nine o'clock", "start_ms": 1200, "duration_ms": 2400, "is_final": true }

When include_interim=true, you get an interim shortly after speech starts and roughly one per second after. Interims grow as a prefix; a word heard wrong early can be corrected in a later interim or the final.

After a flush, you get a final transcript followed by the confirmation echoing your id:

{ "type": "flush_confirmation", "id": "turn-3" }

The endpoint does not detect turn ends on its own. Audio can stop and stay silent without a final arriving, so a voice agent should flush on its own end-of-turn signal. For server-side detection, see Turn Detection.

Example

Streams a file in one-second chunks, flushes, and prints every final transcript:

import asyncio
import json
import os

import websockets

URL = "wss://api.inference.nebul.io/v1/speech-to-text/realtime"
PARAMS = "model=reson8/reson8-transcribe&language=nl&include_interim=true"

async def main():
    headers = {"Authorization": f"Bearer {os.environ['NEBUL_API_KEY']}"}
    async with websockets.connect(f"{URL}?{PARAMS}", additional_headers=headers) as ws:
        async def send_audio():
            with open("recording.wav", "rb") as f:
                while chunk := f.read(32000):
                    await ws.send(chunk)
                    await asyncio.sleep(1)  # approximate real time
            await ws.send(json.dumps({"type": "flush_request", "id": "final"}))

        sender = asyncio.create_task(send_audio())
        async for message in ws:
            event = json.loads(message)
            if event["type"] == "transcript" and event.get("is_final", True):
                print("final:", event["text"])
            if event["type"] == "flush_confirmation":
                break
        await sender

asyncio.run(main())

For a live microphone, stream chunks as they arrive instead of reading from a file, and keep the socket open for the whole call. One session handles a long call; open a new one per speaker turn only if you want isolated timing.

Billing

Usage is metered per session as a running total of transcribed audio: one input token per millisecond of audio with recognized speech. At the list price of EUR 0.10 per 1M input tokens, an hour of billable audio costs EUR 0.36. Silence and dropped surplus audio produce no transcript timing, and nothing without timing is billed. The Model Catalog lists current pricing.

Realtime and HTTP differ

The realtime endpoint and the HTTP transcriptions endpoint use different engine components: a streaming decoder that finalizes with limited lookahead, and a batch decoder that sees the whole file. Expect small wording differences between the two on the same audio. If you need byte-identical reruns of the same input, use HTTP.

Errors and close codes

A rejected handshake returns HTTP 400 (unknown model or invalid parameter), 401 (missing or invalid key), or 429 (usage limit), with the reason in X-Error-Message.

After the handshake, problems arrive as a close code plus a machine-readable reason. Branch on the code for coarse handling and on the reason for specifics:

A connection that drops without a close frame (code 1006 in browsers) is a network failure. Treat it like 1011: reconnect and resend unconfirmed audio.

For prerecorded files, batch formats, and subtitle output, see Speech to Text.