How to Recover an Interrupted LLM API Stream

Last verified: July 20, 2026

Recover an interrupted LLM API stream by checkpointing two values while reading Server-Sent Events: the stream ID from the first JSON chunk and the latest numeric event ID. With LumeAPI, reconnect to GET /v1/chat/completions/stream/resume/{stream_id}?last_event_id=N using the same bearer key. The server replays cached events after N, so the client must deduplicate event IDs and append only new content.

Do not blindly submit the original POST again. A new generation can produce different text, duplicate cost and repeat downstream actions. Resume when the stream is still available; deliberately restart only after an expired-stream response and only when the task is safe to repeat.

Why streaming failures are different

A normal request either returns a complete body or an error. A stream can fail after delivering useful output. Your application must distinguish:

• no connection was established;
• headers arrived but no event was received;
• some events arrived and the connection dropped;
• [DONE] arrived, but the client missed final bookkeeping;
• the user intentionally aborted;
• a proxy buffered data and appeared to stall.

GitHub issues in the official OpenAI Node SDK and Codex repositories show recurring questions about buffering, knowing when a stream ends and mid-stream transport errors. Those posts validate the user problem; the recovery contract below comes from LumeAPI's current developer documentation.

LumeAPI's resumable SSE contract

The documented Chat Completions stream uses:

retry: 3000

id: 1
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":"Hello"}}]}

id: 2
data: {"id":"chatcmpl-abc","choices":[{"delta":{},"finish_reason":"stop"}]}

id: 3
data: [DONE]

The first JSON object's id becomes stream_id. The SSE id: line is the event checkpoint. Heartbeat comments such as : PING can appear during long gaps and should not be parsed as JSON.

Parse SSE frames without assuming network chunks are events

TCP/read chunks can split one SSE frame or combine several. Buffer lines until a blank line ends the frame.

from dataclasses import dataclass
from typing import Iterable, Iterator

@dataclass(frozen=True)
class SseEvent:
    event_id: int | None
    data: str

def parse_sse(lines: Iterable[str]) -> Iterator[SseEvent]:
    event_id: int | None = None
    data_lines: list[str] = []

    for raw_line in lines:
        line = raw_line.rstrip('\r\n')
        if line == '':
            if data_lines:
                yield SseEvent(event_id, '\n'.join(data_lines))
            event_id = None
            data_lines = []
            continue
        if line.startswith(':') or line.startswith('retry:'):
            continue
        if line.startswith('id:'):
            event_id = int(line[3:].strip())
        elif line.startswith('data:'):
            data_lines.append(line[5:].lstrip())

    if data_lines:
        yield SseEvent(event_id, '\n'.join(data_lines))

The parser treats heartbeats as comments, collects multiline data: values and does not equate a socket read with a complete JSON object.

Save checkpoints and deduplicate replayed events

import json
from dataclasses import dataclass

@dataclass
class StreamState:
    stream_id: str | None = None
    last_event_id: int = 0
    text: str = ''
    completed: bool = False

def apply_event(state: StreamState, event: SseEvent) -> None:
    if event.event_id is not None and event.event_id <= state.last_event_id:
        return
    if event.data == '[DONE]':
        state.completed = True
    else:
        payload = json.loads(event.data)
        state.stream_id = state.stream_id or payload.get('id')
        choices = payload.get('choices') or []
        if choices:
            delta = choices[0].get('delta') or {}
            state.text += delta.get('content') or ''
    if event.event_id is not None:
        state.last_event_id = event.event_id

Persist the checkpoint somewhere appropriate for the workflow. An in-memory object is enough for a command-line client; a long-running web task may need a durable row keyed by the application request.

Start the stream with raw HTTP

import os
import requests

BASE_URL = 'https://api.lumeapi.site/v1'
HEADERS = {
    'Authorization': f"Bearer {os.environ['LUMEAPI_KEY']}",
    'Content-Type': 'application/json',
}

def start_stream(model: str, prompt: str) -> requests.Response:
    response = requests.post(
        f'{BASE_URL}/chat/completions',
        headers=HEADERS,
        json={
            'model': model,
            'messages': [{'role': 'user', 'content': prompt}],
            'stream': True,
        },
        stream=True,
        timeout=(10, 45),
    )
    response.raise_for_status()
    return response

Keep connect and read timeouts separate. A heartbeat interval around 30 seconds means an overly short read timeout can create false failures. Use the current documented value when setting production thresholds.

Resume from the next event

def resume_stream(state: StreamState) -> requests.Response:
    if not state.stream_id:
        raise RuntimeError('Cannot resume before stream_id is known.')

    url = f'{BASE_URL}/chat/completions/stream/resume/{state.stream_id}'
    response = requests.get(
        url,
        headers={
            'Authorization': HEADERS['Authorization'],
            'Last-Event-ID': str(state.last_event_id),
        },
        params={'last_event_id': state.last_event_id},
        stream=True,
        timeout=(10, 45),
    )
    response.raise_for_status()
    return response

The query parameter or Last-Event-ID header is sufficient; the example sends both for clarity. The same bearer key is required.

Combine initial reading and one bounded resume

def consume_response(response: requests.Response, state: StreamState) -> None:
    lines = response.iter_lines(decode_unicode=True)
    for event in parse_sse(lines):
        apply_event(state, event)

def generate_with_resume(model: str, prompt: str) -> str:
    state = StreamState()
    try:
        with start_stream(model, prompt) as response:
            consume_response(response, state)
    except requests.RequestException:
        if not state.stream_id:
            raise

    if not state.completed:
        with resume_stream(state) as response:
            consume_response(response, state)

    if not state.completed:
        raise RuntimeError('Stream ended without [DONE] after one resume.')
    return state.text

One resume is a starting policy, not a universal optimum. Bound attempts and total deadline. Do not loop forever while the user waits.

Handle an expired stream

LumeAPI documents 404 when the stream hub expired—more than 30 minutes after completion—or the server restarted. Do not repeatedly call the resume endpoint. Choose one of three explicit outcomes:

1. return the partial text marked incomplete;
2. restart the original prompt if it is safe and the user accepts possible differences/cost;
3. fail the task and ask the caller to retry.

For tool calls or side effects, never restart without idempotency and durable workflow state.

Test the recovery path

A complete test plan should:

• verify Content-Type: text/event-stream;
• save the first chatcmpl-… stream ID;
• confirm numeric event IDs increase;
• tolerate : PING comments;
• force a disconnect after event N;
• resume with last_event_id=N;
• prove N is not appended twice;
• detect [DONE];
• simulate 404 expiry;
• verify abort is not treated as an outage;
• measure total attempts, latency and cost.

Without a customer inference key, this article validates the parser and documented protocol but does not claim a live forced-disconnect result. Run the checklist in staging with the exact network path used by users.

Why LumeAPI is useful here

LumeAPI documents a resumable stream route in addition to its OpenAI-compatible Chat Completions endpoint. One key can be used with supported GPT, Claude and Gemini text models, while per-call logs record model, token usage, cost, latency, stream status and request ID. That makes a partial-stream incident easier to trace without treating every disconnect as a brand-new generation.

For broad retry, timeout and circuit-breaker design, use the production failure runbook and LLM API gateway.

Sources and verification

• LumeAPI docs and server-side developer reference for SSE frames, heartbeats, resume URL, event replay and expiry behavior verified July 20, 2026.
• Official OpenAI Node SDK for SSE streaming and error conventions.
• OpenAI Node buffering issue and Codex stream-disconnect issue as community demand signals.

All Python blocks were syntax-checked and the parser was tested with local SSE fixtures on July 20, 2026. No billable model request was made.