GPT API in Node.js with Streaming: Production Guide (2026)

Last verified: July 21, 2026

Short path: Model catalog and rates on GPT API. Non-streaming Node setup: OpenAI-compatible API in Node.js.

Production chat and copilot UIs expect streaming — tokens arriving incrementally over Server-Sent Events (SSE) instead of one blocking JSON response. Waiting for a full completion before rendering the first word often adds one to three seconds of perceived latency even when total generation time is unchanged.

You can stream GPT models through LumeAPI with the official OpenAI Node.js SDK by setting baseURL to https://api.lumeapi.site/v1, passing stream: true, and iterating async chunks. The same client can later call Claude or Gemini by changing only the model field.

LumeAPI is an independent third-party OpenAI-compatible gateway. It is not OpenAI. Test streaming, tool calling, and error behavior on your heaviest prompts before full cutover.

Quick Answer

1. Install openai (v4.x) in a server-side Node project — never expose the API key in browser bundles.
2. Configure baseURL: 'https://api.lumeapi.site/v1' and apiKey: process.env.LUMEAPI_KEY.
3. Call client.chat.completions.create({ model, messages, stream: true }).
4. Use for await (const chunk of stream) and append chunk.choices[0]?.delta?.content.
5. Set client timeout (60–120s for long answers), cap max_retries, and abort upstream when the HTTP client disconnects.
6. Default streaming chat to gpt-5.4-mini or gpt-5.6-terra; escalate to gpt-5.6-sol only when evals require it.

Streaming does not change token billing — you pay for input and output tokens generated, not for the stream flag.

Choose a GPT model for streaming

LumeAPI listed these GPT text models on July 21, 2026. Confirm the live GPT API page before deployment.

Output tokens often dominate agent-style streams. Cap max_tokens per surface and log usage after each stream completes.

Project setup

mkdir gpt-stream-demo && cd gpt-stream-demo
npm init -y
npm install openai

Set the key on the server:

export LUMEAPI_KEY='your-lumeapi-key'

Examples use ECMAScript modules (.mjs). For CommonJS, adapt import to require.

Minimal streaming script

Create stream.mjs:

import OpenAI from 'openai';

const apiKey = process.env.LUMEAPI_KEY;
if (!apiKey) throw new Error('Set LUMEAPI_KEY before running.');

const client = new OpenAI({
  apiKey,
  baseURL: 'https://api.lumeapi.site/v1',
  timeout: 90_000,
  maxRetries: 2,
});

const stream = await client.chat.completions.create({
  model: 'gpt-5.6-terra',
  messages: [
    {
      role: 'system',
      content: 'Answer in at most three sentences.',
    },
    {
      role: 'user',
      content: 'What should an API health check verify?',
    },
  ],
  stream: true,
  max_tokens: 400,
});

for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
}
console.log();

Run:

node stream.mjs

The request goes to https://api.lumeapi.site/v1/chat/completions. Empty chunks are normal — always guard choices and delta.content.

Stream with AbortController (client disconnect)

When a browser tab closes mid-stream, stop upstream generation to avoid billing unused tokens:

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.LUMEAPI_KEY,
  baseURL: 'https://api.lumeapi.site/v1',
});

export async function streamToWritable(messages, writable, signal) {
  const stream = await client.chat.completions.create(
    {
      model: process.env.LUMEAPI_MODEL || 'gpt-5.4-mini',
      messages,
      stream: true,
      max_tokens: 800,
    },
    { signal },
  );

  for await (const chunk of stream) {
    if (signal?.aborted) break;
    const text = chunk.choices[0]?.delta?.content;
    if (text) writable.write(text);
  }
}

In Express, tie req.on('close') to controller.abort().

Express SSE endpoint (production sketch)

import express from 'express';
import OpenAI from 'openai';

const app = express();
app.use(express.json());

const client = new OpenAI({
  apiKey: process.env.LUMEAPI_KEY,
  baseURL: 'https://api.lumeapi.site/v1',
  timeout: 120_000,
});

app.post('/api/chat/stream', async (req, res) => {
  const controller = new AbortController();
  req.on('close', () => controller.abort());

  res.setHeader('Content-Type', 'text/event-stream; charset=utf-8');
  res.setHeader('Cache-Control', 'no-cache, no-transform');
  res.setHeader('Connection', 'keep-alive');
  res.flushHeaders?.();

  try {
    const stream = await client.chat.completions.create(
      {
        model: 'gpt-5.4-mini',
        messages: req.body.messages,
        stream: true,
        max_tokens: 1024,
      },
      { signal: controller.signal },
    );

    for await (const chunk of stream) {
      const text = chunk.choices[0]?.delta?.content;
      if (text) {
        res.write(`data: ${JSON.stringify({ text })}\n\n`);
      }
    }
    res.write('data: [DONE]\n\n');
  } catch (err) {
    if (err.name !== 'AbortError') {
      res.write(`data: ${JSON.stringify({ error: 'stream_failed' })}\n\n`);
    }
  } finally {
    res.end();
  }
});

app.listen(3000);

Validate and authenticate req.body.messages server-side. Never forward unbounded client prompts.

Error handling for streaming

import OpenAI from 'openai';

async function createStreamSafe(client, params, maxAttempts = 3) {
  let delay = 500;
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      return await client.chat.completions.create({ ...params, stream: true });
    } catch (err) {
      if (err instanceof OpenAI.RateLimitError && attempt < maxAttempts - 1) {
        await new Promise((r) => setTimeout(r, delay + Math.random() * 200));
        delay = Math.min(delay * 2, 30_000);
        continue;
      }
      throw err;
    }
  }
}

Streaming cost example

Suppose a support bot stream uses 2,000 input and 600 output tokens on gpt-5.4-mini (LumeAPI rates):

2,000 / 1e6 × $0.225 = $0.00045
  600 / 1e6 × $1.35  = $0.00081
total                  $0.00126 per conversation turn

At 50,000 turns/month ≈ $63 on mini vs several times more on Sol for the same volume.

Tool calls and structured output while streaming

Tool-call deltas may arrive across multiple chunks. Do not JSON.parse partial tool arguments. Buffer until finish_reason indicates completion, or use non-streaming mode for tool orchestration.

For structured JSON, see structured output Python guide — test whether your model id supports response_format on LumeAPI.

Interrupted streams: recover interrupted LLM stream.

Why use LumeAPI for GPT streaming?

• One API key and USD wallet for GPT, Claude, Gemini, and listed image/video models.
• OpenAI-compatible request shape — reuse existing Node clients and agent frameworks.
• Published catalog rates — GPT tiers often ~70% below reference list prices shown on model pages.
• Usage logs with model id, tokens, cost, and latency per request.

OpenAI compatibility does not guarantee every beta endpoint (Assistants, native batch admin APIs, etc.).

Production checklist

• [ ] API key only on server; frontend calls your backend route.
• [ ] Central config for baseURL, default model, timeout, max_tokens.
• [ ] Abort upstream on client disconnect.
• [ ] Bounded retries on 429/5xx only.
• [ ] Log model, duration, token usage (no secrets in logs).
• [ ] Load-test concurrent streams — rate limits are often concurrency-bound.
• [ ] Compare cheap OpenAI API guide for tier routing.

FAQ

Does streaming cost more than non-streaming?

No. Billing is per token. Streaming improves time-to-first-token UX.

Can I stream from Next.js API routes?

Yes — use Route Handlers or API routes with ReadableStream / SSE headers. Keep the key server-side.

Which GPT model is fastest for streaming?

Usually gpt-5.4-mini for latency-sensitive volume. Measure on your prompts.

What if chunks stop mid-stream?

Check network proxies (some buffer SSE), client timeouts, and stream recovery.

Node or Python for GPT streaming?

Same API semantics. This guide is Node-specific; Python: GPT API Python example.

Does LumeAPI support OpenAI Realtime API?

This guide covers Chat Completions streaming only. Check docs for supported endpoints per model.

Sources and verification

• LumeAPI GPT API and models catalog for ids and rates (July 21, 2026).
• OpenAI Node.js SDK for stream, timeouts, and errors.

Code blocks were syntax-checked; no billable end-to-end quality benchmark is claimed.