How to Point LangChain at an OpenAI-Compatible API

Short path: OpenAI-compatible API · Multi-model API · Production LLM API · Models

Last verified: August 3, 2026

To point LangChain at an OpenAI-compatible API, instantiate ChatOpenAI with the provider's base_url, inference key and exact model ID. For LumeAPI, use base_url="https://api.lumeapi.site/v1". Start with a one-message invoke test, then separately verify streaming, tools and structured output because Chat Completions compatibility does not guarantee every optional feature.

LangChain's current model documentation explicitly supports a custom base URL for providers that implement the OpenAI Chat Completions API.

Install the integration package

python -m pip install -U langchain-openai

Set environment variables in the shell that starts your application:

export LUMEAPI_API_KEY='replace-with-your-inference-key'
export LUMEAPI_MODEL='gpt-5.6-terra'

Use a secret manager in production. Do not commit .env files or confuse the model key with the Research publishing credential.

Minimal ChatOpenAI request

import os

from langchain_openai import ChatOpenAI


def build_model() -> ChatOpenAI:
    return ChatOpenAI(
        model=os.getenv("LUMEAPI_MODEL", "gpt-5.6-terra"),
        api_key=os.environ["LUMEAPI_API_KEY"],
        base_url="https://api.lumeapi.site/v1",
        temperature=0,
        timeout=45,
        max_retries=2,
    )


model = build_model()
message = model.invoke("Reply with exactly: langchain connection ok")
print(message.content)

The important argument is base_url. LangChain's reference also recognizes OpenAI base-URL environment variables, but an explicit value in the provider factory is easier to review and less likely to be changed by an unrelated process-level setting.

Stream a response

model = build_model()

for chunk in model.stream("List three safe retry rules for an LLM API."):
    if chunk.content:
        print(chunk.content, end="", flush=True)
print()

Do not assume every chunk.content is a plain string for every multimodal or future model route. This example is scoped to text output. Preserve finish metadata and usage separately if your application bills or audits requests.

Configuration and ownership map

This separation matters because a gateway can route multiple models, but LangChain still owns orchestration and your application still owns validation, retries and side effects.

Add model selection without accepting arbitrary IDs

ALLOWED_MODELS = {
    "fast": "gpt-5.4-mini",
    "balanced": "gpt-5.6-terra",
    "hard": "gpt-5.6-sol",
}


def model_for_tier(tier: str) -> ChatOpenAI:
    try:
        model_id = ALLOWED_MODELS[tier]
    except KeyError as exc:
        raise ValueError(f"Unknown model tier: {tier}") from exc

    return ChatOpenAI(
        model=model_id,
        api_key=os.environ["LUMEAPI_API_KEY"],
        base_url="https://api.lumeapi.site/v1",
        temperature=0,
        timeout=45,
        max_retries=2,
    )

An allowlist prevents user input from silently selecting an unapproved or higher-cost route. Recheck the live model catalog before deploying shared defaults.

Diagnose the failure at the correct layer

max_retries=2 is not a universal production policy. Retry transient transport and rate-limit failures with a total deadline. Do not retry validation errors unchanged, and do not let a retried agent step repeat an email, ticket or payment.

What this setup does not prove

A successful invoke proves basic authentication, route selection and text generation. It does not prove:

• tool schemas are accepted by the selected model;
• structured output is enforced;
• token usage is populated in the metadata location your telemetry expects;
• embeddings work through the same model ID;
• an agent graph can safely resume after failure;
• every first-party OpenAI parameter is supported.

Create a small capability test suite for the features your LangChain application actually uses. Save the model ID, dependency versions, request ID, latency and normalized result. This produces better evidence than a screenshot of one successful chat.

When to use LangChain versus the raw SDK

Use ChatOpenAI when the application already benefits from LangChain messages, runnables, tools, retrieval or graph orchestration. Use a raw compatible SDK when the task is only one model request and additional abstraction would make debugging harder. The gateway decision and the orchestration-library decision are separate.

For an existing LangChain workload with high token spend, see the LangChain API cost guide. This page owns configuration and compatibility; the cost page owns optimization, avoiding keyword cannibalization.

Sources and testing boundary

• LangChain model documentation
• LangChain <code>ChatOpenAI</code> reference
• LumeAPI developer documentation
• LumeAPI model catalog

The Python blocks were syntax-checked on August 3, 2026. No customer inference key was available for a billable LangChain request, so users should run the exact smoke test with their installed package version before production use.