OpenAI-Compatible Structured Output in Python

Last verified: July 20, 2026

To get reliable structured output from an OpenAI-compatible API, request a JSON Schema when the selected model and gateway document support for it, then validate the returned content again in your application. Valid JSON is not enough: {"priority": "urgent"} parses successfully but still violates a schema that allows only low, medium, or high.

LumeAPI exposes supported text models through an OpenAI-compatible Chat Completions base URL. Basic compatibility does not automatically guarantee every advanced response_format option on every model. Treat structured output as a per-model contract test and keep a validated fallback for models that support only JSON mode or instruction-based JSON.

Install the client and validator

python -m pip install --upgrade openai pydantic
export LUMEAPI_KEY='your-lumeapi-key'

import json
import os
from typing import Literal

import openai
from openai import OpenAI
from pydantic import BaseModel, ConfigDict, ValidationError

client = OpenAI(
    api_key=os.environ['LUMEAPI_KEY'],
    base_url='https://api.lumeapi.site/v1',
    timeout=30.0,
    max_retries=1,
)
MODEL = os.getenv('LUMEAPI_MODEL', 'gpt-5.4')

Keep model selection configurable. A schema that works with one model ID must not be assumed to work unchanged with another family.

Define the application contract first

class SupportTicket(BaseModel):
    model_config = ConfigDict(extra='forbid')

    category: Literal['billing', 'technical', 'account']
    priority: Literal['low', 'medium', 'high']
    summary: str
    needs_human: bool

The application model is the source of truth. extra='forbid' catches unexpected fields that might otherwise flow silently into downstream code. Add length and numeric bounds where business rules require them.

Request strict JSON Schema output

Pydantic can generate the JSON Schema sent in response_format.

schema = SupportTicket.model_json_schema()

response = client.chat.completions.create(
    model=MODEL,
    messages=[
        {
            'role': 'system',
            'content': 'Classify the support ticket using the supplied schema.',
        },
        {
            'role': 'user',
            'content': 'My card was charged twice and I need help today.',
        },
    ],
    response_format={
        'type': 'json_schema',
        'json_schema': {
            'name': 'support_ticket',
            'strict': True,
            'schema': schema,
        },
    },
)

content = response.choices[0].message.content or ''
ticket = SupportTicket.model_validate_json(content)
print(ticket.model_dump())

Even when the API advertises strict schema output, keep the local validation. It protects the application from capability changes, integration bugs, partial responses and incorrectly handled refusals.

Handle refusal, truncation and empty content

Do not pass empty or incomplete content into business logic.

def parse_ticket_response(response: object) -> SupportTicket:
    choice = response.choices[0]
    message = choice.message

    refusal = getattr(message, 'refusal', None)
    if refusal:
        raise RuntimeError('The model refused the classification request.')
    if choice.finish_reason == 'length':
        raise RuntimeError('Structured output was truncated.')

    content = message.content or ''
    if not content.strip():
        raise RuntimeError('The model returned no structured content.')

    try:
        return SupportTicket.model_validate_json(content)
    except ValidationError as exc:
        raise RuntimeError('Output failed the application schema.') from exc

Log the sanitized validation category and request ID, not sensitive ticket text. A validation failure should be observable instead of becoming a mysterious downstream error.

Add a compatibility fallback

If a selected model rejects json_schema, decide explicitly whether JSON-object mode is acceptable. JSON-object mode can encourage valid JSON but may not enforce the full schema at generation time. Local validation remains mandatory.

def classify_ticket(text: str) -> SupportTicket:
    messages = [
        {
            'role': 'system',
            'content': (
                'Return JSON only with category, priority, summary, and '
                'needs_human. Use only documented enum values.'
            ),
        },
        {'role': 'user', 'content': text},
    ]

    try:
        response = client.chat.completions.create(
            model=MODEL,
            messages=messages,
            response_format={
                'type': 'json_schema',
                'json_schema': {
                    'name': 'support_ticket',
                    'strict': True,
                    'schema': SupportTicket.model_json_schema(),
                },
            },
        )
    except openai.BadRequestError:
        response = client.chat.completions.create(
            model=MODEL,
            messages=messages,
            response_format={'type': 'json_object'},
        )

    content = response.choices[0].message.content or ''
    return SupportTicket.model_validate_json(content)

Only fall back on a documented capability rejection. A malformed prompt, authentication failure or insufficient balance should be fixed rather than sent again. If JSON-object mode is also unsupported, use instruction-based JSON only when your validation-and-retry budget makes the residual failure rate acceptable.

Do not use regex to extract arbitrary JSON

A pattern that grabs text between the first { and last } breaks on nested objects, braces inside strings, multiple objects and truncated output. Parse the entire response with json.loads() or let Pydantic validate it. If your model wraps JSON in Markdown fences, treat that as a failed contract or implement a narrow, tested fence remover before parsing.

def parse_json_object(content: str) -> dict[str, object]:
    value = json.loads(content)
    if not isinstance(value, dict):
        raise ValueError('Expected a JSON object.')
    return value

Build a structured-output compatibility matrix

Run at least these cases for each exact model ID:

Track schema-valid rate, task correctness, retries, latency and cost per accepted record. A 100% parse rate can still hide wrong classifications, so evaluate semantic accuracy separately.

Choose between tools and structured output

Use structured output when the model should return data for your application. Use tool calling when the model should ask the application to perform an operation.

• Classification, extraction and UI configuration usually fit structured output.
• Weather lookup, database search, email and payment actions fit tools.
• A tool's arguments are structured data, but the surrounding lifecycle also needs dispatch, authorization and a result message.

Read the OpenAI-compatible tool-calling guide for that lifecycle.

Production safeguards

• Version each schema and store the version with the result.
• Use server-side model allowlists.
• Limit output tokens so runaway responses are bounded, but leave enough room for the schema.
• Validate before database writes or automated decisions.
• Retry only validation failures that a prompt or model change can plausibly fix.
• Use a dead-letter queue or human review after the retry budget.
• Re-run a fixed test set during model or SDK upgrades.
• Monitor model, validation result, latency, tokens and cost.

Why LumeAPI fits schema testing

A single LumeAPI key and OpenAI-compatible base URL let a team run the same schema harness against supported GPT, Claude and Gemini IDs without maintaining three billing and client integrations. Model choice remains explicit, and LumeAPI call logs expose model, tokens, cost and latency for each request.

The public site lists supported mainstream model discounts up to 70% for specific models and references; that is not a universal rate. Structured-output decisions should be based on schema-valid task success and cost per accepted record, not token price alone. Review AI API pricing, the model catalog, and the OpenAI-compatible API before deployment.

Sources and limitations

• OpenAI Structured Outputs guide for JSON Schema response formatting, refusals and constraints.
• Official OpenAI Python library for custom base URLs and Chat Completions types.
• LumeAPI docs, model catalog, and production LLM guide for current gateway and operating context.

All Python blocks were syntax-checked on July 20, 2026. No customer inference key was used. LumeAPI's public chat parameter table does not promise universal advanced response_format parity, so run the matrix with your own key and selected models before relying on strict JSON Schema in production.