How to Use LLM API Usage Logs for Cost and Debugging

Last verified: July 20, 2026

LLM API usage logs should let engineering and finance answer one question together: what happened to this application task, which model attempts ran, how many tokens were billed, what did they cost and how long did they take? The minimum useful record includes timestamp, outcome type, model, key name, input/output tokens, USD cost, latency, stream flag and request ID.

LumeAPI's current Usage interface exposes those request-level fields in one console. Application logs still matter because the gateway cannot know whether the model output passed your schema, satisfied the user or completed a business workflow. Reconcile both layers instead of treating token usage as task success.

Verified LumeAPI usage fields

The LumeAPI backend response model and Usage page were checked on July 20, 2026. Each record can contain:

The console displays time, type, model, key, token split, cost, latency and detail. Treat detail as potentially sensitive operational data and apply access controls and retention rules.

Do not log only the model response

A useful application event should record task outcome without storing secrets or user content.

from dataclasses import asdict, dataclass
import json

@dataclass
class AiTaskLog:
    task_id: str
    task: str
    model: str
    request_id: str | None
    attempt: int
    outcome: str
    validation_ok: bool | None
    latency_ms: int
    input_tokens: int | None
    output_tokens: int | None

def emit_task_log(record: AiTaskLog) -> None:
    print(json.dumps(asdict(record), separators=(',', ':')))

Do not add the API key, authorization header, full prompt, full output, email address or customer document. If debugging needs content, use a restricted, time-bounded workflow with explicit policy.

Record one non-streaming request

import os
import time
import uuid
from openai import OpenAI

client = OpenAI(
    api_key=os.environ['LUMEAPI_KEY'],
    base_url='https://api.lumeapi.site/v1',
    timeout=30.0,
)

def classify_ticket(text: str) -> str:
    task_id = str(uuid.uuid4())
    model = 'gpt-5.4-mini'
    started = time.perf_counter()
    response = client.chat.completions.create(
        model=model,
        messages=[
            {'role': 'system', 'content': 'Return billing, technical, or account.'},
            {'role': 'user', 'content': text},
        ],
        max_tokens=20,
    )
    answer = response.choices[0].message.content or ''
    valid = answer.strip().lower() in {'billing', 'technical', 'account'}
    usage = response.usage

    emit_task_log(AiTaskLog(
        task_id=task_id,
        task='ticket_classification',
        model=model,
        request_id=getattr(response, '_request_id', None),
        attempt=1,
        outcome='success',
        validation_ok=valid,
        latency_ms=round((time.perf_counter() - started) * 1000),
        input_tokens=usage.prompt_tokens if usage else None,
        output_tokens=usage.completion_tokens if usage else None,
    ))
    if not valid:
        raise ValueError('Model output failed classification validation.')
    return answer.strip().lower()

The gateway log proves a model call occurred and was metered. The application record adds task_id and validation_ok, which show whether it accomplished the intended job.

Record failures without losing request IDs

import openai

def safe_error_fields(exc: Exception) -> dict[str, object]:
    return {
        'error_type': type(exc).__name__,
        'status': getattr(exc, 'status_code', None),
        'request_id': getattr(exc, 'request_id', None),
        'retryable': isinstance(exc, (
            openai.APITimeoutError,
            openai.APIConnectionError,
            openai.RateLimitError,
            openai.InternalServerError,
        )),
    }

Preserve failed-attempt records. If a timeout triggers a fallback, the completed business task can have two gateway calls and two costs. Hiding the failed primary makes cost per task look artificially low.

Reconcile gateway calls to application tasks

Use this hierarchy:

1. exact request ID when available in both layers;
2. application task ID propagated as supported metadata;
3. model, key, timestamp window and attempt order as a fallback match;
4. never join only on token count or cost.

A synthetic reconciliation row might look like:

{
  "task_id": "task-7c8f",
  "task": "ticket_classification",
  "accepted": true,
  "attempts": [
    {"model": "gpt-5.4-mini", "outcome": "timeout", "cost_usd": 0.0004},
    {"model": "gemini-3.5-flash", "outcome": "success", "cost_usd": 0.0012}
  ],
  "total_cost_usd": 0.0016
}

This is an illustrative schema, not a real customer record. It shows why gateway call count and product task count are different metrics.

Calculate cost per successful task

def cost_per_success(records: list[dict]) -> float | None:
    total_cost = sum(float(row['cost_usd']) for row in records)
    successful_tasks = {
        row['task_id']
        for row in records
        if row.get('accepted') is True
    }
    if not successful_tasks:
        return None
    return total_cost / len(successful_tasks)

Group every retry and fallback attempt under the task. Token price alone does not capture validation failures, duplicated calls or human escalation.

Metrics worth monitoring

Do not average away incidents. Track percentiles and segment by model, task, streaming flag, status and deployment version.

Streaming needs special care

Developers have repeatedly asked in the official OpenAI Node repository how to obtain token usage for streaming requests. SDK and API behavior evolves, and an interrupted stream may not deliver final usage in the same way as a completed non-streaming call.

For LumeAPI, the gateway's metered Usage record is therefore important. Also record:

• whether [DONE] was observed;
• last event ID;
• whether resume was attempted;
• partial output length;
• user abort versus network failure.

See the stream recovery guide for event checkpointing.

An incident workflow

When a user reports a slow or costly answer:

1. Find the application task_id.
2. Identify all attempts and selected models.
3. Match request IDs to LumeAPI Usage records.
4. Compare application duration with gateway latency_s.
5. Inspect token split and output limit.
6. Check whether a retry, fallback or validation failure added cost.
7. Compare the deployment/prompt version.
8. Correct the cause and add a regression alert.

If application latency is much higher than gateway latency, investigate queues, retrieval, tools and network delivery. If both rise, investigate the model route or provider.

Why LumeAPI's unified log matters

One LumeAPI key and console can cover supported GPT, Claude and Gemini calls, with the same USD wallet used for listed text, image and video models. Request-level model, token, cost and latency records reduce the work of normalizing separate provider billing exports.

That does not replace product analytics. The gateway observes calls; your application defines successful outcomes. The strongest system joins both views while protecting user data. Review the LLM API gateway, AI API pricing, and multi-model API.

Sources and verification

• LumeAPI backend UsageLogOut response model and Usage page source for the field dictionary verified July 20, 2026.
• Official OpenAI Python library and Node.js SDK for request IDs, usage objects and error conventions.
• OpenAI Node usage-in-stream issue and Reddit request-tracing discussion as community demand signals.

All Python blocks were syntax-checked on July 20, 2026. Examples contain no customer logs, and no billable inference request was made.