Does the ChatGPT API Remember Conversations? History, Memory and Token Cost

Last updated: August 6, 2026

No—an ordinary Chat Completions request does not automatically remember a previous request. Your application must send the conversation context that the model needs on every turn. Some provider APIs offer stored conversation objects or response chaining, but that is provider-specific state, not the same thing as durable user memory, and the relevant context can still count toward model input.

The production answer is not “send everything forever.” Keep a small working set: fixed instructions, explicitly pinned facts, a rolling summary, the latest turns and only the older memories relevant to the new question.

LumeAPI is an independent third-party API gateway, not OpenAI or ChatGPT. The LumeAPI route in this guide is OpenAI-compatible Chat Completions, so the application owns conversation storage and memory policy.

Short path: To build a stateful assistant, start with the chatbot API, use the OpenAI-compatible API, and inspect the available model catalog.

Quick Answer

In short

Treat a conversational model as a reasoning function with an input, not as your customer database. Store the canonical transcript in your application. Before each call, build a bounded context package from five layers: instructions, pinned facts, rolling summary, recent turns and retrieved memories. Log what was included, measure input tokens, and give users a way to inspect and delete durable memory.

ChatGPT memory, API state and application memory are different

OpenAI's current conversation-state guide says text-generation requests can be manually managed by supplying previous messages. Its Responses API also supports chaining with previous_response_id and conversation objects. An OpenAI-compatible gateway may implement Chat Completions without implementing every provider-specific state feature, so check the exact endpoint rather than assuming “compatible” means identical storage behavior.

Why conversation cost can grow faster than the chat itself

Suppose each new user-and-assistant turn contributes about 500 input tokens to the next request. If you resend the full transcript for ten calls, the rough cumulative input is:

500 × (1 + 2 + 3 + ... + 10)
= 500 × 55
= 27,500 input tokens

The transcript contains only about 5,000 tokens after ten turns, but earlier text was processed repeatedly. Under this simplified fixed-turn assumption, cumulative input grows with N × (N + 1) / 2, not merely with N.

The estimate is deliberately provider-neutral. Real usage also includes system instructions, tool results, summaries, tokenization differences, cached-token pricing and output. Use the usage object returned by your actual route as the source of truth.

For scale only, suppose a provider charged a hypothetical $0.50 per million input tokens. The 27,500 repeated-input tokens above would cost about $0.01375, while processing 5,000 input tokens once would cost $0.00250. That is not a quote for LumeAPI or OpenAI, and a real conversation also creates output charges. The example shows why a small per-chat difference can become material across hundreds of thousands of sessions. Replace both the token counts and rates with measured usage and the live price for your chosen route.

def cumulative_full_history_tokens(turns: int, tokens_added_per_turn: int) -> int:
    if turns < 0 or tokens_added_per_turn < 0:
        raise ValueError("Inputs must be non-negative")
    return tokens_added_per_turn * turns * (turns + 1) // 2


print(cumulative_full_history_tokens(10, 500))  # 27,500

To estimate money, multiply measured input and output tokens by the live rates for the exact model and provider. Do not copy a model price from an old forum answer.

A five-layer memory policy

Build the request in this order, with a separate budget for each layer:

1. Fixed instructions: identity, safety rules and response contract. Keep this stable and short.
2. Pinned facts: facts the user explicitly wants remembered, such as preferred language or a project constraint.
3. Rolling summary: compact state of older conversation, including decisions and unresolved tasks.
4. Recent turns: the last few complete user/assistant exchanges, preserved verbatim.
5. Retrieved memories: older items selected because they are relevant to the new request, with ownership and confidence checks.

If the context budget is exceeded, discard low-value retrieved memories first, then older recent turns. Do not silently remove safety instructions or user-approved constraints. When a summary changes a material fact, retain a reference to the original event so it can be audited.

Runnable Python memory buffer for Chat Completions

This minimal class makes the boundary explicit. It does not pretend to summarize text by chopping characters. Your application supplies a reviewed summary when old turns roll out.

from collections import deque
from dataclasses import dataclass, field
from typing import Deque


@dataclass
class ConversationMemory:
    system_prompt: str
    recent_turn_limit: int = 4
    pinned_facts: list[str] = field(default_factory=list)
    rolling_summary: str = ""
    recent: Deque[tuple[str, str]] = field(default_factory=deque)

    def build_messages(self, user_text: str) -> list[dict[str, str]]:
        messages = [{"role": "system", "content": self.system_prompt}]
        if self.pinned_facts:
            messages.append({
                "role": "system",
                "content": "User-approved memory:\n- " + "\n- ".join(self.pinned_facts),
            })
        if self.rolling_summary:
            messages.append({
                "role": "system",
                "content": "Older conversation summary:\n" + self.rolling_summary,
            })
        for old_user, old_assistant in self.recent:
            messages.extend([
                {"role": "user", "content": old_user},
                {"role": "assistant", "content": old_assistant},
            ])
        messages.append({"role": "user", "content": user_text})
        return messages

    def commit(self, user_text: str, assistant_text: str) -> None:
        self.recent.append((user_text, assistant_text))
        while len(self.recent) > self.recent_turn_limit:
            self.recent.popleft()

    def replace_summary(self, reviewed_summary: str) -> None:
        self.rolling_summary = reviewed_summary.strip()

Connect it to LumeAPI with the OpenAI Python client:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LUMEAPI_KEY"],
    base_url="https://api.lumeapi.site/v1",
    timeout=30.0,
    max_retries=0,
)
model = os.environ["LUMEAPI_MODEL"]
memory = ConversationMemory("Answer accurately and state uncertainty.")


def chat(user_text: str) -> str:
    response = client.chat.completions.create(
        model=model,
        messages=memory.build_messages(user_text),
        max_tokens=800,
    )
    answer = response.choices[0].message.content or ""
    memory.commit(user_text, answer)
    print({"usage": response.usage, "finish": response.choices[0].finish_reason})
    return answer

This is an in-memory example, not a complete multi-user service. In production, partition records by authenticated tenant and conversation, use transactions, encrypt sensitive data, impose retention limits and never let one user's retrieved memory enter another user's prompt.

When to summarize, retrieve or start a new chat

Summaries should be structured: stable facts, decisions, open questions, constraints and source references. Avoid a free-form “story of the chat” that can convert an uncertain statement into a permanent fact.

A memory record needs more than text

A durable memory item should include:

tenant_id
user_id
conversation_id
fact_or_event
source_message_id
created_at
last_confirmed_at
confidence
consent_or_policy_basis
expires_at
deletion_status

Retrieval must filter by tenant and user before semantic similarity. Then rank by relevance, freshness, confidence and importance. A highly similar memory belonging to another account is a security incident, not a good retrieval result.

Test whether the memory policy actually helps

Run a small fixed evaluation before expanding retention. Use conversations that contain a stable preference, a corrected fact, a topic change, a deletion request and a deliberately similar fact from another test account.

Record context tokens, retrieved item IDs, answer quality and deletion outcome for each case. A memory change passes only when it improves task completion without violating isolation or materially increasing irrelevant input. “The chatbot sounds more personal” is not enough evidence.

A realistic production scenario

A support chatbot serves a returning customer who says, “Use the same delivery address as last time.” Sending the last 100 messages would be expensive and may expose unrelated support details. Sending only the newest two turns loses the address decision.

The application instead loads one user-approved pinned address reference, a summary of the unresolved replacement order and the latest four turns. It asks for confirmation before placing the order because an address can become stale. The request stays bounded, the user gets continuity, and the application—not the model—owns the audit trail and deletion process.

What most guides get wrong

Many guides reduce memory to messages.append(...). That demonstrates a second turn but does not solve cost growth, multi-user isolation, privacy, stale facts or deletion. Others imply that a conversation ID eliminates token cost. An ID can move transcript assembly to a provider, but it does not prove that prior context is ignored by inference or billing.

Another weak pattern is summarizing only after the request fails. Memory should have a budget before production traffic arrives, and summaries should preserve decisions and provenance rather than merely shorten text.

Expert take

Conversation history and memory are separate engineering products. History preserves what was said; memory selects what remains useful. A robust assistant keeps the transcript as evidence, constructs a bounded prompt for each task and makes durable memory visible, editable and deletable. The best memory layer is not the one that remembers the most—it is the one that retrieves the smallest trustworthy context that lets the user finish the current task.

FAQ

Does the OpenAI API remember previous messages automatically?

A standalone Chat Completions request sees the messages you send in that request. OpenAI also offers provider-managed state through the Responses API, but that is a different interface and policy. Check the exact API you use.

Can I use a conversation ID with an OpenAI-compatible API?

Your own application can map a local conversation ID to stored messages. Provider-specific conversation objects are not guaranteed by generic Chat Completions compatibility.

How many previous messages should I send?

There is no universal count. Reserve a measured token budget, keep recent complete turns, preserve pinned constraints and summarize or retrieve older material. Log real usage instead of estimating by message count alone.

Will summarizing history reduce cost?

It can reduce repeated input when the summary is much shorter than the discarded turns. The summary call itself also costs tokens and can lose facts, so measure the break-even point and retain provenance for important claims.

How do I make memory persist between sessions?

Store approved memory records in your backend under an authenticated user and tenant, then retrieve only relevant records for the next request. Do not rely on a Python process's in-memory list for production persistence.

How should users delete AI memory?

Provide a control that removes the primary record and its derived retrieval entries, caches and summaries according to your retention policy. Confirm completion without exposing deleted content in logs.

Sources and verification

• OpenAI conversation state guide — current API behavior and provider-managed state options, checked August 6, 2026.
• OpenAI API data controls — endpoint retention boundaries, checked August 6, 2026.
• V2EX: API history grows and becomes expensive and V2EX: saving tokens in multi-turn chat — Chinese developer demand signals, not product authority.
• Reddit: persistent chats and Reddit: handling chatbot history — international user-problem signals, not factual authority.
• Baidu Tieba was searched for Chinese-language versions of this task on August 6, 2026, but no sufficiently specific, reliably indexed technical thread was used as evidence.