External Memory: Vector Stores & Databases

External memory solves exactly the problem in-context memory can't: persistence beyond a single session, and recall of information that was never going to fit in the context window in the first place. Rather than keeping everything in the live prompt, external memory stores information in a separate system — a vector database, a traditional relational or document database, or a hybrid of both — and retrieves only the relevant slice of it into the context window when it's actually needed for the current request.

A vector store is the most common choice for external memory that needs to be retrieved by MEANING rather than by exact key. Past conversation turns, notes an agent has taken, or facts it's learned get embedded and stored; when a new query comes in, it gets embedded too, and a similarity search pulls back whichever stored memories are most relevant to the current context — the same retrieval mechanism underlying RAG generally, just applied to the agent's OWN past experience rather than a fixed external document corpus.

A traditional database is often the better fit for memory that needs to be retrieved by exact key rather than semantic similarity — a user's stored preferences, account details, or structured facts ('user_id: 4471, preferred_language: es, subscription_tier: pro') are naturally a database lookup, not a similarity search, since you usually know exactly which user's record you need rather than searching for 'something like this.'

The real power of external memory shows up specifically in multi-session, multi-day agent deployments: without it, every new conversation with a user starts as a completely blank slate — the agent has no way to recall a user's name, preferences, or the outcome of a task it helped with three days ago, no matter how good its in-context memory was DURING that earlier session, because that context window is long gone once the session ended. External memory is what turns a stateless, single-session assistant into something that genuinely accumulates knowledge about a user or a task over time.

from openai import OpenAI

client = OpenAI()

def embed(text: str) -> list[float]:
    return client.embeddings.create(model="text-embedding-3-small", input=text).data[0].embedding

class ExternalMemory:
    """Minimal external memory: a vector store for semantic recall,
    plus a plain dict for exact-key lookups like user preferences."""
    def __init__(self):
        self.vector_memories: list[tuple[list[float], str]] = []
        self.key_value_memory: dict[str, str] = {}

    def remember(self, text: str) -> None:
        self.vector_memories.append((embed(text), text))

    def set_fact(self, key: str, value: str) -> None:
        self.key_value_memory[key] = value

    def recall_similar(self, query: str, k: int = 3) -> list[str]:
        import numpy as np
        q_emb = np.array(embed(query))
        scored = [
            (float(np.dot(q_emb, np.array(emb)) / (np.linalg.norm(q_emb) * np.linalg.norm(emb))), text)
            for emb, text in self.vector_memories
        ]
        return [text for _, text in sorted(scored, reverse=True)[:k]]

memory = ExternalMemory()
memory.set_fact("user_4471_language", "es")           # exact-key lookup
memory.remember("User mentioned they're vegetarian.")   # semantic recall
memory.remember("User's order last week was delayed.")

print(memory.recall_similar("What food restrictions does the user have?"))