Memory Strategies: Buffer, Window, Summary, Entity

Each memory strategy represents a different point on the recall-cost trade-off curve.

Strategy 1: Full Conversation Buffer

messages = []  # grows with every turn
messages.append({'role': 'user', 'content': user_input})
resp = client.messages.create(model=..., messages=messages)
messages.append({'role': 'assistant', 'content': resp.content[0].text})

• ✓ Perfect recall, zero information loss
• ✗ Context grows unboundedly; will overflow for long conversations
• Use for: < 20-turn conversations, or when context window is very large (128K+)

Strategy 2: Windowed Buffer

K = 10  # keep last K turns
context = messages[-(K * 2):]  # K turns = 2K messages
resp = client.messages.create(model=..., system=system, messages=context)

• ✓ Bounded cost, simple implementation
• ✗ Hard cutoff — everything before window is completely gone
• Bug to avoid: don't include the system prompt in the window slice
• Use for: customer support (recent context is all that matters)

Strategy 3: Summary Memory Trigger: when token count exceeds threshold (e.g., 3,000 tokens) Action: summarize oldest N turns → compress into a summary block

context = [system_prompt]
        + [summary_block]   # 'Earlier: user mentioned X, asked about Y...'
        + [recent_turns]    # last 10 turns verbatim

• ✓ Bounded cost, preserves semantic gist of old context
• ✗ Loses exact wording; summarization adds latency and cost
• Use for: tutoring, long support sessions, therapy chatbots

Strategy 4: Entity Memory Extract structured facts after each turn:

entities = {}  # e.g., {'name': 'Alice', 'plan': 'Enterprise', 'issue': 'billing'}
# Inject into every context:
system_with_entities = system + f'\nKnown facts: {json.dumps(entities)}'

• ✓ Perfect recall for structured facts across unlimited turns
• ✗ Only works for things that can be extracted as key-value pairs
• ✗ Extraction errors persist indefinitely
• Use for: personal assistant, CRM bot, medical intake