Feedback Loop Memory

Category: Learning

Problem

Agents repeat the same mistakes because they have no mechanism to learn from corrections. A user corrects a formatting error, a factual mistake, or an unwanted behavior, but the next time a similar situation arises, the agent makes the same error. Without persisting corrections, every session resets the agent to its baseline behavior.

Architecture

This pattern stores correction events with high importance scores and semantic content describing both the mistake and the correct behavior. Before generating responses in similar domains, the agent recalls relevant corrections and uses them as negative examples or behavioral constraints in the prompt.

Flow

• Detect correction signals in user messages (explicit corrections, "no", "actually...")
• Store the correction with context about what was wrong and what is right
• Before generating responses, recall corrections relevant to the current topic
• Inject corrections as constraints to prevent repeating mistakes

Implementation

from dakera import Dakera
import time

client = Dakera(base_url="http://localhost:3300", api_key="dk-...")

# Store a user correction
def store_correction(user_id: str, wrong: str, correct: str, domain: str):
    """Persist a correction event for future recall."""
    client.memory.store(
        content=f"CORRECTION: Do not {wrong}. Instead: {correct}",
        namespace=f"user-{user_id}-feedback",
        metadata={
            "type": "correction",
            "domain": domain,
            "wrong_behavior": wrong,
            "correct_behavior": correct,
            "timestamp": time.time(),
            "importance": 0.98
        }
    )

# User corrects the agent
store_correction(
    "alice",
    wrong="suggest print statements for debugging Python",
    correct="use logging module with proper log levels",
    domain="python-debugging"
)

store_correction(
    "alice",
    wrong="write verbose multi-paragraph explanations",
    correct="keep responses under 3 sentences unless asked for detail",
    domain="response-style"
)

# Recall corrections before generating a response
def get_relevant_corrections(user_id: str, query: str) -> list:
    """Retrieve past corrections relevant to the current query."""
    results = client.memory.recall(
        query=query,
        namespace=f"user-{user_id}-feedback",
        top_k=5,
        metadata_filter={"type": "correction"}
    )
    return [r["content"] for r in results["results"]]

# Before responding to a debugging question
corrections = get_relevant_corrections("alice", "How do I debug this Python function?")
# Returns: ["CORRECTION: Do not suggest print statements for debugging Python.
#            Instead: use logging module with proper log levels"]

# Inject into system prompt as constraints
system_prompt = "You are a helpful assistant.\n\nIMPORTANT constraints from past feedback:\n"
for c in corrections:
    system_prompt += f"- {c}\n"

When to Use This Pattern

• Assistants that should improve from user feedback over time
• Domain-specific agents where users have strong preferences
• Customer support bots that need to learn from escalations
• Any agent where repeated mistakes erode user trust

Key Considerations

• Use very high importance (0.95+) for corrections so they always surface
• Include both the wrong and correct behavior for clear contrast
• Tag corrections by domain to avoid over-constraining unrelated responses
• Periodically review and consolidate corrections to prevent contradictions