DAKERA
Dakera on LoCoMo: 88.2% Recall@20
Full methodology and per-category breakdown
Dakera scores 88.2% Recall@20 on the LoCoMo benchmark — 10 conversations, 1,536 evaluated questions, no LLM in the retrieval path. The metric is LLM-judged retrieval recall: whether the ground-truth answer is present in the top-20 retrieved memories. Here is every number, broken down by category.
Four question categories
LoCoMo tests four distinct recall challenges. Dakera leads on three; temporal inference is our hardest category and an active area of improvement.
What 88.2% actually means
The benchmark dispute
Other vendors claim higher LoCoMo scores: Zep at 94.7%, Mem0 at 92.5%. Both numbers are actively disputed — Mem0 contests Zep's methodology, Zep disputes Mem0's. The disagreement is about what the benchmark actually measures.
Dakera's 88.2% is a Recall@20 figure — an LLM judge checks whether the ground-truth answer is present in the top-20 retrieved memories. There is no LLM in the retrieval path: retrieval is hybrid HNSW + BM25 fusion followed by a non-LLM temporal re-ranker. Note this measures a different thing than the QA-accuracy scores some vendors report (where an answering LLM generates a response that is then judged), so the numbers are not directly rankable against each other. Full methodology is documented at the benchmark methodology post.
Dakera's LoCoMo scores — full breakdown
All numbers below are Dakera's own Recall@20 results on the LoCoMo dataset (v0.11.107, September 2026). Methodology documented in full at the benchmark methodology post.
| Category | Score | Questions | Description |
|---|---|---|---|
| Overall | 88.2% | 1,536 (10 conversations) | Recall@20 · no LLM in retrieval path · adversarial category excluded |
| Cat1 — Single-hop recall | 86.9% | 282 questions | Direct facts from recent or distant sessions |
| Cat2 — Multi-hop reasoning | 85.4% | 321 questions | Cross-session reasoning chains requiring multiple recall steps |
| Cat3 — Temporal inference | 73.9% | 92 questions | Time-anchored questions — our most challenging category, actively improving |
| Cat4 — Open-domain | 91.0% | 841 questions | Mixed topics and entities spanning multiple sessions |
Evaluation methodology
All scores use the public LoCoMo release — 10 simulated long-term conversations (up to ~35 sessions each); 1,536 of the 1,540 QA questions are evaluated (the adversarial category is excluded). We do not use sampled subsets. Retrieval uses hybrid HNSW + BM25 fusion followed by a non-LLM temporal re-ranker — there is no LLM in the retrieval path. Recall is scored by an LLM judge on the LoCoMo framework.
Version: Dakera v0.11.107, evaluated September 2026. The full evaluation protocol is documented in the benchmark methodology post.
How we evaluate
We publish our evaluation methodology in full so you can audit exactly how the number is produced.
Dataset: The public LoCoMo release — 10 simulated long-term conversations (up to ~35 sessions each); 1,536 of the 1,540 QA questions across four categories are evaluated (the adversarial category is excluded). We do not use sampled subsets. Two percentage points on a 100-question eval is statistically noise; on 1,536 questions it represents a real signal.
POST /v1/memory API. Session boundaries are preserved. No preprocessing or summarization.POST /v1/recall query. Dakera returns its top-20 memories using HNSW + BM25 hybrid retrieval with a non-LLM temporal re-ranker.The full evaluation protocol is documented in our benchmark methodology post. Questions about the numbers? Open an issue on GitHub.
What 88.2% means for your agent
Benchmark scores translate directly to agent behavior. Memory recall failures cause agents to repeat questions, forget context, and give inconsistent answers.
At 88.2% recall, your agent remembers what the user told it — across sessions. No "as I mentioned earlier" failures.
91.0% open-domain recall means agents carry context across days, weeks, and months — not just within a single conversation.
<10ms P99 query latency with no LLM in the retrieval path. Measured on v0.11.107, warm in-memory index (20k vectors, top_k=20), single-threaded, on an idle 8-core x64 host — server-side P99 under 10ms (end-to-end incl. client ≈16ms). Latency scales with index size.
How Dakera compares
Other vendors publish LoCoMo results using different protocols and, in most cases, a different metric. Dakera's 88.2% is Recall@20 (does retrieval surface the answer?); the competitor figures below are QA-accuracy (does an answering LLM produce the correct answer?). These measure different things and are not directly rankable — the table is context, not a leaderboard.
| System | LoCoMo score | Metric & methodology |
|---|---|---|
| Dakera v0.11.107 | 88.2% | Recall@20 (LLM-judged retrieval recall) · no LLM in retrieval path |
| Letta Filesystem | 74.0% | QA-accuracy · Letta's published evaluation (letta.com/research) |
| Zep (corrected, 2025) | 75.1% | QA-accuracy · Zep's own corrected score after fixing evaluation errors |
| Mem0 (Letta independent eval, 2025) | 68.5% | QA-accuracy · third-party evaluation by Letta |
Mem0's self-reported 2026 score (92.5%) and Zep's 2026 claim (94.7%) use different models and evaluation setups — not directly comparable. See methodology for details.
The memory layer that documents its numbers
Dakera is self-hosted and open core. Run it in 5 minutes, measure recall on your own data, or reserve a spot on the managed cloud.
Methodology: Full write-up · Compare: vs Mem0 · vs Zep · vs Letta · All →