My custom memory graph actually beat Memora on the LoCoMo benchmark

Context window limits remain the most frustrating constraint in agentic programming. Nothing derails productivity faster than an LLM agent that wipes its mental state between sessions, forcing you to rehash architecture and project goals yet again. I once relied on simple .md files or basic MCP tools to preserve state, but flat-file approaches lack the sophistication needed for anything beyond small projects. If you're content managing context with just a claude.md file, stick with it. As complexity grows, however, you quickly discover that a knowledge graph capable of reconciling contradictions and handling version drift becomes essential.

I began experimenting with a tool called memoryweb tailored to my personal workflow. It excelled at tracking problems and fixes as discrete "stories" in isolation, but the design crumbled when considering collaborative scenarios. Multiple users introducing new memories led to chaos—competing decisions merged into shared files, outdated information persisted, and context for key architectural calls vanished.

Recordari emerged from that frustration. Rather than storing unstructured text, it employs typed memories and typed edges. This enables agents to formally label relationships between memories, tagging them explicitly as "contradicts" or "resolved." The server surfaces semantically overlapping memories that may conflict, prompting the agent to validate accuracy. Ownership tracking ensures automated agents don’t overwrite human judgments without scrutiny.

Curious how this approach measured up against established benchmarks, I tested it using Microsoft Research’s Memora evaluation framework (ICML 2026). Every variable was controlled: identical model, shared LoCoMo dataset, and consistent exclusion criteria.

Results spoke clearly:

• Recordari end-to-end score: 0.831
• Memora published score: 0.801
• Multi-session questions (Recordari): 0.869
• Raw storage baseline: 0.760

Crucially, these numbers reflect real performance—not simulations. The harness interacts directly with the live production API, using the same MCP search and recall mechanisms deployed by actual agents, configured at top_k=30 with 1-hop edge expansion.

This methodology generalizes across platforms including Claude Code, ChatGPT, and any LLM agent supporting MCP. For developers seeking continuity-free agent workflows, upgrading from flat files to a structured knowledge graph delivers measurable gains.

Benchmark details are publicly available at:

https://github.com/corbym/locomo-recordari

Test the LoCoMo benchmark yourself via anonymous access:

https://admin.recordar.io/start