Ditching Vector DBs: State Management via Cloudflare Durable Objects

Most of us are conditioned to think that long-term memory for LLMs requires a RAG pipeline, a vector database, and a complex embedding strategy. But for a personal assistant, that overhead is often overkill. I recently open-sourced a project called pawmc that takes a different approach: user-steered compounding via stateful objects.

The architecture relies on Cloudflare Workers and Durable Objects (DO). Instead of searching through a database of chunks to find relevant context, the system uses a singleton Durable Object to maintain a running summary per category and topic. When you start a session, you define the context; as the conversation progresses, the backend updates a persistent summary. This allows context to compound across different sessions without the latency or "hallucinated retrieval" common in small-scale RAG setups.

From a technical perspective, the choice of Durable Objects is critical here. Because a DO is a singleton, it provides a consistent point of state. I’ve configured the system to use Workers AI as the default provider, which means the entire stack can run within Cloudflare’s free tier. If you need more reasoning power or a larger context window, there is built-in support for OpenRouter, though that obviously introduces token costs.

The primary goal was to eliminate "agentic magic" and focus on deterministic state. In a typical RAG flow, you're at the mercy of your embedding model's similarity score. In this compounding summary model, the user steers the memory. By categorizing topics, you effectively create a set of "memory slots" that the LLM can reference and update, ensuring that the most important takeaways from a previous conversation are preserved in the summary rather than lost in a sea of vector embeddings.

If you're looking to deploy this, you'll need the Wrangler CLI. The deployment is straightforward, but ensure your wrangler.toml is correctly configured for the Durable Object binding, otherwise, you'll hit a 500 Internal Server Error when the worker attempts to instantiate the state manager.

A few key specs for those interested in the implementation:
1. Cost: $0/month on the Cloudflare free tier using the default Workers AI models.
2. State: Managed entirely within a single Durable Object, avoiding the need for an external PostgreSQL or Pinecone instance.
3. Provider: Default is Workers AI, with an optional toggle for OpenRouter for access to frontier models.

This project grew out of a desire to see how far "simple" state management could go before needing a full-blown vector store. For a personal knowledge assistant, a well-maintained summary is often more reliable than a k-nearest neighbor search through 1,000 fragmented snippets of text.

The code is available at github.com/kol3x/pawmc. If you're tired of managing embedding pipelines for simple personal tools, this is a viable alternative.