Simple self-hosted LLM assistant with user-steered compounding
I built a lightweight personal LLM assistant running on Cloudflare Workers + Durable Objects. The idea is simple: you pick a category and topic when starting a conversation, and the backend keeps a running summary per category/topic—so context compounds naturally across sessions without any RAG or embeddings overhead.
No agentic magic, no vector stores. Just a clean singleton Durable Object that maintains state, Workers AI as the default provider, and optional OpenRouter support for bigger models. It runs free on Cloudflare's free tier with the default setup.
I started this to learn Durable Objects and turn a rough idea into something usable. After losing my job recently, I spent more time refining it with friends' feedback—focusing on simplicity in deployment and daily use. This week I open-sourced it so others can try.
Repo: https://github.com/kol3x/pawmc
Details:
- $0 cost on Cloudflare free tier (default config)
- Everything in a singleton Durable Object
- Workers AI as the default provider
- OpenRouter support for stable access and larger models (paid tokens)
Story tracker · related coverage
LLM Routers: The Rise of a New Infrastructure Category
5d ago
All Replies (3)
Q
Quinn48
Advanced
1h ago
Ran mine on a $5 DigitalOcean droplet—works surprisingly well for basic Q&A, though I had to trim the model size down to 7B to fit within memory.
0
S
Tried something similar on a Raspberry Pi 4—needed to offload context to disk after 2K tokens.
0
N
Curious about the context window handling—do you truncate or summarize when conversations get long?
0