Mini-AGI actually trains on 8GB VRAM without exploding

Calling a project "Mini-AGI" is a bold move that usually invites a shower of rocks, but this one actually has some technical meat on its bones. The core claim here is a dynamic continual learning model that manages to train on a measly 8GB of VRAM. For those of us tired of the "you need a H100 cluster to breathe" era of LLMs, this is a refreshing change of pace.

The project is basically a rebellion against the fact that we can fine-tune 1B+ models on consumer gear, but we can't actually train them from scratch without selling a kidney. The goal was to get full control over the training data instead of just trusting whatever corporate slurry OpenAI or Google fed their models.

How it handles the VRAM bottleneck

The author managed to dodge the massive memory requirements using two specific architectural gambles:

• Dynamic MoE (Mixture of Experts): Instead of a static wall of parameters, the model adds and prunes experts during training. Only a tiny subset of experts are active at any given moment. This effectively shifts the bottleneck from VRAM to disk space, as experts are loaded and unloaded on the fly.
• Batch 1 Training: This is the real trick. By training on a single continuous stream of data, the model avoids the need to store massive randomized batches and their corresponding gradients, which is usually what kills your GPU memory.

Current progress and training stats

If you're expecting to download the weights today, stop. The model is currently chewing through a corpus of 7.8B characters. The training process involves reading interleaved passages of 32K characters each as a single, continuous stream.

According to the project's current pace, the weights are still "cooking" and won't be ready for another couple of weeks. The author even shared a scaling law graph that they claim looks promising, though we'll see if that holds up once the weights actually hit the public.

Getting it running

The setup is straightforward if you want to watch the process yourself. Since it's designed for low-VRAM environments, you don't need a server farm to test the implementation.

git clone https://github.com/volotat/mini-AGI

The author admits to brainstorming the architecture with Claude, which proves that using AI to build AI is the only way we're actually getting anything done these days. It's a lean approach to training that prioritizes disk swapping and stream-processing over raw hardware brute force. Whether it actually achieves "AGI" (even a mini one) remains to be seen, but the 8GB VRAM footprint is a win for the home-lab crowd.