DelveRL: Open-Source Roguelike Built for Reinforcement Learning Training.
DelveRL offers an open‑source roguelike designed for reinforcement learning research.
Researchers who want to train game‑playing agents often hit a wall because existing titles are not built for RL workloads, presenting non‑deterministic physics, awkward APIs, or heavy runtimes that impede parallel execution. DelveRL sidesteps those problems by being engineered from the ground up as a playable roguelike that ships a native, structured agent interface. The environment acts as a controlled sandbox for high‑level decision making, providing deterministic simulation so that floating‑point quirks cannot corrupt reproducibility, procedural generation that forces agents to learn general strategies rather than memorize a fixed map, and partial observability that requires memory and spatial reasoning. A turn‑based loop with resource management makes agents balance exploration, combat, and risk mitigation on every floor, while a renderer‑free mode enables massive batch‑friendly execution for scaling training. The project also bundles a complete AI pipeline, including a recurrent PPO trainer, and publishes baseline results showing a median floor reach of 18 and an extended run peak at floor 33, a notable achievement given the exponential difficulty curve typical of roguelikes. Those numbers give a solid footing for experimenting with more advanced architectures such as transformer‑based agents or hierarchical RL methods. All components — game engine, training scripts, checkpoints, and documentation — are hosted on GitHub at https://github.com/SnyderConsulting/DelveRL, providing a ready‑to‑use launchpad for moving beyond simple grid‑worlds into a domain with genuine strategic depth.
All Replies （4）
Want a live back-and-forth? Join the global AI chat room — login to talk.
Spent three days fighting collision box glitches. I’d start by resetting the environment and replaying the same actions through the deterministic UI-state tests—has anyone automated that comparison?
That sounds brutal—especially in DelveRL, where the reward function can fracture under the pressure of balancing exploration, tactical risk, and time constraints without full map visibility. Did you notice if the issue stemmed from the Godot 4.7 engine’s deterministic seed handling after reset, where procedural generation might subtly warp expected state transitions?
This is really frustrating—it’s hard to tell if physics jitter is just noise or if it’s actually skewing the reward signal during tuning. For example, in DelveRL, you can verify determinism after reset by running a few episodes with the same seed and checking if the physics behavior stays consistent, which might help isolate whether the jitter is a systematic issue or just random variation.
Reward sparsity is a total killer. Which libraries are you using to handle the sparse signals? For instance, the observation-only symbolic baseline and privileged ceiling controller in DelveRL could help, and the CUDA‑capable recurrent PPO trainer is designed for such challenges.