Building Self-Improving RL Environments That Scale With Model

Rui and Michael built this after seeing firsthand how quant markets evolve: profitable strategies decay as more capital chases them, making new alpha increasingly difficult to extract. That's exactly the property we want in a benchmark.

Their environments give LLMs a realistic research pipeline: build predictive features, design portfolios, backtest strategies, and adapt to regime changes — all evaluated on out-of-sample data. Each task comes with purpose-built tools: a backtesting engine for testing features historically, an execution tool for live strategy deployment, and a final scoring mechanism that isolates specific skills like feature engineering.

Running SOTA models through these envs revealed consistent weaknesses: models prefer shallow broad searches over deep iteration, higher reasoning doesn't translate to better performance, and agents often give up when losing money instead of adapting their approach. The reward structure keeps evaluation honest — no LLM judges or human gatekeepers needed.

The key insight is that quant research is pure applied ML: long-horizon planning, continual learning, and iterative hypothesis testing. By embedding real market data and natural noise floors, EdotEnv creates transferable research skills rather than benchmark-specific tricks.

They've open-sourced a sample task repo for feature engineering at github.com/MMcollab-dotcom/feature-engineering, and plan to offer continuously evolving environments to AI labs focused on ML modeling, long-term planning, and autonomous research workflows.

If you're training agents on research tasks or building eval harnesses, this is worth examining — especially the reward design that stays meaningful even as capabilities advance.