Self-evolving ML framework tops benchmarks in 12 hours

A self-evolving AI framework just cut 12 hour ML trials in half.

The team reports a new LLM based system called MLEvolve, a self-evolving multi-agent framework designed to discover machine learning algorithms end to end. Rather than treating search as a single monolithic process, the approach extends tree search into Progressive Monte Carlo Graph Search, enabling cross-branch information flow through graph based reference edges. In practice, this enables ideas and results from one branch to inform others, which helps the overall search progressively narrow from broad exploration to focused exploitation as the run proceeds. To keep learning from experience without locking the system into stale behavior, the authors introduce Retrospective Memory. It blends a cold start knowledge base with a dynamic global memory for task-specific experience retrieval and reuse. The result is a more stable long-horizon iteration, achieved in part by decoupling strategic planning from code generation with adaptive coding modes that preserve flexibility while tightening execution.

The paper shows that this combination delivers state of the art performance on MLE-Bench across multiple dimensions, including an improved average medal rate and a higher valid submission rate under a 12-hour budget, effectively delivering better results in half the time compared with the previous standard. Benchmarks indicate the framework not only excels on ML algorithm discovery tasks but also outperforms specialized methods such as AlphaEvolve on mathematical algorithm optimization tasks, signaling strong cross-domain generalization. The authors emphasize that the framework can evolve with accumulated experience rather than being a fixed pipeline, a design choice they argue is essential for long-horizon optimization in complex ML engineering tasks.

From an engineering perspective the key constraint is the time budget. The 12-hour run gives practitioners a concrete, repeatable limit to optimize for when designing automated ML pipelines. The result is not merely faster runs. It is a shift in how search strategy is allocated over time. The team notes that the progressive schedule, inspired by entropy-based reasoning, helps prevent early premature convergence and keeps useful exploration paths alive longer, while gradually increasing exploitation as confidence grows. The architecture also treats planning and coding as separate concerns, reducing the risk that rapid code generation degrades strategic direction or introduces brittle configurations.

For ML engineers and product leaders, the emergence of a robust memory system is a practical hinge point. Retrospective Memory helps reuse prior successes and mistakes, which can dramatically shorten warm starts on new tasks, but it does raise questions about memory hygiene and staleness. Careful indexing and retrieval are essential to keep the system from reusing outdated heuristics. The paper shows these risks are mitigated by the memory design, yet it remains a live area for engineering teams to monitor in production environments.

Two to four concrete practitioner insights follow from the work.

1. First, the time budget itself is a design constraint that shapes the search strategy. If you reset batteries too often or too slowly, you lose the benefit of the progressive schedule.

2. Second, decoupling planning from code generation is not a silver bullet. It demands clean interface contracts and modular components so that evolving plans can be translated into robust code without destabilizing the overall search.

3. Third, memory management, both knowledge and experience, requires disciplined curation to avoid stale or conflicting signals that derail long-horizon optimization.

4. Fourth, cross-domain generalization looks promising here, but practitioners should validate on domain specific tasks before scaling, since the framework's advantages may hinge on the structure of the search problem and the quality of reference edges used to propagate information.

The paper shows robust gains, and the team reports that code for MLEvolve is available on GitHub, inviting practitioners to test and extend the framework within their own ML engineering stacks. As ML teams push toward more autonomous long-horizon optimization, MLEvolve offers a concrete blueprint for how to orchestrate memory, planning, and cross-branch collaboration to accelerate discovery without sacrificing stability.