AI that "self-evolves" to discover machine-learning algorithms — "MLEvolve"

This work (MLEvolve) lets AI discover machine-learning (ML) algorithms in a self-evolving way.

LLM agents are increasingly applied to long-horizon tasks such as scientific discovery and machine-learning engineering (MLE), where sustained self-evolution is a key capability. But existing MLE agents suffer from inter-branch information isolation, memoryless search, and a lack of hierarchical control — together hindering long-horizon optimization.

MLEvolve is an LLM-based self-evolving multi-agent framework for end-to-end ML algorithm discovery. By extending tree search to Progressive MCGS, it enables cross-branch information flow through graph-based reference edges and gradually shifts the search from broad exploration to focused exploitation with an entropy-inspired progressive schedule. To let the agent evolve with accumulated experience, it introduces Retrospective Memory, combining a cold-start domain knowledge base with a dynamic global memory for task-specific experience retrieval and reuse. For stable long-horizon iteration, it further decouples strategic planning from code generation with adaptive coding modes.

On MLE-Bench, MLEvolve achieves state-of-the-art performance across multiple dimensions including average medal rate and valid submission rate under a 12-hour budget (half the standard runtime). It also outperforms specialized algorithm-discovery methods including AlphaEvolve on math-algorithm optimization tasks, showing strong cross-domain generalization.