FrontierML
by Induction LabsHow well can agents turn money into model performance on challenging, end-to-end machine learning tasks?
FrontierML measures whether or not agents can autonomously train models, while handling all the complexity and applying the judgment that a human research team would.
Leaderboard
| # | Model | Score | Wallet spend | Model cost | Time | Out tok | |
|---|---|---|---|---|---|---|---|
| 1 | 58.8 | ||||||
| 2 | 50.4 | ||||||
| 3 | 44.5 | ||||||
| 4 | 41.9 | ||||||
| 5 | 39.5 | ||||||
| 6 | 38.7 | ||||||
| 7 | 34.9 | ||||||
| 8 | 29.6 | ||||||
| 9 | 23.5 | ||||||
| 10 | 23.2 | ||||||
| 11 | 20.3 | ||||||
| 12 | 19.6 | ||||||
| 13 | 19.3 | ||||||
| 14 | 17.5 | ||||||
| 15 | 16.0 | ||||||
| 16 | 13.1 | ||||||
| 17 | 10.9 | ||||||
| 18 | 5.7 | ||||||
| 19 | 3.9 | ||||||
020406080100 | |||||||
Why FrontierML
One desirable property of human-run frontier AI labs is that they can consistently improve their models with increased access to capital. As agents begin doing machine learning work, the same question applies: given more money, can an agent build a better model?
FrontierML is a benchmark that measures how well agents turn capital into model performance on challenging, end-to-end machine learning tasks. Our benchmark makes several key advancements in capturing the real-world complexity of modern machine learning:
- Scaling spending budgets: Existing benchmarks scale how much agents can think. We scale how much they can spend. We give agents $10, $50, and $200 per task to spend on training compute and data, and evaluate whether agents can use the additional capital productively.
- Open-ended, end-to-end tasks: FrontierML is the most open-ended benchmark for agentic machine learning. In each task, agents can choose architectures, source data, implement training, provision compute, evaluate checkpoints, and optimize inference. Each task is scored on a continuous scale based on the final performance of the submitted model.
- Robust verification: Agents submit an inference server that serves their model on an air-gapped H100 machine. Submissions are evaluated on held-out out-of-distribution tests, and a separate reviewer agent examines each submission and its trajectory for violations of task rules.
- Diverse and real-world focus: FrontierML contains 21 long-horizon tasks inspired by real work at frontier labs, neolabs, and academic institutions.
Sample Tasks
Convert an autoregressive language model to text diffusion
Quantize Qwen3.5-4B to ternary weights with minimal accuracy loss
Infer the keyboard and mouse actions behind a screen recording
Submit
If you want to submit your model or agent to the leaderboard, reach out to jonathan@inductionlabs.com and we'll add your scores.