Browse the data

Every graded trial in the benchmark. Scan the heatmap, read the task catalog, or filter the trial table. Every cell opens the agent's full trajectory.

Wallet

Color by

Display

Rows

tasks × models · score (0–100)	mean	Claude Opus 5.5			Claude Sonnet 5.5			Claude Fable 5.1			Claude Haiku 5.5			GPT-6 Astra			GPT-6.1 Sol			GPT-6 Luna			Gemini 3.8 Flash			Grok 4.7			Muse Spark 1.3			Muse Wonder			Kimi K3			Qwen3.8 Max			GLM 5.3
tasks × models · score (0–100)	mean	$10 45.4	$50 52.4	$200 58.8	$10 45.5	$50 47.3	$200 50.4	$10 35.7	$50 37.4	$200 41.9	$10 28.2	$50 29.1	$200 34.9	$10 37.6	$50 42.8	$200 44.5	$10 31.6	$50 36.2	$200 38.7	$10 13.2	$50 13.3	$200 13.1	$10 8.1	$50 9.5	$200 5.7	$10 12.2	$50 10.8	$200 10.9	$10 14.1	$50 12.6	$200 16.0	$10 21.6	$50 17.2	$200 19.3	$10 16.2	$50 17.8	$200 20.3	$10 19.9	$50 19.4	$200 17.5	$10 18.6	$50 17.4	$200 19.6
AI-text detection	74.5	98.3	98.9	99.2	96.1	96.1	94.9	71.9	83.6	83.2	86.8	93.8	74.1	90.7	91.9	94.2	90.8	87.2	89.5	83.8	84.0	84.6	9.4	64.4	0.0	44.1	83.0	84.5	63.2	43.5	56.3	73.0	48.8	69.2	66.6	47.4	68.9	76.5	63.8	54.1	91.0	55.2	92.3
PII redaction	52.2	73.1	77.3	77.9	70.6	72.2	68.4	74.8	70.0	71.3	64.3	58.9	69.4	64.8	68.0	59.0	60.5	64.3	55.7	21.8	18.5	17.4	34.8	49.8	33.8	43.8	0.0	31.5	0.0	35.0	41.1	67.3	39.3	63.4	36.3	56.1	65.5	52.6	57.9	52.5	51.0	39.6	64.6
Video frame interpolation	48.4	91.1	86.7	87.8	86.0	86.3	93.2	78.5	81.8	58.2	63.7	64.3	65.4	85.3	91.2	84.6	87.8	72.0	86.5	0.0	0.0	25.7	0.0	7.1	0.0	0.0	0.0	0.0	23.7	49.8	31.2	68.8	49.8	55.9	0.0	32.5	79.5	31.2	0.0	0.0	26.9	45.7	52.3
Figure to TikZ	46.9	56.4	57.0	65.1	54.7	57.7	58.7	54.5	54.0	56.5	42.4	53.3	52.2	52.9	57.0	59.1	50.5	51.0	60.4	41.4	45.7	32.9	30.6	27.8	34.9	36.0	35.5	7.8	49.3	43.6	47.8	41.4	44.4	43.0	48.4	48.3	41.4	44.4	46.8	56.3	39.5	40.1	49.0
Inverse dynamics model	44.5	72.6	66.9	78.5	73.3	69.0	79.1	56.8	64.4	63.2	32.2	43.2	37.4	44.9	55.9	67.2	45.8	53.7	39.9	36.5	30.2	16.3	34.8	31.4	16.1	23.5	19.7	25.1	19.3	47.8	46.0	34.5	48.1	29.5	34.7	35.6	31.6	36.0	67.5	57.3	34.4	35.0	34.1
Data-efficient pretraining	35.1	65.1	83.1	99.8	47.8	71.1	80.5	26.9	40.9	80.6	49.5	67.2	49.2	59.5	77.8	74.5	40.9	77.5	66.1	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	16.3	0.0	0.0	0.0	33.8	18.9	5.5	0.0	13.7	36.3	34.0	0.0	49.5	41.0	37.8	28.1
Zero-shot classification	34.1	59.3	57.7	59.0	62.0	63.6	0.0	59.6	0.0	62.1	52.3	57.4	48.1	52.5	47.8	46.7	38.7	50.2	56.5	36.9	29.4	24.8	0.0	0.0	29.6	0.0	31.0	17.4	26.7	0.0	43.2	48.6	16.0	37.1	41.2	40.2	0.0	0.0	43.2	40.4	13.8	41.1	0.0
Screenshot generation	30.6	74.2	63.0	78.9	57.8	59.7	64.6	44.6	0.0	35.0	35.9	21.6	40.6	47.3	59.3	50.6	30.1	40.5	42.1	14.1	16.5	21.0	8.6	16.2	0.0	28.6	21.1	17.9	16.8	0.0	19.8	32.5	22.4	18.6	32.7	0.0	35.2	24.9	29.6	32.1	0.0	0.0	32.7
Voice endpointing	27.1	74.7	68.8	55.8	49.4	46.1	57.4	65.0	0.0	50.7	33.6	3.7	19.9	50.5	62.9	60.4	45.8	0.0	42.3	29.0	32.4	43.7	40.0	0.0	0.0	43.3	0.0	0.0	34.1	0.0	0.0	0.0	5.2	0.0	0.0	0.0	0.0	50.5	33.2	0.0	40.8	0.0	0.0
LLM post-training	26.4	57.6	43.3	67.4	49.2	49.3	45.2	26.0	46.2	0.0	40.9	17.4	61.3	51.6	48.3	67.2	11.5	65.7	36.6	0.0	0.0	0.0	0.0	0.0	0.0	0.0	16.1	0.0	35.3	0.0	0.0	0.0	34.8	48.8	13.3	26.1	12.9	0.0	38.1	0.0	24.9	41.2	34.3
OCR latent compression	23.5	26.9	54.4	82.6	55.8	67.5	63.9	56.1	67.9	67.8	8.6	13.8	30.1	36.4	57.5	64.1	37.6	43.7	35.2	2.1	0.4	0.7	2.2	0.0	0.8	26.1	4.9	4.8	0.0	0.0	0.0	15.3	3.3	1.9	7.1	15.5	0.0	6.6	4.6	5.3	4.2	6.3	5.7
Music transcription	21.8	61.9	61.0	64.3	73.8	79.5	74.5	28.8	47.2	28.1	0.0	0.0	0.0	49.4	73.4	76.7	51.1	35.1	73.1	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.6	0.0	0.0	0.0	4.6	0.0	13.3	2.6	0.0	9.5	8.2	0.0	0.0	0.0	0.0	0.0
Lean proof validity	19.8	37.5	45.3	57.5	41.8	31.9	57.9	28.6	40.7	43.2	26.6	26.6	41.5	37.5	29.8	35.8	36.3	27.7	33.1	1.4	15.7	5.0	0.0	0.0	2.5	6.0	0.0	12.1	4.3	20.7	0.0	0.0	7.0	0.1	17.6	17.2	20.6	18.0	0.0	0.0	0.0	0.0	3.0
Math proof simplification	15.6	33.9	29.4	40.5	31.6	37.5	34.9	21.4	29.8	25.5	20.1	18.0	27.4	21.5	28.0	24.3	15.6	14.8	8.5	2.8	0.1	0.0	4.6	2.6	2.1	5.3	6.5	2.2	14.3	20.5	19.2	11.4	6.7	3.3	16.3	11.2	15.0	23.0	15.2	0.0	11.7	0.7	0.0
Screenshot diffing	14.2	26.8	34.4	38.0	29.7	33.5	35.2	31.1	23.2	46.4	19.2	29.6	38.5	16.0	15.5	8.2	6.2	10.2	10.5	6.9	4.6	2.9	0.0	0.0	0.0	0.0	8.8	8.0	7.9	3.4	12.7	13.0	1.6	6.5	17.1	19.6	0.0	9.2	2.9	8.7	11.6	0.0	0.0
AR to diffusion conversion	12.2	0.0	80.9	69.5	30.2	0.0	47.6	0.0	76.7	27.9	0.3	12.3	39.2	4.8	2.1	0.0	1.7	55.3	45.9	0.0	2.1	0.5	0.0	0.0	0.0	0.0	0.0	0.0	0.2	0.0	0.0	0.8	1.0	2.8	0.0	0.0	9.3	0.0	0.1	0.0	0.0	0.0	0.0
Speculative drafting	11.7	21.2	37.0	41.8	20.8	33.8	48.4	15.6	26.8	29.5	5.5	18.6	21.3	16.8	16.5	25.8	8.1	0.0	25.2	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	8.7	9.5	13.1	5.2	0.0	9.9	0.0	0.0	0.0	7.0	0.0	12.0	13.7
Ternary quantization	8.3	22.7	36.5	49.1	24.7	18.9	41.2	7.7	29.3	28.9	8.7	1.2	7.9	1.3	0.2	19.5	1.5	2.1	4.6	0.0	0.0	0.1	0.4	0.0	0.0	0.5	0.2	0.0	0.0	0.5	9.3	0.0	0.0	0.2	5.5	0.0	1.4	2.5	5.4	3.7	0.0	10.1	2.3
SVG generation	3.1	0.0	5.2	18.0	0.0	8.2	13.9	2.7	2.0	20.3	0.0	10.0	10.1	0.0	8.3	14.0	3.2	7.2	0.0	0.0	0.0	0.0	5.3	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Pokémon Random Battle	1.1	0.0	7.4	2.7	0.0	11.9	0.0	0.1	1.0	1.5	1.1	0.0	0.0	6.4	7.2	3.1	0.0	1.8	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Vision for a ternary LM	0.2	0.4	6.5	2.4	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

0100