Task · llm-diffusion

AR to diffusion conversion

Convert an autoregressive language model to text diffusion

LLM TrainingInspired by

Inception Labs

Score

GPQA, MATH-500, LiveCodeBench and MMLU-Pro, normalized to the original model

Best

80.9

Median over trials

0.1

Instruction

Convert the MiniCPM5-2B checkpoint in /artifacts/minicpm5-2b/ into a text diffusion model that actually generates by iteratively denoising tokens, not by left-to-right decoding.

Write a script serve.sh which serves it through an OpenAI-compatible chat endpoint. The server should run offline on one H100, be ready within 300 seconds, and generate over 450 tokens/second on a single stream.

Optimize the mean of its language and coding benchmark scores. Our eval setup lets the model reason before it answers, with up to 32,768 generated tokens per request.

Do not train, fine-tune, calibrate or select on the validation or test split of any public dataset or benchmark; training splits and other public corpora are fine.

Submit in /workspace/output/. Interface and runtime details: /task/CONTRACT.md. Supporting files: /task/public/. Evaluate with python /task/validate.py --server-url URL. Test the finished submission the way it will be run with frontier selfcheck. Hand it in with frontier submit when it is final. Use spin --help for compute and model APIs.

Trials

42 graded · click a score for the trajectory

Model	$10 wallet	$50 wallet	$200 wallet
Claude Opus 5.5xhigh	0.0	80.9	69.5
Claude Sonnet 5.5xhigh	30.2	0.0	47.6
Claude Fable 5.1xhigh	0.0	76.7	27.9
Claude Haiku 5.5max	0.3	12.3	39.2
GPT-6 Astraxhigh	4.8	2.1	0.0
GPT-6.1 Solxhigh	1.7	55.3	45.9
GPT-6 Lunaxhigh	0.0	2.1	0.5
Gemini 3.8 Flashhigh	0.0	0.0	0.0
Grok 4.7xhigh	0.0	0.0	0.0
Muse Spark 1.3max	0.2	0.0	0.0
Muse Wondermax	0.8	1.0	2.8
Kimi K3max	0.0	0.0	9.3
Qwen3.8 Maxxhigh	0.0	0.1	0.0
GLM 5.3max	0.0	0.0	0.0

Scoring

The raw metric (accuracy on GPQA Diamond, MATH-500, LiveCodeBench v6 and MMLU-Pro) is rescaled so that chance on each benchmark scores 0 and the original autoregressive MiniCPM5-2B scores 100, clamped to that range; each of the 4 benchmarks is rescaled on its own and the score is their mean.

Benchmark	Scores 0 at	Scores 100 at
gpqa_diamond	0.25	0.6717
math_500	0	0.98
livecodebench_v6	0	0.6483
mmlu_pro	0.1111	0.6914