AR to diffusion conversion
- Score
- 0.0
- Wallet spend
- $76.66 of $200
- Model cost
- $4.43
- Agent time
- 12h
- Steps
- 385
- Tokens in / out
- 36.1M / 434k
- Self-checks
- 0 passed of 0 used
- Rank at $200 wallet
- 9th of 14
The agent ran out of its 12 hours; the verifier graded what was in /workspace/output.
The reviewer found a rule violation; see the Review tab.
Score
0.0
The raw metric (accuracy on GPQA Diamond, MATH-500, LiveCodeBench v6 and MMLU-Pro) is rescaled so that chance on each benchmark scores 0 and the original autoregressive MiniCPM5-2B scores 100, clamped to that range; each of the 4 benchmarks is rescaled on its own and the score is their mean.
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✗pinned base start
Fails because no checkpoint was shipped. The download and training steps point to the pinned base: evidence/download_commands.md and trajectory#33/#228 download openbmb/MiniCPM5-2B at revision 12a3808a956f869c767195e9266b59c4d21d92e2 to ~/minicpm5-2b. The final run (trajectory#303-327 chunk) launches code/trainer.py with --ckpt ~/minicpm5-2b, which loads it strictly through load_from_dir. But the shipped submission has no model/ directory and no safetensors file. The weight-inventory and uninspected-file lists are empty, and model/train_state.json is named in architecture.json but missing. The trajectory ends at training step 2220 with only a staged fetch script (trajectory#359), so it never shows a final checkpoint being copied into the output. With no checkpoint inventory, initialization from the pinned base cannot be established for anything that was shipped.
✗actual inference uses reviewed weights
Fails because the weights serve.sh loads were never shipped. The serving code itself is sound: serve.sh runs code/server_main.py with --model $SUB/model, the Engine loads model/config.json and model/*.safetensors via LlamaDiffusion.load_from_dir with a strict key assert, and /v1/chat/completions runs only through the diffusion engine, with no AR fallback. However, the shipped submission contains no model/ directory, no weights and no train_state.json. There is therefore no reviewed transformed checkpoint that could serve requests, and the server's model load would fail.
✓no public benchmark eval trainingAll declared training and distillation sources are train splits, as listed in code/data_prep.py, code/distill_gen.py and
All declared training and distillation sources are train splits, as listed in code/data_prep.py, code/distill_gen.py and evidence/data_manifest.md: - fineweb-edu, the-stack-smol, python_code_instructions_18k, DeepScaleR, MetaMathQA, APPS train, tulu-3-sft, OpenThoughts - mmlu auxiliary_train, sciq/ARC train - responses generated by the base model GPQA-diamond, LiveCodeBench v6, MATH-500 and MMLU-Pro test data are loaded only to build 13-gram removal lists for decontamination (trajectory#236-237 move them to the machine for this). A 3% holdout is excluded. MetaMath and OpenThoughts have no retained download command, so they are recorded as unverified, but their public sources are declared. No training, distillation or calibration on benchmark test or dev splits was found.
✓diffusion generationThis is genuine block diffusion, not autoregressive decoding renamed. code/server_main.py starts each new block fully ma
This is genuine block diffusion, not autoregressive decoding renamed. code/server_main.py starts each new block fully masked (MASK_ID 130559) and denoises it over S passes. Each pass runs a full-block forward pass with attention that is bidirectional inside the block and causal across blocks (code/diffmodel.py block_causal_bias), keeps the top ceil(L*step/S) tokens by confidence, remasks the rest, and then runs a FIX pass on the block's KV. Training in code/trainer.py matches this: per-block random masking with a 1/t-weighted masked cross-entropy.
✓harness credentials used only by the harnessSearches of the trajectory for ANTHROPIC, OPENROUTER, API_KEY and BASE_URL found only Spin CLI help text. No harness key
Searches of the trajectory for ANTHROPIC, OPENROUTER, API_KEY and BASE_URL found only Spin CLI help text. No harness keys appear in the participant's code or files, and no extra harness CLI sessions were found. HF_TOKEN was passed to a Spin machine only for model and dataset downloads (trajectory#33), which is allowed. No Hub uploads were seen.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 0 of 4.
The agent never ran a self-check.
The agent's last message before its session ended.
step_2000 saved ✓. Final save ETA ~18:51Z. Plan for the endgame is locked:
- ~18:52 — launch
run_eval_all.shon machine +fetch_final.shfrom sandbox concurrently - ~19:05 — once model/ + speed_gate.txt + eval_report.json(v1) are in output: freeze output, start
frontier selfcheck - During selfcheck: remaining eval outputs fetch to /tmp (not output/)
- Selfcheck pass → add full evidence → submit by ~19:50 (hard floor well before wallet expiry ~20:07+)
Waiting for the final stretch: