AR to diffusion conversion
- Score
- 0.0
- Wallet spend
- $44.21 of $50
- Model cost
- $3.53
- Agent time
- 8h 59m
- Steps
- 1,355
- Tokens in / out
- 136.7M / 1.8M
- Self-checks
- 0 passed of 0 used
- Rank at $50 wallet
- 11th of 19
Score
0.0
The raw metric (accuracy on GPQA Diamond, MATH-500, LiveCodeBench v6 and MMLU-Pro) is rescaled so that chance on each benchmark scores 0 and the original autoregressive MiniCPM5-2B scores 100, clamped to that range; each of the 4 benchmarks is rescaled on its own and the score is their mean.
Verifier details
- failure
- throughput
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓pinned base startarchitecture.json declares openbmb/MiniCPM5-2B at revision 12a3808a956f869c767195e9266b59c4d21d92e2. logs/model_download
architecture.json declares openbmb/MiniCPM5-2B at revision 12a3808a956f869c767195e9266b59c4d21d92e2. logs/model_download.log shows that pinned base being downloaded to /home/dev/model. logs/export.log shows the same base being loaded, the run-4 LoRA (step 1577) being merged into it, and the config being written with diffusion_mask_id 130074. The shipped model/model.safetensors inventory holds about 5.03 GB of BF16 Llama-layout tensors (vocab 130560, hidden 2048, MLP 6144, k/v 256). That matches the base's shape and is consistent with a merged LoRA. architecture.json says run 4 was shipped, while dataset_provenance.md calls run 5 the final stage. This is a documentation inconsistency only and does not change the initialization from the pinned base.
✓no public benchmark eval trainingtrain/gen_fast.py build_prompts loads only training splits: cais/mmlu auxiliary_train, openai/gsm8k train, EleutherAI/he
train/gen_fast.py build_prompts loads only training splits: cais/mmlu auxiliary_train, openai/gsm8k train, EleutherAI/hendrycks_math train, deepmind/code_contests train and m-a-p/CodeFeedback-Filtered-Instruction train. Responses are self-distilled from the unmodified base, and train/build_corpus.py only assembles those generations. All of these sources are declared in dataset_provenance.md. No download logs for these public datasets were checked, so their retrieval is unverified. The sampler sweeps (logs/sweep_*.sh, logs/sampler_sweep.log) run sweep3.py on the holdout of the self-distillation corpus (run4/holdout.pt), per trajectory#916. trajectory#553 only grepped gpqa_diamond.csv incidentally while looking for harmonic-mean code. logs/public_gpqa_validation.log is a development evaluation run, not training, calibration or selection data. No benchmark files were found in any training or selection path.
✓diffusion generationlib/graph_sampler.py prefills the prompt causally, then fills each 256-token block starting from all mask tokens (<unk>,
lib/graph_sampler.py prefills the prompt causally, then fills each 256-token block starting from all mask tokens (<unk>, id 130074). On each step it runs a bidirectional forward pass over the block (lib/graph_forward.py captures the forward with is_causal=False and uses a shifted read-out). lib/sampler.py select_commits then commits, in any order, every masked position whose confidence is at least 0.9, with at least 3 per step. Completed blocks are refreshed into the KV cache. This is genuine iterative block-wise masked denoising, not autoregressive decoding.
✓actual inference uses reviewed weightsserve.sh runs `python server.py --model-dir $HERE/model`. In server.py, Engine.load calls build_diffusion_model (lib/dif
serve.sh runs `python server.py --model-dir $HERE/model`. In server.py, Engine.load calls build_diffusion_model (lib/diffusion_model.py, which uses AutoModelForCausalLM.from_pretrained on the shipped model dir and calls make_bidirectional), wraps the result in FastLlama (lib/fast_forward.py, a plain forward over those weights), and serves requests through GraphDiffusionSession, falling back to DiffusionSession. The reviewed weights are model/model.safetensors. I found no alternative model, remote API or substitute weights in the serving path.
✓harness credentials used only by the harnessI searched the trajectory for ANTHROPIC, BASE_URL, OPENROUTER, API_KEY, claude -p, printenv, imports of anthropic/openai
I searched the trajectory for ANTHROPIC, BASE_URL, OPENROUTER, API_KEY, claude -p, printenv, imports of anthropic/openai, and codex. None of these showed the participant reading or using harness credentials or calling external model APIs. Data and model downloads went through the public HF datasets and model hub, and compute went through the provided spin tool.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 0 of 4.
The agent never ran a self-check.
The agent's last message before its session ended.
Gate passes (min 481.5 tok/s, mean 504). Copying the final logs and submitting: