All tasks
Task · llm-diffusion

AR to diffusion conversion

Convert an autoregressive language model to text diffusion

LLM TrainingInspired byInception Labs

Grading pipeline

  1. 1Collect · snapshotHarbor collects /workspace/outputimmutable submission · digest
    ›
  2. 2Reviewautomatic · trusted verifier4 task-local checks
    ›
  3. 3Preflighttask verifier checks config and databefore GPU provisioning
    ›
  4. 4Launch · readyoffline · 1× H100 80GBbash /submission/serve.shGET /health = 200 · ≤ 300 s
    ›
  5. 5Speed gatesingle-stream > 450 tok/s
    ›
  6. 6Benchmarksdev: GPQA 198 · LCB 1055held-out: MATH-500 500 · MMLU-Pro 1,400 of 12,032c=32
    ›
  7. 7Scoremean of the 4, equal weighteach normalized: chance 0, original model 1
    ›
  8. 8Harbor resultresult.json · reward.txt

Environments

Entry
Harbor → tests/test.sh → grade.py → frontier_ml.grading.grade
Verifier
23.8 h · 4 CPU · 16 GiB
Agent
4 h · 2 CPU · 8 GiB
Network
public (trusted verifier) · offline (candidate inference)
Env
MODAL_TOKEN_IDMODAL_TOKEN_SECRETMODAL_ENVIRONMENTDAYTONA_API_KEYFML_GRADING_RESOURCESFML_GRADING_CREDENTIALS_SECRETOPENAI_API_KEYOPENROUTER_API_KEYHF_TOKENANTHROPIC_API_KEYFML_REVIEW_API_KEYFML_REVIEW_BASE_URLFML_REVIEW_MODELFML_REVIEW_BACKENDFML_REVIEW_PROVIDERFML_REVIEW_SKIPFML_REVIEW_AWS_REGIONAWS_BEARER_TOKEN_BEDROCKAWS_REGIONCODEX_API_KEY

Verifier config

concurrency
32
inference_timeout_seconds
82800
require_h100
true
base_model
openbmb/MiniCPM5-2B
base_revision
12a3808a956f869c767195e9266b59c4d21d92e2
minimum_tokens_per_second
450
inference_replicas
4
verifier_lifetime_seconds
86400
lcb_max_concurrency
64
language_suite
gpqa-math500-lcbv6-mmlupro-v6
benchmark_paths
{"gpqa_diamond":"/tests/private/benchmarks/gpqa_diamond.csv","math_500":"/tests/private/benchmarks/math500.jsonl","livecodebench_v6":"/tests/private/benchmarks/livecodebench-v6","mmlu_pro":"/tests/private/benchmarks/mmlu-pro.jsonl"}
livecodebench
{}
generation_concurrency
32
max_tokens
32768
temperature
1
request_timeout_seconds
3600
judge_model
gpt-6-luna
judge_timeout_seconds
120

Architecture review

Base model openbmb/MiniCPM5-2B@12a3808. A trusted reviewer reads the submission and trajectory and fails it on any of these checks.

  1. pinned_base_start
    Provenance and actual checkpoint inventory establish initialization from the operator-configured pinned base checkpoint.
  2. no_public_benchmark_eval_training
    Trace the datasets, splits, data-generation recipes, and training/conversion logs. Task-time training, fine-tuning, distillation, and calibration must not use any public benchmark test or validation split, including unlabeled inputs, rewritten examples, and synthetic or teacher-generated targets derived from them. Supplied development examples are evaluation-only. This check covers work performed for this task, not the original pretraining of the prescribed base checkpoint.
  3. diffusion_generation
    Generation implements genuine iterative denoising/diffusion of token states, rather than ordinary left-to-right autoregressive decoding renamed as diffusion. Trace update schedule and inference execution.
  4. actual_inference_uses_reviewed_weights
    Trace serve.sh through loading and generation to establish that the reviewed transformed checkpoint and diffusion path are actually used for requests.