All tasks
Task · llm-diffusion
AR to diffusion conversion
Convert an autoregressive language model to text diffusion
LLM TrainingInspired by
Inception Labs
Grading pipeline
Environments
- Entry
- Harbor → tests/test.sh → grade.py → frontier_ml.grading.grade
- Verifier
- 23.8 h · 4 CPU · 16 GiB
- Agent
- 4 h · 2 CPU · 8 GiB
- Network
- public (trusted verifier) · offline (candidate inference)
- Env
- MODAL_TOKEN_IDMODAL_TOKEN_SECRETMODAL_ENVIRONMENTDAYTONA_API_KEYFML_GRADING_RESOURCESFML_GRADING_CREDENTIALS_SECRETOPENAI_API_KEYOPENROUTER_API_KEYHF_TOKENANTHROPIC_API_KEYFML_REVIEW_API_KEYFML_REVIEW_BASE_URLFML_REVIEW_MODELFML_REVIEW_BACKENDFML_REVIEW_PROVIDERFML_REVIEW_SKIPFML_REVIEW_AWS_REGIONAWS_BEARER_TOKEN_BEDROCKAWS_REGIONCODEX_API_KEY
Verifier config
- concurrency
- 32
- inference_timeout_seconds
- 82800
- require_h100
- true
- base_model
- openbmb/MiniCPM5-2B
- base_revision
- 12a3808a956f869c767195e9266b59c4d21d92e2
- minimum_tokens_per_second
- 450
- inference_replicas
- 4
- verifier_lifetime_seconds
- 86400
- lcb_max_concurrency
- 64
- language_suite
- gpqa-math500-lcbv6-mmlupro-v6
- benchmark_paths
- {"gpqa_diamond":"/tests/private/benchmarks/gpqa_diamond.csv","math_500":"/tests/private/benchmarks/math500.jsonl","livecodebench_v6":"/tests/private/benchmarks/livecodebench-v6","mmlu_pro":"/tests/private/benchmarks/mmlu-pro.jsonl"}
- livecodebench
- {}
- generation_concurrency
- 32
- max_tokens
- 32768
- temperature
- 1
- request_timeout_seconds
- 3600
- judge_model
- gpt-6-luna
- judge_timeout_seconds
- 120
Architecture review
Base model openbmb/MiniCPM5-2B@12a3808. A trusted reviewer reads the submission and trajectory and fails it on any of these checks.
- pinned_base_startProvenance and actual checkpoint inventory establish initialization from the operator-configured pinned base checkpoint.
- no_public_benchmark_eval_trainingTrace the datasets, splits, data-generation recipes, and training/conversion logs. Task-time training, fine-tuning, distillation, and calibration must not use any public benchmark test or validation split, including unlabeled inputs, rewritten examples, and synthetic or teacher-generated targets derived from them. Supplied development examples are evaluation-only. This check covers work performed for this task, not the original pretraining of the prescribed base checkpoint.
- diffusion_generationGeneration implements genuine iterative denoising/diffusion of token states, rather than ordinary left-to-right autoregressive decoding renamed as diffusion. Trace update schedule and inference execution.
- actual_inference_uses_reviewed_weightsTrace serve.sh through loading and generation to establish that the reviewed transformed checkpoint and diffusion path are actually used for requests.