Inverse dynamics model
- Score
- 67.1
- Wallet spend
- $55.84 of $200
- Model cost
- $33.86
- Agent time
- 4h 2m
- Steps
- 499
- Tokens in / out
- — / —
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 4th of 19
Score
67.1raw metric 0.6709
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 583
- fp
- 236
- fn
- 336
- precision
- 0.7118
- recall
- 0.6344
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 1323.9447
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenancearchitecture.json and data_manifest.json declare where the training data came from: public HF datasets p-doom/AGI-CAST-0
architecture.json and data_manifest.json declare where the training data came from: public HF datasets p-doom/AGI-CAST-0.6k and p-doom/AGI-CAST-idm-actions, each pinned to a revision, plus the allowed public example. prepare_agicast_data.py retains the hf_hub_download commands and explicit TRAIN_VIDEOS/VAL_VIDEOS lists. build_stage2_data.py only uses stage-1 AGI-CAST clips, extra AGI-CAST keypress chunks and repeated public-example windows. Selection (soup_eval.py, train_idm.py) uses held-out AGI-CAST train videos and the public example. In the trajectory, the participant read the agent-studio/IDM-Single test-set README but never downloaded or used it. There was no access to p-doom/idm-eval or inductionlabs/frontier-ml-tasks. The AGI-CAST-idm-actions labels are weak labels produced by p-doom/idm and published as a public corpus. That is a data source, not a forbidden eval split.
✓non specialist initializationtrain_idm.py attaches a LoRA adapter (get_peft_model) to Qwen/Qwen3-VL-8B-Instruct, a general vision-language model, pin
train_idm.py attaches a LoRA adapter (get_peft_model) to Qwen/Qwen3-VL-8B-Instruct, a general vision-language model, pinned in architecture.json. merge_and_save.py loads that same Qwen base, applies the trained adapter with PeftModel and calls merge_and_unload. soup_eval.py averages LoRA checkpoints that were all trained on the Qwen base. The trajectory shows the Qwen base fetched with snapshot_download and no download of p-doom/idm or any other specialist IDM weights. The shipped model/config.json and shard index match the Qwen3-VL architecture.
✓actual inference uses reviewed weightsserve.sh sets HF offline mode and IDM_MODEL_DIR to the shipped model directory, then runs server.py. server.py loads Aut
serve.sh sets HF offline mode and IDM_MODEL_DIR to the shipped model directory, then runs server.py. server.py loads AutoModelForImageTextToText.from_pretrained(model_dir, local_files_only=True) and decodes each uploaded video into frames with ffmpeg. It runs model.generate on frame windows, parses the output and applies pixel-diff filters. The request id is only printed to a log. There are no remote API calls and no stored-label lookup. The weights it loads are the shipped model-0000x-of-00004.safetensors shards.
✓harness credentials used only by the harnessIn the trajectory, participant commands used only HF_TOKEN for Hugging Face reads, which is allowed. The Gemini 429 mess
In the trajectory, participant commands used only HF_TOKEN for Hugging Face reads, which is allowed. The Gemini 429 messages were errors from the harness itself, not participant code. Spin compute commands are allowed. No GEMINI/ANTHROPIC/OPENAI keys or base URLs appear in the shipped code or in participant commands.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 12m into the runtook 7m 45s
startup 68.1 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 68.1 s
- ✓POST /actions · {"actions": 16, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-1,0"}, {"frame": "F01", "type": "MouseMove", "details": "-1,0"}, {"frame": "F02", "type": "MouseMove", "details": "-1,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 14.34 s, the first (cold) 14.47 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 114.6 s on the server the smoke request warmed up, 615.7 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed3h 52m into the runtook 8m 5s
startup 83.8 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 83.8 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-1,101"}, {"frame": "F01", "type": "MouseMove", "details": "-1,10"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 14.87 s, the first (cold) 15.70 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 119.2 s on the server the smoke request warmed up, 640.6 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.8 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests