Inverse dynamics model
- Score
- 56.8
- Wallet spend
- $2.70 of $10
- Model cost
- $97.90
- Agent time
- 4h 28m
- Steps
- 174
- Tokens in / out
- 47.5M / 257k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 3rd of 14
Score
56.8raw metric 0.5684
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 488
- fp
- 310
- fn
- 431
- precision
- 0.6115
- recall
- 0.5310
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 694.9980
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceTraining used public mlfoundations-cua-dev corpora that are declared with pinned revisions, plus Xvfb recordings the par
Training used public mlfoundations-cua-dev corpora that are declared with pinned revisions, plus Xvfb recordings the participant generated. The corpora are llamafactory-idm-data (synthetic_editor_videos_frames.zip, random_synthetic_sft_data.zip) and cua-idm-frame-data (idm_dataset_real_websites_v2.zip). Sources are documented in architecture.json and DATA_MANIFEST.md, and the trajectory has the download commands and logs: vm_setup2.sh pulled Qwen3-VL-4B-Instruct, the editor and realweb zips, and gen_pairs.tar. The only idm-eval material used was the single public example clip, used only to sanity-check thresholds and the pipeline, which the task allows. For p-doom datasets the session only listed names and read READMEs (AGI-CAST and the idm model card). It downloaded no idm-eval clips or labels, no AGI-CAST-idm-actions outputs, and no private inductionlabs data.
✓non specialist initializationEvery round starts from Qwen/Qwen3-VL-4B-Instruct, a general VLM pinned to a declared revision and downloaded in VM setu
Every round starts from Qwen/Qwen3-VL-4B-Instruct, a general VLM pinned to a declared revision and downloaded in VM setup. Training is LoRA (train_pair.py). Round 3 resumed from the participant's own round-2 adapter. export_merge.py loads the base with AutoModelForImageTextToText.from_pretrained, applies PeftModel merge_and_unload and saves the result. The shipped safetensors contain the standard Qwen3-VL-4B tensor layout with LoRA already merged: embed 151936x2560, q_proj 4096x2560, k/v_proj 1024x2560, MLP 9728. For p-doom/idm the session read only the model card README. No specialist IDM weights were downloaded or used as initialization.
✓actual inference uses reviewed weightsserve.sh installs the vendored wheels offline, sets IDM_MODEL_DIR=$HERE/model and HF_HUB_OFFLINE=1, then runs serve.py.
serve.sh installs the vendored wheels offline, sets IDM_MODEL_DIR=$HERE/model and HF_HUB_OFFLINE=1, then runs serve.py. serve.py builds an IDM object (idm_model.py) that loads the local model/ directory with AutoModelForImageTextToText.from_pretrained; no adapter is set by default. POST /actions decodes the uploaded video. Hand-written rules (rules.py/heur.py) detect MouseMove/MouseScroll, and the local merged VLM predicts KeyPress/MouseClick on changed frame pairs. The request id is only logged. There are no remote API calls and no label lookup tables.
✓harness credentials used only by the harnessSearches of the code and trajectory found no use of ANTHROPIC_API_KEY, harness base URLs, or launches of other harness C
Searches of the code and trajectory found no use of ANTHROPIC_API_KEY, harness base URLs, or launches of other harness CLIs. Credential use in the session was limited to HF_TOKEN for Hugging Face downloads (passed to vm_setup2.sh) and the Spin compute/storage CLI, both explicitly permitted.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed1h 59m into the runtook 4m 36s
startup 35.5 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 35.5 s
- ✓POST /actions · {"actions": 10, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-3,46"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}, {"frame": "F04", "type": "MouseMove", "details": "5,5"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 5.50 s, the first (cold) 15.85 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 44.1 s on the server the smoke request warmed up, 237.1 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 10.3 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed2h 47m into the runtook 4m 12s
startup 33.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 33.7 s
- ✓POST /actions · {"actions": 9, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-3,46"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}, {"frame": "F04", "type": "MouseMove", "details": "5,5"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 4.46 s, the first (cold) 13.15 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 35.8 s on the server the smoke request warmed up, 192.2 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 8.7 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 3passed4h 24m into the runtook 4m 20s
startup 33.2 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 33.2 s
- ✓POST /actions · {"actions": 9, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-3,46"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}, {"frame": "F04", "type": "MouseMove", "details": "5,5"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 4.62 s, the first (cold) 13.14 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 36.9 s on the server the smoke request warmed up, 198.6 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 8.5 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Third selfcheck passed with the round-3 model. Final housekeeping: record notes to memory, check spend, release the GPU, and preview the submission.