Inverse dynamics model
- Score
- 55.7
- Wallet spend
- $9.29 of $10
- Model cost
- $1.75
- Agent time
- 5h 55m
- Steps
- 674
- Tokens in / out
- 67.2M / 605k
- Self-checks
- 3 passed of 4 used
- Rank at $10 wallet
- 4th of 19
Score
55.7raw metric 0.5569
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 511
- fp
- 405
- fn
- 408
- precision
- 0.5579
- recall
- 0.5560
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 1206.7400
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe training data is declared and comes from public training sources. The videos are p-doom/AGI-CAST-0.6k and the labels
The training data is declared and comes from public training sources. The videos are p-doom/AGI-CAST-0.6k and the labels are p-doom/AGI-CAST-idm-actions actions/actions.parquet. Both are pinned to a revision in data_manifest.json and architecture.json, and train/download_assets.py fetches them. The labels are weak labels predicted by an IDM. They are still a public training corpus, and the policy does not forbid them. Selection (rebalancing target, checkpoint, resolution and merge rule) used only the allowed public validation clip. p-doom/idm-eval is a different dataset: 44 macOS clips named clips_recording_{uuid}. Nothing suggests it is a subset of AGI-CAST. In the session, step 25 fetched only the idm-eval README dataset card, a description of structure and stats. No idm-eval clips, action JSONs, annotations or gesture GT were downloaded, inspected or used. inductionlabs/frontier-ml-tasks private data was never touched. No validation or test split of another benchmark was used for training.
✓non specialist initializationThe base model is Qwen/Qwen3-VL-8B-Instruct at revision 0c351dd0, a general VLM rather than an IDM. On the GPU box it ca
The base model is Qwen/Qwen3-VL-8B-Instruct at revision 0c351dd0, a general VLM rather than an IDM. On the GPU box it came from snapshot_download in train/download_assets.py (trajectory step 64 area, steps-57-75). The shipped weights/ came from `hf download Qwen/Qwen3-VL-8B-Instruct` (trajectory steps-297-315). Every training run passed --model-id .../models/Qwen3-VL-8B-Instruct. train/train_lora.py applies a fresh LoRA through get_peft_model. adapter/adapter_config.json records a LoRA r=64 on the Qwen base, and the weight inventory shows standard Qwen3-VL tensors. p-doom/idm and its descendants were never downloaded or used to initialize anything.
✓actual inference uses reviewed weightsserve.sh runs offline (HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE), points IDM_MODEL_DIR at weights/ and IDM_ADAPTER_DIR at
serve.sh runs offline (HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE), points IDM_MODEL_DIR at weights/ and IDM_ADAPTER_DIR at adapter/, and execs src/server.py. src/server.py loads Qwen3VLForConditionalGeneration from the local weights plus PeftModel from the local adapter. It generates actions over 10-frame windows, does a base-model KeyPress pass through disable_adapter, and applies a hand-written static-frame rule. There is no remote inference call. The clip id is only logged and is not used to look up labels by ID, hash or order.
✓harness credentials used only by the harnessIn the session, HF_TOKEN was used only for Hugging Face API and download calls, which the policy allows. Spin compute an
In the session, HF_TOKEN was used only for Hugging Face API and download calls, which the policy allows. Spin compute and R2 storage variables were used, and these are not harness credentials. Harness API keys and base URLs were never used, copied or echoed. The only OPENAI_API_KEY mention came while reading a reference repository script. No extra harness CLI (claude, codex, gemini) was spawned.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed2h 34m into the runtook 6m 24s
startup 73.5 s of 300 s; failed POST /actions: RuntimeError: http_status: failed 3 attempts: RuntimeError: Submission returned HTTP 422: b'{"detail":[{"type":"missing","loc":["query","request"],"msg":"Field required","input":null}]}' | RuntimeError: Submission returned HTTP 422: b'{"detail":
3 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 73.5 s
- ✗POST /actions · RuntimeError: http_status: failed 3 attempts: RuntimeError: Submission returned HTTP 422: b'{"detail":[{"type":"missing","loc":["query","request"],"msg":"Field required","input":null}]}' | RuntimeError: Submission returned HTTP 422: b'{"detail":[{"type":"missing","loc":["query","request"],"msg":"Field required","input":null}]}' | RuntimeError: Submission returned HTTP 422: b'{"detail":[{"type":"missing","loc":["query","requ
- 2passed2h 43m into the runtook 9m 2s
startup 75.8 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 75.8 s
- ✓POST /actions · {"actions": 27, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-1,1"}, {"frame": "F01", "type": "MouseMove", "details": "-1,1"}, {"frame": "F02", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 14.84 s, the first (cold) 16.88 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 121.7 s on the server the smoke request warmed up, 654.3 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 2.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 3passed4h 17m into the runtook 7m 21s
startup 59.9 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 59.9 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-1,1"}, {"frame": "F01", "type": "MouseMove", "details": "-1,1"}, {"frame": "F02", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 11.44 s, the first (cold) 12.44 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 91.5 s on the server the smoke request warmed up, 491.8 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 1.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 4passed5h 40m into the runtook 8m 26s
startup 62.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 62.3 s
- ✓POST /actions · {"actions": 15, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-1,1"}, {"frame": "F01", "type": "MouseMove", "details": "-1,1"}, {"frame": "F02", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 17.58 s, the first (cold) 17.02 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 136.3 s on the server the smoke request warmed up, 732.5 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.
Final state of the submission (/workspace/output, 146 files, 22.5 GB):
- Model —
Qwen3-VL-8B-Instruct(rev0c351dd0…, not a specialist IDM) + a rank-64 LoRA adapter trained on 4,149 ten-frame windows of public AGI-CAST screencasts, with the training mix rebalanced to the action-type mix of the provided example clip. - Serving —
serve.shinstalls the vendored cp312 wheels into/tmp/idm_envand starts a FastAPI server exposingPOST /actions; the clip is decoded once at 5 FPS and read twice — once with the adapter, once withdisable_adapter()(base weights) — and the two passes are merged with a clash rule derived from the evaluator's own tolerances (/task/actions.py). The base pass contributes key presses only; the probe (train/probe_base_localisation.py) showed its scrolls are a fixed position prior rather than a detection, so merging them would add ~1 FP per clip. - Public-clip score — micro-F1 0.8571 with the base pass (tp 12, fp 2, fn 2) vs 0.8148 tuned-only.
- Self-checks — 3 of 4 passed, including the final one (
6691023293a7): startup 62.3 s of 300 s, 3/3 gates passed, projected 43-clip batch within the 3600 s deadline and ≤ 2 failed clips. The one failure was the earliest version of the server, before the multipart/actionsendpoint existed. - Data hygiene — no clip of
p-doom/idm-evalwas ever downloaded, opened or trained on; only the single provided example clip was used, for monitoring and for choosing the action-mix target, serving resolution, adapter checkpoint and merge rule.data_manifest.jsonrecords every corpus, revision and download command.