Inverse dynamics model
- Score
- 37.4
- Wallet spend
- $13.96 of $200
- Model cost
- $6.94
- Agent time
- 3h 4m
- Steps
- 283
- Tokens in / out
- 111.8M / 451k
- Self-checks
- 4 passed of 4 used
- Rank at $200 wallet
- 8th of 14
Score
37.4raw metric 0.3738
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 323
- fp
- 486
- fn
- 596
- precision
- 0.3993
- recall
- 0.3515
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 48.9254
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceAll training sources are declared in data_manifest.json and training/download_commands.sh, and train.py reads only the p
All training sources are declared in data_manifest.json and training/download_commands.sh, and train.py reads only the prepared recordings and screenshots. The sources are: - open-world-agents/D2E-480p: 55 selected recordings. - open-world-agents/OSTask-demo: all 5 recordings. - andrewt28/keystroke-typing-videos: train/ split only (train/metadata.jsonl and train mp4s). - vyokky/GUI-360: train screenshots used only as backgrounds. - Synthetic overlays built from the above. The trajectory supports this. At steps 25 and 43 only the typing train/ folder was listed and fetched; the test and validation folders were only named, never downloaded. At steps 26 and 44 the participant downloaded the OSTask recordings and the typing train metadata. Searches found no access to p-doom/idm-eval beyond the single allowed validation clip, and none to inductionlabs/frontier-ml-tasks private data. The holdout was carved from the training corpora. The provided validation clip was used only for sanity checks and the scroll threshold, which is permitted.
✓non specialist initializationWith no --init, training/train.py builds the encoder from timm 'resnet18.a1_in1k' (generic ImageNet weights, pinned revi
With no --init, training/train.py builds the encoder from timm 'resnet18.a1_in1k' (generic ImageNet weights, pinned revision in architecture.json). It copies the stem and layer weights and duplicates conv1 into 6 channels (cat(w,w)*0.5). Later stages v2, v3b and v4 resume from the participant's own earlier checkpoints, loaded strictly. No specialist IDM (such as p-doom/idm) was loaded anywhere. The checkpoint layout (enc.conv1.weight [64,6,7,7], enc.layer*, proj, transformer and task heads) matches idm_model.py.
✓actual inference uses reviewed weightsserve.sh installs local wheels and runs server.py. server.py builds IDM from idm_model.py, loads weights/model.safetenso
serve.sh installs local wheels and runs server.py. server.py builds IDM from idm_model.py, loads weights/model.safetensors with a strict load_state_dict, and samples the uploaded video at 5 FPS. It runs the model forward and decodes the outputs using config.json thresholds. The request id is ignored, and there are no remote calls, hidden backbones or label lookups. The trajectory shows the v4 export copied to /workspace/output/weights/model.safetensors (step 254 region) and the final server smoke test running on the output files.
✓harness credentials used only by the harnessSearches of the trajectory found no use, printing or copying of ANTHROPIC_API_KEY or any *_BASE_URL by the participant's
Searches of the trajectory found no use, printing or copying of ANTHROPIC_API_KEY or any *_BASE_URL by the participant's code. They also found no extra claude or codex CLI sessions. Credentials appeared only in the task prompt and documentation text. The participant used HF_TOKEN for Hugging Face downloads and the Spin CLI for compute, both of which are allowed.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed1h 35m into the runtook 51sfree (stopped before any GPU time)
failed review prerequisites: The review needs architecture.json and serve.sh at the top of the submission
1 gates
- ✗review prerequisites · The review needs architecture.json and serve.sh at the top of the submission
- 2passed1h 37m into the runtook 3m 26s
startup 61.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 61.3 s
- ✓POST /actions · {"actions": 2, "dropped_actions": 0, "warning": null, "first": [{"frame": "F01", "type": "MouseMove", "details": "-1,-3"}, {"frame": "F23", "type": "MouseMove", "details": "0,-1"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 2.89 s, the first (cold) 3.00 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 24.4 s on the server the smoke request warmed up, 131.1 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 3passed2h 24m into the runtook 2m 28s
startup 55.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 55.7 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-1,1"}, {"frame": "F01", "type": "MouseMove", "details": "-1,0"}, {"frame": "F03", "type": "MouseMove", "details": "-1,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.53 s, the first (cold) 1.49 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 12.5 s on the server the smoke request warmed up, 66.9 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 4passed2h 52m into the runtook 2m 53s
startup 65.2 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 65.2 s
- ✓POST /actions · {"actions": 16, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "2,4"}, {"frame": "F01", "type": "MouseMove", "details": "1,8"}, {"frame": "F03", "type": "MouseMove", "details": "-1,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 2.18 s, the first (cold) 2.10 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 17.7 s on the server the smoke request warmed up, 94.9 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 5passed3h 0m into the runtook 2m 15s
startup 52.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 52.7 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "2,4"}, {"frame": "F01", "type": "MouseMove", "details": "1,8"}, {"frame": "F03", "type": "MouseMove", "details": "-1,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.26 s, the first (cold) 1.23 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 10.0 s on the server the smoke request warmed up, 53.6 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in: /workspace/output (92 files, 4.03 GB) was graded as it stood at submit time. It is the model I verified with the last of the four selfchecks, which passed with the same contents.
What was submitted
- Model: ImageNet ResNet-18 (
timm/resnet18.a1_in1k, revision491b427b) over frame pairs at 768×480, followed by a bidirectional temporal transformer. Heads cover MouseMove, MouseScroll, MouseClick and a 128-key vocabulary. - Serving:
serve.shinstalls the vendored torch 2.11.0+cu128 and PyAV wheels offline into/tmpand startsserver.py. Startup took 53–65 s, each request about 1–2 s, and no failed clips. - Thresholds: move 0.4, scroll 0.9, click 0.4, key 0.4.
- Data: public training splits of D2E-480p (55 recordings from 29 games, CC-BY-NC-4.0), OSTask-demo (CC-BY-4.0), the typing dataset's train split (AFL-3.0), and 4,000 GUI-360 train-split screenshots (MIT) used as backgrounds. Synthetic cursor, click and scroll overlays were added with exact labels.
- Provenance:
architecture.json,data_manifest.json,training/(scripts and download commands),logs/(run logs,RUNS.md).
Results
- Public validation clip: 0.67 with the final thresholds (9 of 14 events matched, 4 false positives). That's one clip, so it's a sanity check, not a score estimate.
- Re-encodes of that clip (1280×720, 2560×1440, 25 fps): 0.64, 0.67, 0.56.
- Holdout (games, typing, one OSTask recording): micro F1 0.72. This is not the eval domain.
Choices and caveats
- Cursor size was the big fix. My first synthetic cursor was about 3× the real one at training resolution, and the model went nearly silent on the desktop clip (score 0.25). Matching the size gave 0.59 at 512 px, and 768 px input improved robustness further.
- Scroll threshold 0.9 is partly a judgement call. Holdout F1 is unchanged between 0.8 and 0.9, but the desktop variants favour 0.9. The validation clip informed that choice, which the rules allow, and the manifest says so.
- Weak areas on desktop: clicks and keyboard shortcuts (Cmd+A, deletions, Cmd+C/V) are mostly missed. Scroll direction could not be checked on desktop: the OSTask recordings have no scroll events, and the clip's 2-pixel scroll is invisible. The sign follows the wheel convention, which matches the validator's scoring.
- Key names are partly guessed: punctuation names and the Ctrl→Cmd folding are assumptions about the held-out naming.
- Licence: the weights were trained on D2E-480p, which is CC-BY-NC-4.0. Non-commercial terms are likely to apply to the derived model, so please check before any commercial use.
- Budget: $13.96 of $200 spent.