Inverse dynamics model
- Score
- 36.5
- Wallet spend
- $36.64 of $50
- Model cost
- $4.78
- Agent time
- 10h 41m
- Steps
- 2,171
- Tokens in / out
- 216.2M / 1.6M
- Self-checks
- 4 passed of 4 used
- Rank at $50 wallet
- 13th of 19
Score
36.5raw metric 0.3646
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 295
- fp
- 404
- fn
- 624
- precision
- 0.4220
- recall
- 0.3210
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 336.3199
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training, calibration and selection source is declared in training/data_manifest.md, architecture.json and trainin
Every training, calibration and selection source is declared in training/data_manifest.md, architecture.json and training/download.log. (1) Synthetic clips come from src/idm/gen.py. (2) The single allowed /task/public example clip supplied the pointer sprites (training/assets/real/sprites.npz) and was replayed through src/idm/replay.py. (3) 600 public OpenGVLab/GUI-Odyssey screenshots, used only as page backgrounds with no labels and no split, were fetched by training/download_gui_odyssey.py using HF_TOKEN; trajectory#30-39, #58-91 and #10-11 show only those downloads and metadata browsing. (4) Calibration used held-out synthetic clips. Searches of the trajectory and the files found no access to other p-doom/idm-eval clips or labels, no inductionlabs/frontier-ml-tasks data and no validation or test split of another benchmark. Every idm-eval mention is either the task prompt or the participant's own documentation.
✓non specialist initializationThe model is the participant's own small CNN (src/idm/models.py), trained from random initialisation with torch.manual_s
The model is the participant's own small CNN (src/idm/models.py), trained from random initialisation with torch.manual_seed(0) according to training/init.log. In src/idm/train.py, the --init warm start only reuses the participant's own earlier checkpoints. The first run in the trajectory (ckpt2, e.g. trajectory#560 / steps-538-560) was launched without --init, so it trained from scratch. Later runs warm-start from ckpt2/run2/r5, as training/train.log records. No p-doom/idm checkpoint or other model download appears anywhere. The weights/model.pt inventory holds only verifier[0..4] and char[0..4] conv/BN/fc tensors (e.g. char[4].fc.weight [96,1536]), which matches the declared architecture of about 330k parameters per network.
✓actual inference uses reviewed weightsserve.sh sets CKPT=weights/model.pt and runs `python -m idm.server`. In src/idm/server.py, POST /actions calls PREDICTOR
serve.sh sets CKPT=weights/model.pt and runs `python -m idm.server`. In src/idm/server.py, POST /actions calls PREDICTOR.predict(video), and clip_id is used only for logging. src/idm/infer.py torch.loads the 'verifier' and 'char' state lists from that checkpoint, rebuilds them with models.build, averages the ensemble softmax outputs and applies classical diff/scroll analysis plus threshold rules from the checkpoint cfg. There are no network calls, remote inference, hash or label lookup tables. All wheels are approved by the trusted binary audits.
✓harness credentials used only by the harnessIn the trajectory, ANTHROPIC, BASE_URL and the router address appear only in the Spin CLI documentation (SPIN_* variable
In the trajectory, ANTHROPIC, BASE_URL and the router address appear only in the Spin CLI documentation (SPIN_* variables, which are allowed) and in a `ps aux` listing at trajectory#1888 that shows the harness's own router process. Listing processes is allowed. No step reads, uses or copies harness API keys or base URLs, and no extra harness CLI (claude -p, codex, etc.) was launched. The participant used HF_TOKEN only for Hugging Face downloads, which is allowed.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed6h 3m into the runtook 1m 22s
startup 24.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 24.3 s
- ✓POST /actions · {"actions": 12, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-700,268"}, {"frame": "F01", "type": "MouseMove", "details": "-7,45"}, {"frame": "F04", "type": "MouseMove", "details": "0,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 3.81 s, the first (cold) 3.82 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 30.2 s on the server the smoke request warmed up, 162.2 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed8h 43m into the runtook 1m 36s
startup 25.4 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 25.4 s
- ✓POST /actions · {"actions": 12, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-700,268"}, {"frame": "F01", "type": "MouseMove", "details": "-7,45"}, {"frame": "F04", "type": "MouseMove", "details": "0,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 4.05 s, the first (cold) 3.87 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 32.6 s on the server the smoke request warmed up, 175.2 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 3passed9h 45m into the runtook 1m 40s
startup 24.8 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 24.8 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-700,268"}, {"frame": "F01", "type": "MouseMove", "details": "-7,45"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 4.93 s, the first (cold) 5.04 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 39.5 s on the server the smoke request warmed up, 212.1 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 4passed10h 39m into the runtook 1m 50s
startup 24.2 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 24.2 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-700,268"}, {"frame": "F01", "type": "MouseMove", "details": "-7,45"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 5.57 s, the first (cold) 5.73 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 44.3 s on the server the smoke request warmed up, 238.0 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.2 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.
Final model — a 5-member ensemble of from-scratch CNNs (no pretrained weights, no specialist IDM model), shipped in /workspace/output/:
| member | steps | note |
|---|---|---|
| r5 | 20 000 | fixed labelling, 85 % typed characters |
| r6 | 18 000 | fixed labelling, 60 % typed characters |
| r7 | 8 000 | warm start from r5, 90 % typed chars, seed 1200 |
| r8 | 8 000 | warm start from r5, 60 % typed chars, seed 1300 |
| base | 10 000 | base run of training/train.log |
Selection (on 90 held-out synthetic clips, seed 777001, plus the provided example clip counted 6×, scored with /task/actions.py rules and the target type mix):
| candidate | default | tuned |
|---|---|---|
| best single run (r7) | 0.8506 | 0.8687 |
| ens_r5r6run2 (previous) | 0.8548 | 0.8720 |
| ens_all (4 members) | 0.8552 | 0.8735 |
| ens_all5 (shipped) | 0.8568 | 0.8739 |
A ninth candidate (the same ensemble with r6's 20 000-step snapshot) scored 0.8736 and was not shipped — recorded in training/tuning.json and training/eval_synth.log.
Verification: example clip answered perfectly (micro F1 1.0000, 14/14, 0 FP/0 FN), validate.py diagnostic score 1.0 with 0 failed requests, and the fourth and final frontier selfcheck passed (startup 24.2 s of 300 s, 3/3 gates, 670 s projected of the 3600 s budget).
Provenance: training data is synthetic recordings generated locally (src/idm/gen.py) with GUI-Odyssey screenshots as page backgrounds and Google Fonts for glyph rendering; the one permitted example clip supplied the pointer sprites and was replayed as labelled training data. No other clip or label of p-doom/idm-eval was used, and nothing was trained, tuned or selected on a public validation/test split — thresholds were tuned on synthetic clips the task permits plus the provided sample. Both spin leases are stopped.