Inverse dynamics model
- Score
- 73.3
- Wallet spend
- $7.86 of $10
- Model cost
- $54.78
- Agent time
- 8h 5m
- Steps
- 1,051
- Tokens in / out
- 106.8M / 953k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 1st of 14
Score
73.3raw metric 0.7334
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 678
- fp
- 252
- fn
- 241
- precision
- 0.7290
- recall
- 0.7378
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 122.3967
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training and calibration source is declared. The sources are self-generated synthetic Playwright/Xvfb screen recor
Every training and calibration source is declared. The sources are self-generated synthetic Playwright/Xvfb screen recordings of public web pages (the generator code ships in training/gen), PIL text strips rendered with Debian fonts, norvig.com/big.txt, the apple_cursor theme, and the one permitted /task/public example. The provenance log keeps the curl download commands and sha256 hashes for big.txt and the cursor theme, plus the apt font list. Threshold calibration used the participant's own held-out synthetic clips (seed % 20 == 0), not any public validation or test split. In the trajectory, the only contact with p-doom was a metadata listing (trajectory#20) and README/dataset-card fetches for other p-doom datasets (trajectory#21). No idm-eval clips or labels, and no private frontier-ml-tasks data, were downloaded or used.
✓non specialist initializationarchitecture.json declares base_model none. Both networks, PairNet and StripNet, are custom architectures defined in idm
architecture.json declares base_model none. Both networks, PairNet and StripNet, are custom architectures defined in idm/model.py and trained from scratch. The --init flags in train_a.py and train_b.py point only at the participant's own earlier runs, for example /root/runs/a3/net.pt and /root/runs/b3/net.pt. No from_pretrained call or p-doom/idm checkpoint appears anywhere. The tensor names in the weight inventory (stemA, s4.blocks, ...) match the custom PairNet.
✓actual inference uses reviewed weightsThe serving path is serve.sh → server.py → service.py → idm/infer.py Pipeline. Pipeline loads the shipped idm/weights/{a
The serving path is serve.sh → server.py → service.py → idm/infer.py Pipeline. Pipeline loads the shipped idm/weights/{a1,a2,a3ft,a4ft,a4ft2,a5ft,fb1-fb4}.pt files listed in idm/config.json with torch.load(weights_only=True). It runs those networks, plus pixel-based scroll-direction matching, on the decoded frames of the received video. idm/decode.py only applies thresholds to the resulting probabilities. clip_id is used only in log lines. There are no network calls, remote inference or label lookups, and HF_HUB_OFFLINE=1 is set.
✓harness credentials used only by the harnessSearches of the trajectory found no use, copying or exporting of harness API keys or base URLs in participant code. They
Searches of the trajectory found no use, copying or exporting of harness API keys or base URLs in participant code. They also found no `claude -p` or codex launches and no extra harness CLI invocations. The Agent subagents at trajectory#70 and trajectory#778 are the harness's own built-in tool, not extra CLIs.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed36m 8s into the runtook 2m 41s
startup 66.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 66.3 s
- ✓POST /actions · {"actions": 5, "dropped_actions": 0, "warning": null, "first": [{"frame": "F03", "type": "MouseMove", "details": "0,0"}, {"frame": "F07", "type": "MouseMove", "details": "0,0"}, {"frame": "F08", "type": "MouseMove", "details": "0,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.84 s, the first (cold) 2.72 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 15.1 s on the server the smoke request warmed up, 81.0 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 1.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed3h 16m into the runtook 2m 15s
startup 50.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 50.3 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": "action 7: MouseScroll requires an integer normalized amount", "first": [{"frame": "F00", "type": "MouseMove", "details": "0,0"}, {"frame": "F01", "type": "MouseMove", "details": "0,0"}, {"frame": "F02", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.25 s, the first (cold) 1.16 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 10.0 s on the server the smoke request warmed up, 53.8 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 3passed8h 2m into the runtook 2m 22s
startup 55.4 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 55.4 s
- ✓POST /actions · {"actions": 12, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "0,0"}, {"frame": "F01", "type": "MouseMove", "details": "0,0"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.75 s, the first (cold) 1.74 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 14.0 s on the server the smoke request warmed up, 75.1 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
The submission is confirmed and my session is over; /workspace/output will be graded as it stood at submission. I won't see the held-out score.
- Selfcheck: it passed all 3 gates, with startup at 50–66 s of the 300 s allowed and about 1 s per clip. The architecture review did not run in the selfcheck, so I have no read on whether it will accept the package.
- Expected accuracy: on 460 held-out synthetic clips the shipped ensemble scores micro F1 0.912. On the one real example clip it scores about 0.74, because that clip's Cmd+A, Backspace and one click are missed. I expect the real-domain score to be below the synthetic one.
- Scroll direction: the model's scroll heads only detect that a scroll happened. The direction comes from a pixel rule. It raised held-out MouseScroll F1 from 0.775 to 0.915 and read the example clip's scroll correctly.
- Late changes: I added the wheel-provenance logs to the inference evidence list in
architecture.jsonafter the selfcheck copy was taken. Only that metadata file changed. - Spin: about $7.85 of the $10 budget was spent. The one remaining machine was released at submit.