Inverse dynamics model
- Score
- 36.0
- Wallet spend
- $2.70 of $10
- Model cost
- $8.90
- Agent time
- 3h 40m
- Steps
- 217
- Tokens in / out
- 21.7M / 267k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 7th of 14
Score
36.0raw metric 0.3596
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 240
- fp
- 176
- fn
- 679
- precision
- 0.5769
- recall
- 0.2612
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 2360.4981
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe model was trained only on synthetic screen recordings. evidence/data.py builds them with simcore.Clip(seed), applies
The model was trained only on synthetic screen recordings. evidence/data.py builds them with simcore.Clip(seed), applies JPEG simulation, and labels them with gt.parse_events. It reads no outside data, and evaluation also uses synthetic seeds. DATA.md declares this and says the single provided validation example was used only to tune the classical detectors and thresholds, which the policy allows. In the trajectory, "idm-eval" appears only in restatements of the task prompt and in DATA.md text. Searches for load_dataset, hf_hub, frontier-ml-tasks, wget, git clone and hf download found nothing. The only Hugging Face access was a connectivity check, `curl -sI https://huggingface.co`. The vendored wheels came from PyPI and download.pytorch.org (evidence/wheel_download.log).
✓non specialist initializationarchitecture.json declares base_model "none". evidence/train.py creates M.Model() with random weights and loads no pretr
architecture.json declares base_model "none". evidence/train.py creates M.Model() with random weights and loads no pretrained checkpoint. Every key in the weights.pt inventory (model.bb.*, model.fuse.*, model.h_*) matches the custom ConvNeXt-style two-stream network in model.py, with no foreign backbone keys. trajectory#199 shows training ending at step 3200 and producing ckpt_best.pt on the Spin VM at 48,270,347 bytes. trajectory#203 shows that file being copied by scp to weights.pt at the same size as the inventory. No specialist IDM checkpoint was used.
✓actual inference uses reviewed weightsserve.sh installs the vendored wheels offline and runs `server.py --weights weights.pt`. server.py loads model.Model() w
serve.sh installs the vendored wheels offline and runs `server.py --weights weights.pt`. server.py loads model.Model() with the state_dict from weights.pt and handles POST /actions by decoding the uploaded MP4 and calling pipeline.predict. pipeline.py runs the reviewed model's click and key heads together with the hand-written OpenCV/numpy detectors in classical.py for mouse moves and scrolls, which the policy allows. The clip id is not used, and there are no remote calls, label tables or hash lookups.
✓harness credentials used only by the harnessSearches of the trajectory for API_KEY, BASE_URL, anthropic, printenv, claude -p and openrouter found no use of harness
Searches of the trajectory for API_KEY, BASE_URL, anthropic, printenv, claude -p and openrouter found no use of harness credentials or base URLs and no extra harness CLI launches. The only hits were Spin CLI docs and help text. Remote compute went through the Spin CLI (trajectory#143, trajectory#203). The SSH key printed in trajectory#202 is the Spin VM rental credential, not a harness credential.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1failed3h 33m into the runtook 45sfree (stopped before any GPU time)
failed review prerequisites: Initialization metadata must declare a nonempty base_model and base_revision (or none for both when training from scratch)
1 gates
- ✗review prerequisites · Initialization metadata must declare a nonempty base_model and base_revision (or none for both when training from scratch)
- 2passed3h 34m into the runtook 5m 50s
startup 70.9 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 70.9 s
- ✓POST /actions · {"actions": 10, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-6,50"}, {"frame": "F01", "type": "MouseMove", "details": "-13,49"}, {"frame": "F04", "type": "MouseMove", "details": "-7,8"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 23.31 s, the first (cold) 23.55 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 183.7 s on the server the smoke request warmed up, 987.4 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.3 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
SELF-CHECK PASSED — every gate: startup 70.9s/300s (full offline wheel install + H100 load), slowest request 23.6s/300s, whole-eval projection 988s/3600s, 0 failed clips, correct response shape. Time to submit (~08:35 UTC, inside the deadline window):