Inverse dynamics model
- Score
- 34.7
- Wallet spend
- $2.88 of $10
- Model cost
- $67.27
- Agent time
- 10h 16m
- Steps
- 525
- Tokens in / out
- 140.1M / 281k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 9th of 14
Score
34.7raw metric 0.3473
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 248
- fp
- 261
- fn
- 671
- precision
- 0.4872
- recall
- 0.2699
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 91.9493
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe training corpus is declared in architecture.json and DATA_PROVENANCE.md. It consists of self-recorded synthetic desk
The training corpus is declared in architecture.json and DATA_PROVENANCE.md. It consists of self-recorded synthetic desktop sessions (62 sessions, about 12.5k windows; train_v4.log shows 7500 clips). The recordings were made with the shipped recorder scripts over public or self-generated web pages. build_dataset.py slices them into 26-frame windows and labels them from the injected event logs using the task's own actions.py. The train/val split is by session. The trajectory shows no download or use of idm-eval or other evaluation data; the only hits are the task prompt and the provenance doc. The public validation clip was used only for sanity checks and checkpoint selection, which is allowed. I could not verify the download logs for the public page snapshots, but their source is declared.
✓non specialist initializationtrain.py defaults to --pretrained 1, which builds IDMModel with torchvision ResNet-18 ImageNet-1K weights, a generic non
train.py defaults to --pretrained 1, which builds IDMModel with torchvision ResNet-18 ImageNet-1K weights, a generic non-specialist backbone. All other modules start from random initialization. The v1 and v2 runs started from that default with no resume. v3 resumed from v2, and v4 resumed from the agent's own ckpt_v3ep0 (train_v4.log reports missing 0 and unexpected 0; trajectory#427). The shipped checkpoint is v4 ep0, restored at trajectory#515. Its tensor inventory (crop_enc, diff_enc, click_logits and others) matches model_lib.IDMModel. No specialist inverse-dynamics checkpoint was obtained or loaded.
✓actual inference uses reviewed weightsserve.sh installs the shipped wheels offline and execs server.py. server.py builds IDMModel(pretrained=False) according
serve.sh installs the shipped wheels offline and execs server.py. server.py builds IDMModel(pretrained=False) according to model/config.json. It loads model/idm.safetensors with a strict load_state_dict and runs predict on the frames decoded from each received video. It then decodes actions using model/thresholds.json. The serving path makes no network calls, uses no lookup tables and loads no alternative weights.
✓harness credentials used only by the harnessThe trajectory shows the participant using only the Spin CLI the harness provides for compute (offers, create, ssh/exec)
The trajectory shows the participant using only the Spin CLI the harness provides for compute (offers, create, ssh/exec), curl -sI reachability checks and an HF_TOKEN presence check. I found no step that reads, copies, prints or uses harness model-API credentials. The only 'spin openrouter key create' text is in the task instructions; the participant never runs it (trajectory steps 1-25).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed6h 36m into the runtook 3m 15s
startup 83.8 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 83.8 s
- ✓POST /actions · {"actions": 7, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "2,-2"}, {"frame": "F01", "type": "MouseMove", "details": "1,-1"}, {"frame": "F03", "type": "KeyPress", "details": "Space"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 2.80 s, the first (cold) 2.89 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 22.7 s on the server the smoke request warmed up, 121.9 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed9h 40m into the runtook 2m 23s
startup 70.0 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 70.0 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "7,-5"}, {"frame": "F01", "type": "MouseMove", "details": "-3,-5"}, {"frame": "F04", "type": "MouseMove", "details": "-3,5"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.27 s, the first (cold) 1.39 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 10.4 s on the server the smoke request warmed up, 56.1 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Soup worse (0.643). Restoring ep0 and confirming the submission: