Inverse dynamics model
- Score
- 43.2
- Wallet spend
- $16.58 of $50
- Model cost
- $11.18
- Agent time
- 4h 45m
- Steps
- 247
- Tokens in / out
- 98.5M / 455k
- Self-checks
- 3 passed of 3 used
- Rank at $50 wallet
- 9th of 14
Score
43.2raw metric 0.4324
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 382
- fp
- 466
- fn
- 537
- precision
- 0.4505
- recall
- 0.4157
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 179.6289
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe training data is synthetic and generated on the fly. synth.py produces it, and data.py and train.py consume it throu
The training data is synthetic and generated on the fly. synth.py produces it, and data.py and train.py consume it through D.SynthClips. Validation and selection use synthetic seeds of 900M and up, plus the permitted single example clip. DATA_MANIFEST.json declares this and lists the excluded sources. In the trajectory, HF activity was limited to metadata searches and README fetches (steps 24 and 26). p-doom/idm-eval showed up only as a search-result entry and was never downloaded or used. No forbidden corpus or other benchmark's validation or test split was found.
✓non specialist initializationOnly the appearance branch starts from a pretrained model: torchvision ResNet-18 with IMAGENET1K_V1 weights (resnet18-f3
Only the appearance branch starts from a pretrained model: torchvision ResNet-18 with IMAGENET1K_V1 weights (resnet18-f37072fd.pth). It was downloaded from download.pytorch.org, as recorded in logs/resnet18_download.txt and architecture.json. In model.py and train.py, the motion CNN, transformer and heads are built from scratch. The later runs (run2 to run7) resumed or initialized from the participant's own earlier checkpoints (training_scripts/launch_resume7.sh, logs/run5_stdout.log). The tensor names in the weights/model.pt inventory match IDMModel. No specialist IDM checkpoint such as p-doom/idm was found.
✓actual inference uses reviewed weightsserve.sh installs the vendored wheels offline and runs server.py. server.py builds IDMModel(pretrained_resnet=None), loa
serve.sh installs the vendored wheels offline and runs server.py. server.py builds IDMModel(pretrained_resnet=None), loads weights/model.pt with torch.load(weights_only=True) and load_state_dict, and reads weights/thresholds.json. Each request goes through decode_gray_frames (video.py), the network, and decode_clip (decode.py). The model sees only the submitted video bytes. The id field is ignored, and there are no remote inference calls, label lookup tables or hidden backbones. The weights/model.pt inventory matches the model.py architecture.
✓harness credentials used only by the harnessThe trajectory shows no use of ANTHROPIC, OPENAI or OPENROUTER keys or *_BASE_URL variables in participant code. It also
The trajectory shows no use of ANTHROPIC, OPENAI or OPENROUTER keys or *_BASE_URL variables in participant code. It also shows no launches of extra harness CLIs such as claude or codex. The participant's HF_TOKEN was used only for read-only Hugging Face metadata requests (steps 24 to 26), which the policy allows. No shipped code references harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed2h 8m into the runtook 2m 44s
startup 60.5 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 60.5 s
- ✓POST /actions · {"actions": 17, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-28,33"}, {"frame": "F01", "type": "MouseMove", "details": "-1,-2"}, {"frame": "F03", "type": "MouseMove", "details": "5,-25"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 3.30 s, the first (cold) 2.61 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 27.8 s on the server the smoke request warmed up, 149.5 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed3h 17m into the runtook 5m 4s
startup 71.0 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 71.0 s
- ✓POST /actions · {"actions": 16, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-8,81"}, {"frame": "F01", "type": "MouseMove", "details": "-8,53"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 11.09 s, the first (cold) 27.01 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 88.7 s on the server the smoke request warmed up, 476.5 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 16.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 3passed4h 40m into the runtook 3m 27s
startup 69.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 69.7 s
- ✓POST /actions · {"actions": 16, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-8,81"}, {"frame": "F01", "type": "MouseMove", "details": "-8,53"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 5.06 s, the first (cold) 5.21 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 40.3 s on the server the smoke request warmed up, 216.5 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.6 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in: /workspace/output was graded as it stood at 10:16 UTC. Its last frontier selfcheck passed on the same folder, but the model's real-world accuracy is still unproven. The one real clip I could check scores about 0.73 F1.
What's submitted
- Model: a 17.5M-parameter network. A small CNN reads 30 fps frame pairs for motion, and a ResNet-18 reads per-bin frames for appearance. A transformer over the 25 bins drives heads for MouseMove, MouseClick, MouseScroll, and KeyPress (base key plus held modifiers). The only pretrained weights are the ImageNet ResNet-18 used for initialization.
- Serving:
serve.shinstalls the vendored cp312 wheels offline into/tmp(torch 2.8.0+cu128, torchvision, PyAV), then starts the server.GET /healthreturns 200 when ready.POST /actionsreturns JSON, and any decode failure returns[]with HTTP 200. - Selfcheck: the final folder passed on an H100 with 69.7 s startup, 43 clips projected at 217 s against a 3,600 s budget, and zero failed clips.
Measured scores
- Example clip, scored with the task's
validate.pythrough the served endpoint: F1 0.733 (11 TP, 5 FP, 3 FN). MouseMove is 0.76, MouseClick 0.80, MouseScroll 1.00, and KeyPress 0 on this clip. - 200 held-out synthetic clips at the shipped thresholds: micro F1 about 0.87. Per type at the search optimum: MouseMove 0.96, MouseClick 0.85, MouseScroll 0.79, KeyPress 0.44.
- Real-world precision for MouseMove is the main unknown. The example clip shows several confident false positives in idle stretches, and the synthetic data can't measure that gap.
Training data and provenance
- All training data is generated by
synth.py. It renders browser-like desktops with real input events and labels them usingactions.py, so label semantics match the scorer. - No public corpus was used for training, calibration, or selection.
- The example clip was used only for sanity checks and as a tie-break between near-equal thresholds. The task allowed this.
- Thresholds were chosen by coordinate search on 200 held-out synthetic clips.
architecture.jsonandDATA_MANIFEST.jsonlist the sources, download commands, and logs for every run.
Choices I made that you may want to review
- Excluded sources:
- The p-doom/idm-eval dataset was never downloaded or opened. Its name and tags appeared in a search result.
- I did not use p-doom/idm or its labels. I also left out AGI-CAST-idm-actions, whose labels came from p-doom/idm.
- I did not use the AGI-CAST raw screencasts, because the example clip comes from a p(doom) member and the test recordings may overlap.
- I read the README cards of several other public datasets but used none of them.
- Final model: run 7 (21,548 steps). I also tested a resolution-augmented run 8. It tied on synthetic data but scored lower on the example (0.649), so I kept run 7.
- Spend: about $16.6 of the $50 wallet, with about $33.4 left.