Trial · idm-training-5uwr13

Inverse dynamics model

Claude Sonnet 5.5xhighHarness Claude Code 2.1.282

Wallet

Score: 69.0
Wallet spend: $10.07 of $50
Model cost: $120.89
Agent time: 9h 7m
Steps: 1,783
Tokens in / out: 171.5M / 2.6M
Self-checks: 4 passed of 4 used
Rank at $50 wallet: 1st of 14

Loading trajectory…

pass4 of 4 checks passed · reviewed by claude-opus-5-5 · read 1783 trajectory steps

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

✓training data provenanceTraining data is clips the participant made themselves with an Xvfb, Chromium, XTest and ffmpeg simulator (train/sim/*,

Training data is clips the participant made themselves with an Xvfb, Chromium, XTest and ffmpeg simulator (train/sim/*, /data/clips_a). The text corpus is the WikiText-103 train split, and the download command is kept in train/sim/setup_corpus.sh and train/sim/apps/SOURCES.md. Model selection used holdouts carved from the simulated clips (train/train_vlm.py takes a holdout from its own rows). DATA_MANIFEST.md says the permitted example clip was only inspected and used for calibration, never trained on. In the trajectory, the participant listed p-doom repos (#14) and read some dataset and model cards (#15), but downloaded nothing from p-doom/idm-eval or any copy of it, and nothing from frontier-ml-tasks. In #20 they downloaded one public train-split video (keystroke-typing-videos train/0.mp4) to look at. That is train data, it isn't in the shipped corpus, and no public val/test split was used.

DATA_MANIFEST.mdtrain/sim/setup_corpus.shtrain/sim/apps/SOURCES.mdtrain/train_vlm.pytrajectory#14trajectory#15trajectory#20

✓non specialist initializationarchitecture.json and DATA_MANIFEST.md declare the VLM as a LoRA fine-tune of the general model Qwen/Qwen3-VL-2B-Instruc

architecture.json and DATA_MANIFEST.md declare the VLM as a LoRA fine-tune of the general model Qwen/Qwen3-VL-2B-Instruct (revision pinned). train/merge_lora.py loads that base with Qwen3VLForConditionalGeneration.from_pretrained and merges the PEFT adapter. The weights/vlm/model.safetensors inventory matches the Qwen3-VL-2B layout: hidden size 2048, vocab 151936, MLP size 6144. The CursorNet tracker (weights/tracker.pt) is a small CNN of about 0.85M parameters trained from random initialization by train/train_tracker.py. Neither p-doom/idm nor any other specialist IDM was downloaded or used as an initialization; its model card was only viewed.

architecture.jsonDATA_MANIFEST.mdtrain/merge_lora.pytrain/train_tracker.pyweights/vlm/model.safetensorsweights/tracker.pttrajectory#15

✓actual inference uses reviewed weightsserve.sh installs wheels offline with HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE set, then runs uvicorn on server/app.py. I

serve.sh installs wheels offline with HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE set, then runs uvicorn on server/app.py. In server/app.py, POST /actions reads the uploaded video and calls pipeline.predict(video_bytes); clip_id is used only for logging. server/pipeline.py does the following: - loads weights/tracker.pt into CursorNet and loads EventModel from the local weights/vlm; - decodes the video and tracks the cursor; - produces mouse moves and scrolls with hand-written rules in server/rules.py and server/frontend.py; - produces key presses and clicks with greedy generation from the local Qwen3-VL in server/event_model.py. There are no remote calls, no label or hash lookups and no hidden IDM. The wheels in wheels/ all have approved digest audits.

serve.shserver/app.pyserver/pipeline.pyserver/event_model.pyserver/rules.pyweights/vlm/model.safetensorsweights/tracker.pt

✓harness credentials used only by the harnessI searched the trajectory and found no access to harness API keys or the harness base URL from the participant's own cod

I searched the trajectory and found no access to harness API keys or the harness base URL from the participant's own code, no copying of credentials, and no extra harness CLI launched (no 'claude -p'). HF_TOKEN was used to list and download from Hugging Face (#14, #15, Qwen snapshot downloads), which the policy allows. The only other external tool in use was the permitted Spin CLI.

trajectory#14trajectory#15

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.

1
failed26m 43s into the runtook 51sfree (stopped before any GPU time)
failed review prerequisites: The review needs architecture.json and serve.sh at the top of the submission
1 gates
- ✗review prerequisites · The review needs architecture.json and serve.sh at the top of the submission
2
failed27m 48s into the runtook 42sfree (stopped before any GPU time)
failed review prerequisites: Initialization metadata must declare a nonempty base_model and base_revision (or none for both when training from scratch)
1 gates
- ✗review prerequisites · Initialization metadata must declare a nonempty base_model and base_revision (or none for both when training from scratch)
3
passed28m 40s into the runtook 1m 54s
startup 37.8 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 37.8 s
- ✓POST /actions · {"actions": 0, "dropped_actions": 0, "warning": null, "first": []}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 0.58 s, the first (cold) 0.63 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 4.6 s on the server the smoke request warmed up, 24.8 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
4
passed4h 27m into the runtook 4m 55s
startup 40.0 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 40.0 s
- ✓POST /actions · {"actions": 23, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-6,38"}, {"frame": "F01", "type": "MouseMove", "details": "-7,46"}, {"frame": "F03", "type": "MouseMove", "details": "1,-1"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 5.43 s, the first (cold) 7.17 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 43.5 s on the server the smoke request warmed up, 234.1 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 1.7 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
5
passed7h 22m into the runtook 3m 1s
startup 35.2 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 35.2 s
- ✓POST /actions · {"actions": 12, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-2,23"}, {"frame": "F01", "type": "MouseMove", "details": "-10,61"}, {"frame": "F03", "type": "MouseMove", "details": "1,-1"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 2.75 s, the first (cold) 4.00 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 22.0 s on the server the smoke request warmed up, 118.5 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 1.3 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
6
passed9h 3m into the runtook 3m 35s
startup 43.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 43.7 s
- ✓POST /actions · {"actions": 12, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-2,23"}, {"frame": "F01", "type": "MouseMove", "details": "-10,61"}, {"frame": "F03", "type": "MouseMove", "details": "1,-1"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 3.66 s, the first (cold) 5.30 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 29.1 s on the server the smoke request warmed up, 156.5 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 1.7 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests

The agent's last message before its session ended.

The submission is in; I ran frontier submit --confirm at 07:21 UTC and the session is over. The final frontier selfcheck passed (startup 43.7 s, slowest request 5.3 s, 43 clips projected at 158 s, 0 failed clips). It did not run the architecture review, which could still score the submission zero.

Pipeline:
- A small CNN I trained from scratch tracks the cursor, and rules turn that into MouseMove and MouseScroll.
- Qwen3-VL-2B-Instruct (LoRA-merged) reads the screen changes and produces KeyPress and MouseClick.
Late model swap: I trained 250 more steps with colour/theme augmentation and adopted that model over the earlier one. It scored 0.828 vs 0.816 overall on a clean 200-clip simulated holdout (key F1 0.771 vs 0.740) and was equal on the example clip. The earlier run's holdout overlapped this run's training set, so that comparison was contaminated. DATA_MANIFEST.md documents the overlap and the clean comparison.
Real-clip weakness: on the one real example clip it gets the first click, the scroll and 8 of 9 mouse moves right, but misses Cmd+A and Backspace. Changing the colours of the clip's images did not help. A confidence threshold of 0.8 means it outputs no key presses on that clip rather than wrong ones, so real keyboard scores may be low.
Training data: everything was trained on my own simulator clips plus public text corpora. The one provided example clip was used only for sanity checks, never trained on.
Logs and manifests: all training and evaluation logs, DATA_MANIFEST.md and architecture.json are in /workspace/output. The GPU box I rented has been released.

Inverse dynamics model

Score

Verifier details