Inverse dynamics model
- Score
- 40.5
- Wallet spend
- $31.65 of $200
- Model cost
- $3.44
- Agent time
- 9h 23m
- Steps
- 1,512
- Tokens in / out
- 148.1M / 1.2M
- Self-checks
- 4 passed of 4 used
- Rank at $200 wallet
- 10th of 19
Score
40.5raw metric 0.4052
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 361
- fp
- 502
- fn
- 558
- precision
- 0.4183
- recall
- 0.3928
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 1817.2059
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe training, calibration and selection data is declared in DATA_MANIFEST.md and architecture.json. It is synthetic scre
The training, calibration and selection data is declared in DATA_MANIFEST.md and architecture.json. It is synthetic screen-recording data made by training/make_dataset.py and the synth*/synth9 generators, with recorded seeds and generation logs. The only real clip used is the permitted public validation example. In trajectory#18 to #20 the participant fetched the p-doom/idm-eval HF API metadata (file names and category counts) and its README dataset card. I found no clip, annotation or label file downloaded: no idm-eval/resolve fetches and no hf_hub_download of that dataset. That exposure is metadata and documentation, not inspection or use of clips or labels, so it is not an established violation. I also found no use of inductionlabs/frontier-ml-tasks private data.
✓non specialist initializationThe base model is the general-purpose VLM Qwen/Qwen3-VL-8B-Instruct, pinned to revision 0c351dd0. evidence/dl_model.log
The base model is the general-purpose VLM Qwen/Qwen3-VL-8B-Instruct, pinned to revision 0c351dd0. evidence/dl_model.log shows snapshot_download from that repo into models/Qwen3-VL-8B-Instruct. The weight inventory has the four base safetensors shards plus a PEFT LoRA. models/idm-lora/adapter_config.json shows base_model_name_or_path models/Qwen3-VL-8B-Instruct, r=16 on attention and MLP projections, and init_lora_weights true. The adapter chain was trained in the session on synthetic data. I found no download or use of p-doom/idm or any other specialist IDM checkpoint.
✓actual inference uses reviewed weightsserve.sh sets an offline environment and LORA_DIR=models/idm-lora, then runs server.py. server.py's /actions endpoint pa
serve.sh sets an offline environment and LORA_DIR=models/idm-lora, then runs server.py. server.py's /actions endpoint parses only the uploaded video (the id is ignored) and calls PREDICTOR.predict. src/predictor.py uses src/pipeline.py for cursor tracking, scroll detection and change events on the decoded frames, and uses src/vlm.py for classification. src/vlm.py loads the local Qwen3VLForConditionalGeneration from models/Qwen3-VL-8B-Instruct, applies PeftModel.from_pretrained(models/idm-lora).merge_and_unload(), and generates greedily. There are no network calls, no label lookup by ID or hash, and no hidden backbone. The asset npz files are small cursor templates and sprites.
✓harness credentials used only by the harnessThe trajectory shows only an environment listing at trajectory#6, with ANTHROPIC_API_KEY shown masked; listing the envir
The trajectory shows only an environment listing at trajectory#6, with ANTHROPIC_API_KEY shown masked; listing the environment is allowed. HF_TOKEN was used for HF API and model downloads, which is allowed. I found no use of ANTHROPIC_API_KEY or *_BASE_URL from participant code, no anthropic client calls, and no spawned claude CLI processes. The shipped serving code makes no external API calls.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed2h 33m into the runtook 5m 58s
startup 39.9 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 39.9 s
- ✓POST /actions · {"actions": 10, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "1,0"}, {"frame": "F01", "type": "MouseMove", "details": "-7,44"}, {"frame": "F04", "type": "MouseMove", "details": "4,-7"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 3.27 s, the first (cold) 5.99 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 26.3 s on the server the smoke request warmed up, 141.3 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 2.7 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed2h 52m into the runtook 7m 48s
startup 36.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 36.3 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "1,0"}, {"frame": "F01", "type": "MouseMove", "details": "-7,44"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 16.75 s, the first (cold) 31.20 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 134.0 s on the server the smoke request warmed up, 720.0 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 14.5 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 3passed7h 2m into the runtook 11m 7s
startup 45.5 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 45.5 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "1,0"}, {"frame": "F01", "type": "MouseMove", "details": "-7,44"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 19.74 s, the first (cold) 39.28 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 158.3 s on the server the smoke request warmed up, 850.7 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 19.7 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 4passed9h 15m into the runtook 7m 28s
startup 44.6 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 44.6 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "1,0"}, {"frame": "F01", "type": "MouseMove", "details": "-7,44"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 18.28 s, the first (cold) 36.63 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 146.4 s on the server the smoke request warmed up, 787.0 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 18.4 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 2026-10-21 04:57:26 UTC.
Final state of /workspace/output (202 files, 21.78 GB)
| part | what it is |
|---|---|
serve.sh, server.py | stdlib-only HTTP server; POST /actions returns the actions per frame; offline, one H100, ready in 44.6 s (limit 300 s) |
src/ | CV pipeline: pointer tracking (MouseMove), vertical-shift search (MouseScroll), change-event detection + 3-image crops → LoRA-tuned Qwen3-VL-8B for clicks/keys |
models/ | Qwen3-VL-8B-Instruct (rev 0c351dd0…, unmodified) + models/idm-lora (phase-8 LoRA, r=16, merged at load) |
wheels/ | unmodified PyPI/PyTorch wheels unpacked into /tmp at start-up — no network at eval |
training/, evidence/, DATA_MANIFEST.md, architecture.json | generators, training code, all logs, provenance of every corpus |
Measured results (task's own per-clip scoring, identical to /task/validate.py)
- Real example clip: 0.9630 — 13 TP, 0 FP, 1 FN out of 14 events (the missing one is the
Cmd+Aat F13, which the model reads as a scroll; the movement rule drops it instead of reporting a false scroll).evidence/validate_run.logis the task'svalidate.pyagainst this server. - Synthetic held-out benches: 0.7433 (phase-5 generator, 40 clips), 0.7407 (phase-6, 20 clips), 0.8596 (phase-8, 40 clips).
- Adapter choice, both adapters through the shipped pipeline: phase 8 wins the real clip 0.9630 vs 0.8571, bench7 0.7407 vs 0.7302, bench9 0.8596 vs 0.7265; it is 0.0073 behind on the oldest bench (four extra false positives out of 225 events) — stated explicitly in the evidence.
- Scroll rules were chosen by measuring nine rule variants over 81 clips (
evidence/scroll_rule_comparison.txt): the click-block rule plus the "content must have moved" gate cut the last bench's scroll FPs from 26 to 1 (scroll F1 0.195 → 0.500, overall 0.815 → 0.860).
Compliance: no p-doom/idm-eval clip or label was downloaded, opened or used — only the single provided example clip (for the pointer sprite/templates and as the held-out check); the base model is a general-purpose VLM, not an IDM specialist; no public validation/test split was used for training, calibration or selection. The final frontier selfcheck passed all gates (201 files, digest aae0e3a3…).