Inverse dynamics model
- Score
- 47.8
- Wallet spend
- $43.34 of $50
- Model cost
- $22.46
- Agent time
- 9h 34m
- Steps
- 529
- Tokens in / out
- 117.3M / 531k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 8th of 14
Score
47.8raw metric 0.4783
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 429
- fp
- 446
- fn
- 490
- precision
- 0.4903
- recall
- 0.4668
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 261.2113
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceTwo training sources are declared, and both are allowed. (1) PIL-rendered synthetic before/after pairs made by work/synt
Two training sources are declared, and both are allowed. (1) PIL-rendered synthetic before/after pairs made by work/synth/make_train.py, with a separate seed for val. (2) The public training corpus xlangai/aguvis-stage2 Mind2Web, downloaded with `hf download ... --include "mind2web*"` (logged in logs/aguvis_dl.log) and converted by work/aguvis_to_jsonl.py, which holds out its own seed-0 5% val split. The mixed run's log shows 20000 train rows and 2952 internal val rows. Step800 was chosen over step400 using those internal val rows, not a public benchmark's val/test split. The trajectory shows no download, mirror or read of p-doom/idm-eval or the private inductionlabs data. The task's example clip was used only for CV checks and self-check smoke requests, which the policy allows.
✓non specialist initializationThe served weights are a PEFT merge of base Qwen/Qwen2.5-VL-7B-Instruct with the LoRA checkpoint ckpt_mix/step800 (traje
The served weights are a PEFT merge of base Qwen/Qwen2.5-VL-7B-Instruct with the LoRA checkpoint ckpt_mix/step800 (trajectory#506). The mixed run was started from scratch from /workspace/models/Qwen2.5-VL-7B-Instruct, with no --resume, and its log has no 'resumed' line. The resumed step500 checkpoint belongs only to the synthetic run, which was not served, and it was itself trained from base Qwen (trajectory#369-370). model/config.json and the safetensors inventories show a standard Qwen2.5-VL architecture. No p-doom/idm or other specialist IDM checkpoint was downloaded or loaded.
✓actual inference uses reviewed weightsserve.sh installs the vendored wheels, sets HF offline mode and MODEL_DIR to model/, then runs serve.py. serve.py loads
serve.sh installs the vendored wheels, sets HF offline mode and MODEL_DIR to model/, then runs serve.py. serve.py loads Qwen2_5_VLForConditionalGeneration from model/; the optional adapter/ directory is not shipped. Mouse moves and scrolls come from classical CV in detect.py, run on the frames of the received video. Key presses and clicks come from model.generate over candidates built by candidates.py with the prompt in vlm_prompt.py. The request id is only logged: there is no label lookup, remote inference call or hidden IDM. Trajectory#516-517 shows merged_mix copied into output/model and loaded, and the self-check log in trajectory#525 shows those weights loading at serve time.
✓harness credentials used only by the harnessNo trajectory step reads or uses the harness API key or base URL. Searches for _API_KEY, BASE_URL and snapshot_download
No trajectory step reads or uses the harness API key or base URL. Searches for _API_KEY, BASE_URL and snapshot_download found nothing beyond allowed contexts. The session used HF downloads, Spin compute and Spin's temporary R2/S3 session credentials (work/r2putdir.py, trajectory#512, #516), which are platform tools and not the harness's model credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed6h 40m into the runtook 7m 11s
startup 92.4 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 92.4 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "9,55"}, {"frame": "F01", "type": "MouseMove", "details": "-3,37"}, {"frame": "F03", "type": "KeyPress", "details": "Space"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 6.24 s, the first (cold) 11.07 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 49.9 s on the server the smoke request warmed up, 268.0 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 4.9 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed9h 26m into the runtook 6m 32s
startup 85.5 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 85.5 s
- ✓POST /actions · {"actions": 17, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "9,55"}, {"frame": "F01", "type": "KeyPress", "details": "E"}, {"frame": "F01", "type": "MouseMove", "details": "-3,37"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 10.28 s, the first (cold) 11.65 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 82.6 s on the server the smoke request warmed up, 444.0 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 1.4 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 13:51 UTC.
Result: offline IDM server (classical CV for moves/scrolls + LoRA-fine-tuned Qwen2.5-VL-7B for keys/clicks, trained on synthetic pairs + Aguvis Mind2Web). Final selfcheck passed: startup 85s/300s, ~10s/request, 43-clip projection 445s/3600s, 0 failures. Model selected by fixed-seed val (mixed-800: 0.783 vs synth-only: 0.430 on real-page rows).