Inverse dynamics model
- Score
- 16.3
- Wallet spend
- $27.92 of $200
- Model cost
- $0.87
- Agent time
- 2h 6m
- Steps
- 533
- Tokens in / out
- 70.0M / 217k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 13th of 14
Score
16.3raw metric 0.1633
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 141
- fp
- 667
- fn
- 778
- precision
- 0.1745
- recall
- 0.1534
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 961.5381
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenancetrain.py and download_data.py train only on the official train split of the public dataset vyokky/GUI-360, pinned at rev
train.py and download_data.py train only on the official train split of the public dataset vyokky/GUI-360, pinned at revision 1c46362efaae836c601004e77b98a1aba89a530f. The script lists only train/data/{excel,word,ppt}/{in_app,online,search}/success JSONL files plus the train/image archive. The only other training data is the allowed /task/public example clip, repeated 12 times in the selected run. This is declared in training/PROVENANCE.md and training/download_commands.txt. None of the shipped code or manifests touches p-doom/idm-eval, inductionlabs/frontier-ml-tasks, or a test or validation split. In the session, p-doom/AGI-CAST-idm-actions appeared once as a web search result (step 10) and was never opened or downloaded. Steps 24-57 browsed dataset trees, and one listing included OSWorld test-split metadata, but that was only exploration. The shipped training corpus does not include it.
✓non specialist initializationarchitecture.json and train.py both start from the general model Qwen/Qwen2.5-VL-3B-Instruct, pinned at revision 6628554
architecture.json and train.py both start from the general model Qwen/Qwen2.5-VL-3B-Instruct, pinned at revision 66285546d2b821cf421d4f5eb2576359d3770cd3 and fetched with snapshot_download. train.py then applies LoRA (r=16, q_proj/v_proj) and merges it into the base model with merge_and_unload. The shipped model/ safetensors contain the standard Qwen2.5-VL-3B tensor layout, and there is a separate LoRA adapter inventory. Nothing in the code or the session loads p-doom/idm or any other specialist IDM checkpoint.
✓actual inference uses reviewed weightsserve.sh runs fully offline, installs packages only from /submission/wheels, and execs serve.py. serve.py loads Qwen2_5_
serve.sh runs fully offline, installs packages only from /submission/wheels, and execs serve.py. serve.py loads Qwen2_5_VLForConditionalGeneration from /submission/model with local_files_only=True. It decodes frames from the received video with PyAV and calls the model on each adjacent pair of frames (via model_common.conversation and parse_actions). The request id is never used to look anything up. There are no remote API calls and no label tables.
✓harness credentials used only by the harnessThe session got its GPU through Spin compute (step 12), which is allowed, and the HF downloads are allowed. Searches of
The session got its GPU through Spin compute (step 12), which is allowed, and the HF downloads are allowed. Searches of the trajectory found no step that reads, prints or copies harness credentials such as API keys, base URLs or printenv output. No step launches another harness CLI (codex exec, claude, etc.). The shipped serve.py and serve.sh do not touch any credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed1h 58m into the runtook 7m 49s
startup 90.2 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 90.2 s
- ✓POST /actions · {"actions": 12, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "0,0"}, {"frame": "F01", "type": "MouseMove", "details": "0,0"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 25.26 s, the first (cold) 27.15 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 202.7 s on the server the smoke request warmed up, 1089.7 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 1.9 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully at 08:12:24 UTC.
- The offline Qwen2.5-VL model and
POST /actionsservice are in serve.sh and serve.py. - Training data, run logs, and selection notes are in PROVENANCE.md and architecture.json. Training used GUI-360’s train split and only the supplied example clip.
- The supplied example scored 0.923 micro-F1, in-sample because it was used in training.
frontier selfcheckpassed all gates: 90.2-second startup, 27.24-second slowest request, and 1,091.6-second projected runtime for 43 clips.