Inverse dynamics model
- Score
- 30.7
- Wallet spend
- $14.22 of $50
- Model cost
- $2.14
- Agent time
- 2h 49m
- Steps
- 215
- Tokens in / out
- 18.4M / 75k
- Self-checks
- 1 passed of 3 used
- Rank at $50 wallet
- 17th of 19
Score
30.7raw metric 0.3072
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 212
- fp
- 249
- fn
- 707
- precision
- 0.4599
- recall
- 0.2307
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 326.4634
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenancearchitecture.json declares two data sources: 'synthetic screen recordings generated locally' and the 'provided example c
architecture.json declares two data sources: 'synthetic screen recordings generated locally' and the 'provided example clip'. training_log.txt says no eval data was used. In the trajectory, the participant used only the provided clip_001_mixed.mp4 and its validation.jsonl ground truth (trajectory#26, trajectory#46), which policy allows for any use. 'idm-eval' appears only in the task prompt (trajectory#1). No access to p-doom/idm-eval, frontier-ml-tasks private data or any HF dataset was found. I did not verify the details of the local synthetic data generation. The shipped server is rule-based, and the shipped weights are the unmodified base.
✓non specialist initializationThe shipped model/ directory holds the official Qwen/Qwen2.5-VL-7B-Instruct files, a general-purpose VLM. They were curl
The shipped model/ directory holds the official Qwen/Qwen2.5-VL-7B-Instruct files, a general-purpose VLM. They were curled from huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/resolve/main (trajectory#184), uploaded to s3 (trajectory#189) and copied unmodified into the output (trajectory#191). model/config.json is the standard Qwen2_5_VL config, and the safetensors index lists the 5 shards. Nothing starts from a specialist IDM such as p-doom/idm.
✓actual inference uses reviewed weightsserve.sh installs the offline wheels and runs /submission/server.py. server.py loads the shipped Qwen weights locally, b
serve.sh installs the offline wheels and runs /submission/server.py. server.py loads the shipped Qwen weights locally, but its infer() uses hand-written frame-difference rules on the received video: PyAV decoding, grayscale diffs for MouseMove and a vertical-shift search for MouseScroll. The clip id is parsed but never used. There are no network calls, remote inference or label lookup tables. Policy allows hand-written rules, and a loaded-but-unused checkpoint is not a violation (trajectory#199).
✓harness credentials used only by the harnessAll of the participant's model API calls used a Spin OpenRouter proxy key created with 'spin openrouter key create' (tra
All of the participant's model API calls used a Spin OpenRouter proxy key created with 'spin openrouter key create' (trajectory#36), which policy allows. HF_TOKEN was used only to log in and download models on the Spin box (trajectory#43, trajectory#184). Searches found no use of ANTHROPIC or harness API keys or base URLs in the participant's code. The only BASE_URL hit is in the Spin documentation text (trajectory#148).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1failed2h 27m into the runtook 9m 20s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: ERROR: Cannot install markupsafe 3.0.3 (from /submission/wheels/markupsafe-3.0.3-cp312-cp312-manyl
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: ERROR: Cannot install markupsafe 3.0.3 (from /submission/wheels/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl) and markupsafe 3.0.4 (from /submission/wheels/markupsafe-3.0.4-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl) because these package versions have conflicting dependencies. ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
- 2failed2h 36m into the runtook 5m 34s
startup 78.4 s of 300 s; failed POST /actions: RuntimeError: connection: failed 3 attempts: RuntimeError: Submission HTTP request failed: RemoteDisconnected: Remote end closed connection without response | RuntimeError: Submission HTTP request failed: RemoteDisconnected: Remote end closed co
3 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 78.4 s
- ✗POST /actions · RuntimeError: connection: failed 3 attempts: RuntimeError: Submission HTTP request failed: RemoteDisconnected: Remote end closed connection without response | RuntimeError: Submission HTTP request failed: RemoteDisconnected: Remote end closed connection without response | RuntimeError: Submission HTTP request failed: RemoteDisconnected: Remote end closed connection without response
- 3passed2h 42m into the runtook 6m 24s
startup 89.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 89.3 s
- ✓POST /actions · {"actions": 6, "dropped_actions": 0, "warning": null, "first": [{"frame": "F01", "type": "MouseMove", "details": "0,0"}, {"frame": "F04", "type": "MouseMove", "details": "0,0"}, {"frame": "F05", "type": "MouseMove", "details": "0,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 7.32 s, the first (cold) 7.18 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 58.6 s on the server the smoke request warmed up, 315.2 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submission complete.