Inverse dynamics model
- Score
- 25.1
- Wallet spend
- $13.96 of $200
- Model cost
- $25.13
- Agent time
- 2h 24m
- Steps
- 116
- Tokens in / out
- 40.1M / 111k
- Self-checks
- 1 passed of 2 used
- Rank at $200 wallet
- 12th of 14
Score
25.1raw metric 0.2514
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 152
- fp
- 138
- fn
- 767
- precision
- 0.5241
- recall
- 0.1654
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 154.3967
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceThe submission declares its data source as the PSAI train split (anaisleila/computer-use-data-psai, MIT licence) in arch
The submission declares its data source as the PSAI train split (anaisleila/computer-use-data-psai, MIT licence) in architecture.json, logs/data_manifest.json and logs/download_commands.txt. p-doom repos appear in those files only as exclusions. The original download log is not shipped, so I record the download as declared but unverified. In the trajectory, the lease's train.jsonl contains 2115 clips, and every one has split 'train' and source PSAI (trajectory#14). The rule thresholds were tuned on the allowed public example clip (trajectory#17, trajectory#26-32) and on the PSAI train clips (trajectory#34). I found no access to p-doom/idm-eval, to the private inductionlabs data, or to any validation or test split of another dataset.
✓non specialist initializationThe served program loads no neural weights. architecture.json sets base_model to none, and server.py imports only fastap
The served program loads no neural weights. architecture.json sets base_model to none, and server.py imports only fastapi, uvicorn, numpy and PIL. The only model the participant fine-tuned was the general-purpose Qwen2.5-VL-7B-Instruct, with LoRA, at a pinned revision (logs/train.log, trajectory#8). Those runs were abandoned and are not served. I found no use of p-doom/idm or any other specialist IDM.
✓actual inference uses reviewed weightsserve.sh installs the vendored wheels without network access and runs `exec python server.py`. server.py uses ffmpeg to
serve.sh installs the vendored wheels without network access and runs `exec python server.py`. server.py uses ffmpeg to extract 25 frames from the uploaded MP4 and runs a hand-written frame-difference and cursor-blob detector to emit MouseMove actions. It ignores the id field, has no lookup table or hashing, and makes no remote calls. Hand-written rules may replace a trained model under the policy, and no unreviewed weights are served.
✓harness credentials used only by the harnessIn the trajectory, compute ran only through the spin CLI (spin docs, spin compute exec and spin compute download), which
In the trajectory, compute ran only through the spin CLI (spin docs, spin compute exec and spin compute download), which the policy allows (trajectory#15, trajectory#34, trajectory#35). Searches for API_KEY, BASE_URL, openrouter, claude, and other CLIs found only spin's own documentation. No step calls a model API with harness keys, copies credentials, or launches another harness CLI.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed1h 38m into the runtook 8m 48s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: -cu12, nvidia-cudnn-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: -cu12, nvidia-cudnn-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, mdurl, markupsafe, markdown-it-py, jinja2, idna, huggingface-hub, httpx, httpcore, hf-xet, h11, fsspec, filelock, fastapi, click, certifi, anyio, annotated-types, annotated-doc, accelerate Successfully installed accelerate-1.15.0 annotated-doc-0.0.5 annotated-types-0.8.0 anyio-4.15.1 certifi-2026.7.22 click-8.5.0 fastapi-0.118.0 filelock-3.32.3 fsspec-2026.7.0 h11-0.16.0 hf-xet-1.7.0 httpcore-1.0.9 httpx-0.28.1 huggingface-hub-1.33.0 idna-3.20 jinja2-3.1.6 markdown-it-py-4.2.0 markupsafe-3.0.3 mdurl-0.1.2 mpmath-1.3.0 networkx-3.6.1 numpy-2.5.2 nvidia-cublas-cu12-12.8.3.14 nvidia-cuda-cupti-cu12-12.8.57 nvidia-cuda-nvrtc-cu12-12.8.61 nvidia-cuda-runtime-cu12-12.8.57 nvidia-cudnn-cu12-9.7.1.26 nvidia-cufft-cu12-11.3.3.41 nvidia-cufile-cu12-1.13.0.11 nvidia-curand-cu12-10.3.9.55 nvidia-cusolver-cu12-11.7.2.55 nvidia-cusparse-cu12-12.5.7.53 nvidia-cusparselt-cu12-0.6.3 nvidia-nccl-cu12-2.26.2 nvidia-nvjitlink-cu12-12.8.61 nvidia-nvtx-cu12-12.8.55 opentelemetry-api-1.45.1 packaging-26.3 peft-0.21.2 pillow-12.3.0 psutil-7.2.2 pydantic-2.14.0 pydantic-core-2.50.0 pygments-2.21.0 python-multipart-0.0.20 pyyaml-6.0.3 regex-2026.9.29 rich-15.0.0 safetensors-0.8.0 setuptools-78.1.0 shellingham-1.5.4 starlette-1.7.0 sympy-1.14.0 tokenizers-0.23.2 torch-2.7.1+cu128 torchvision-0.22.1+cu128 tqdm-4.70.1 transformers-5.19.0 triton-3.3.1 typer-0.27.3 typing-extensions-4.16.0 typing-inspection-0.4.4 uvicorn-0.37.0 Traceback (most recent call last): File "/submission/server.py", line 40, in <module> app = FastAPI() ^^^^^^^^^ File "/tmp/idm-site/fastapi/applications.py", line 926, in __init__ ] = webhooks or routing.APIRouter() ...[clipped]
- 2passed2h 22m into the runtook 54s
startup 4.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 4.7 s
- ✓POST /actions · {"actions": 7, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "1,1"}, {"frame": "F01", "type": "MouseMove", "details": "1,1"}, {"frame": "F04", "type": "MouseMove", "details": "1,1"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 2.63 s, the first (cold) 2.74 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 21.0 s on the server the smoke request warmed up, 113.0 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in. Grading will use /workspace/output as it stood at submit time.
The server is a cursor tracker, not a language model. On each pair of 5 FPS frames it looks for one to three compact, pointer-sized changes on an otherwise quiet frame, and emits a MouseMove there. Mouse-move details are not scored, so the delta is a unit step. It always returns HTTP 200 and a JSON array.
On the one allowed example that scores 0.57 micro action F1 (mouse-move F1 0.75: 6 true positives, 1 false positive, 3 misses). Keys, clicks, and the scroll are not predicted. The official self-check passed: healthy in 4.7 seconds, each clip about 2.6 seconds, no failed requests.