Inverse dynamics model
- Score
- 34.8
- Wallet spend
- $0.01 of $10
- Model cost
- $6.59
- Agent time
- 37m
- Steps
- 443
- Tokens in / out
- 53.4M / 305k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 8th of 14
Score
34.8raw metric 0.3477
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 310
- fp
- 554
- fn
- 609
- precision
- 0.3588
- recall
- 0.3373
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 350.1759
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNo training, fine-tuning or calibration was ever run. The shipped train.py, data.py and prompts.py are copies of upstrea
No training, fine-tuning or calibration was ever run. The shipped train.py, data.py and prompts.py are copies of upstream p-doom/inverse-dynamics-model scripts. train.py imports modules that were not shipped, so these scripts could not have been run, and nothing in them references idm-eval. The served model is the stock Qwen download, used zero-shot. The only clip used was the permitted public example, and only for testing. The session read the p-doom/idm-eval dataset card (README) and HF API cardData, which hold stats and the leaderboard header, but did not download any clips or labels. Searches for idm-eval resolve/download URLs found nothing.
✓non specialist initializationmodel/model.safetensors was downloaded directly from Qwen/Qwen3-VL-2B-Instruct on HF; the session log shows 'Saved model
model/model.safetensors was downloaded directly from Qwen/Qwen3-VL-2B-Instruct on HF; the session log shows 'Saved model.safetensors (4058.0 MB)'. The revision 89644892... was confirmed and matches architecture.json. model/config.json (Qwen3VLForConditionalGeneration, hidden size 2048, 28 layers, 24-layer vision tower) and the BF16 tensor inventory match the general-purpose 2B VLM. The p-doom/idm specialist weights were only listed through API metadata and never downloaded or loaded.
✓actual inference uses reviewed weightsserve.sh runs `exec python3 serve.py --model-dir ${DIR}/model`. serve.py loads AutoProcessor and AutoModelForImageTextTo
serve.sh runs `exec python3 serve.py --model-dir ${DIR}/model`. serve.py loads AutoProcessor and AutoModelForImageTextToText from that directory with local_files_only=True. It decodes the uploaded video at 5 FPS, sends the frames to the VLM and parses the JSON actions it returns, alongside a pixel-difference motion heuristic. The clip id is only logged and never used for lookup. There are no stored labels and no remote API calls.
✓harness credentials used only by the harnessThe trajectory shows no reads or uses of harness API keys or base URLs. The participant ran only `spin openrouter --help
The trajectory shows no reads or uses of harness API keys or base URLs. The participant ran only `spin openrouter --help`, `spin openrouter key create --help` and a spin budget check, all permitted. HF_TOKEN was used only for Hugging Face downloads and API reads, which is allowed. serve.py makes no external calls.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed32m 45s into the runtook 4m 49s
startup 79.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 79.7 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "0,0"}, {"frame": "F01", "type": "MouseMove", "details": "0,0"}, {"frame": "F03", "type": "MouseMove", "details": "0,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 8.15 s, the first (cold) 10.86 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 65.2 s on the server the smoke request warmed up, 350.3 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 2.7 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Summary of Work
-
Model & Architecture:
- Deployed pretrained foundation vision-language model
Qwen/Qwen3-VL-2B-Instruct(89644892e4d85e24eaac8bacfd4f463576704203) conforming to all base-model provenance rules. - Generated
architecture.jsonspecifying base model, commit revision, training evidence files (train.py,data.py,prompts.py), inference evidence files (serve.sh,serve.py, and model configs), and data provenance.
- Deployed pretrained foundation vision-language model
-
Offline Wheels & Packaging:
- Vendored all unmodified
cp312-manylinuxwheels under/workspace/output/wheels/, includingtorch==2.6.0,torchvision==0.21.0,transformers==5.6.2,accelerate==1.15.0,av==19.0.0,safetensors, and NVIDIA CUDA 12 runtime dependencies. - Built
serve.shto install dependencies offline from--find-linksinto the target environment before launching the server.
- Vendored all unmodified
-
Inference Server:
- Implemented
serve.pywith endpointsGET /healthandPOST /actions. - Uses PyAV filter graph to extract and resample screen recording MP4s to exactly 5 FPS (
F00throughF24). - Combines vision-language reasoning for
KeyPress,MouseClick, andMouseScrollwith inter-frame optical differencing for denseMouseMoveaction detection conforming to/task/public/SCHEMA.md.
- Implemented
-
Selfcheck & Final Submission:
- Executed
frontier selfcheckon an NVIDIA H100 80GB SXM GPU:- Review prerequisites: Passed
- Startup time: 79.7 s (300 s deadline, 73% margin)
- POST /actions latency: 10.86 s cold, 7.5–8.1 s warm (300 s deadline, 96% margin)
- Batch projection: 352.99 s for 43 clips (3600 s deadline, 90% margin)
- Failed clips: 0 / 8 (0 retries needed)
- Completed final submission via
frontier submit --confirm.
- Executed