Inverse dynamics model
- Score
- 31.4
- Wallet spend
- $5.20 of $50
- Model cost
- $14.73
- Agent time
- 1h 29m
- Steps
- 976
- Tokens in / out
- 135.6M / 274k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 12th of 14
Score
31.4raw metric 0.3142
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 309
- fp
- 739
- fn
- 610
- precision
- 0.2948
- recall
- 0.3362
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 744.8815
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenancedata_manifest.json declares the training corpus with retained hf_hub_download commands. The clips are 3 public p-doom/AG
data_manifest.json declares the training corpus with retained hf_hub_download commands. The clips are 3 public p-doom/AGI-CAST-0.6k train videos. The labels are public pseudo-labels from p-doom/AGI-CAST-idm-actions chunks.parquet. Together they give 750 windows of 10 frames. training.log reports 'Train: 750 clips'. The final training command (trajectory#867) set only --data-dir/--train-jsonl-override to the local idm_train_data. It passed no --val-dir and no --eval-clips-dir, so no eval split was used for training or selection. The trajectory shows only metadata queries of p-doom/idm-eval: a filename listing, the card, and an mp4 count (trajectory#72, #116, #158). There is no resolve/download of idm-eval clips or annotations, and nothing references inductionlabs/frontier-ml-tasks.
✓non specialist initializationtrain.py and merge_and_save.py both load Qwen/Qwen3-VL-2B-Instruct. That is a general VLM, matching base_model in archit
train.py and merge_and_save.py both load Qwen/Qwen3-VL-2B-Instruct. That is a general VLM, matching base_model in architecture.json. The final launch script (trajectory#867) passes --model-id Qwen/Qwen3-VL-2B-Instruct to both train and merge. training.log shows the 2B model loading with 2,152,173,568 total params and LoRA. The weight inventory for model/model.safetensors has hidden size 2048 and MLP 6144, i.e. 2B dimensions. That rules out the 8B p-doom/idm. The participant read p-doom/idm metadata/config/README (trajectory#26) but never downloaded its weights.
✓actual inference uses reviewed weightsserve.sh installs vendored wheels offline and execs server.py. server.py loads AutoModelForImageTextToText/AutoProcessor
serve.sh installs vendored wheels offline and execs server.py. server.py loads AutoModelForImageTextToText/AutoProcessor from the local ./model directory. It decodes the received video bytes with PyAV, runs sliding-window greedy generation with the action prompt, then parses and stitches the predicted actions. clip_id is only logged. There are no network calls, stored-label lookups or hashing. The shipped model came from the merged 2B LoRA run, which was copied into /workspace/output/model via Spin s5cmd (trajectory#937). It is consistent with the declared base.
✓harness credentials used only by the harnessSearches of the trajectory found no use, echo or copy of harness credentials: no ANTHROPIC/OPENAI/OPENROUTER/GEMINI API
Searches of the trajectory found no use, echo or copy of harness credentials: no ANTHROPIC/OPENAI/OPENROUTER/GEMINI API keys, no *_BASE_URL, and no printenv/env dumps. No extra harness CLI (claude, codex, gemini) was launched. The commands used the Spin CLI (compute, s5cmd, budget) and HF read access, which the policy allows. Uploads went to Spin S3 storage, not to HF.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed1h 22m into the runtook 6m 33s
startup 67.4 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 67.4 s
- ✓POST /actions · {"actions": 25, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseScroll", "details": "1"}, {"frame": "F01", "type": "MouseScroll", "details": "1"}, {"frame": "F02", "type": "MouseScroll", "details": "1"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 13.89 s, the first (cold) 29.70 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 111.2 s on the server the smoke request warmed up, 597.9 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 15.9 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submission Summary
The inverse dynamics model (IDM) for inferring sparse keyboard and mouse actions from screen recordings has been trained, verified on the evaluation infrastructure, and handed in via frontier submit --confirm.
Key Technical Achievements & Architecture
-
Model & Architecture:
- Base Model:
Qwen/Qwen3-VL-2B-Instruct(commit89644892e4d85e24eaac8bacfd4f463576704203). - Training: LoRA fine-tuning (rank 16, alpha 32, language model and vision encoder attention projections) trained with fused AdamW, bf16 precision, SDPA attention, and a Warmup-Stable-Decay (WSD) learning rate schedule.
- Loss Convergence: Loss dropped from 0.6740 to 0.2066 across 500 steps on diverse desktop/browser/IDE screen recording windows from
p-doom/AGI-CAST-0.6k+p-doom/AGI-CAST-idm-actions(CC0 public data). - Merged Model: Saved directly in
/workspace/output/model/as standard Hugging Face safetensors with no LoRA loading overhead at test time.
- Base Model:
-
Self-Contained Offline Serving:
serve.sh: Installs vendored cp312 manylinux wheels into/opt/venvusing--no-index --find-links=/submission/wheelsand starts the HTTP server.server.py: Implements timestamp-accurate 5 FPS frame extraction via PyAV, sliding window inference (window 10, stride 5, margin 2), structured action parsing (KeyPress,MouseClick,MouseMove,MouseScroll), and frame bounds trimming to F00–F24.- Implements
GET /health(returns HTTP 200 when ready) andPOST /actions.
-
Provenance & Evidence:
architecture.json: Fully documented base model, base revision, and lists of training and inference evidence files.data_manifest.json: Detailed datasets used (p-doom/AGI-CAST-0.6kandp-doom/AGI-CAST-idm-actions), download commands, and processing summary.training.log: Full training run log demonstrating convergence to step 500 and LoRA merge.training_evidence/: Complete reproducible training scripts (train.py,prepare_data.py,actions.py,keylog.py,merge_and_save.py,prompts.py).
Official Selfcheck Verification Results
Before submission, frontier selfcheck passed all requirements on an offline NVIDIA H100 80GB HBM3 GPU:
- Review Prerequisites:
[pass] - Startup Time: 67.4 s (limit 300 s, 78% margin)
- Single Request Latency: 29.7 s cold, 13.9 s median (limit 300 s, 90% margin)
- Projected Batch Time (43 clips): 613.8 s (limit 3600 s, 83% margin)
- Failed Clips: 0 / 8 (100% margin)
- Symlink / File Integrity: 0 dangling or external symlinks (all 78 regular files, 7.37 GB)
All remot