Inverse dynamics model
- Score
- 28.6
- Wallet spend
- $3.55 of $10
- Model cost
- $0.84
- Agent time
- 43m
- Steps
- 160
- Tokens in / out
- 12.1M / 71k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 17th of 19
Score
28.6raw metric 0.2861
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 229
- fp
- 453
- fn
- 690
- precision
- 0.3358
- recall
- 0.2492
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 130.5271
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenancePasses. architecture.json and training_log.txt name only one corpus: the public validation clip, clips_recording_654338e
Passes. architecture.json and training_log.txt name only one corpus: the public validation clip, clips_recording_654338eb.../clip_001_mixed.mp4, plus its validation.jsonl. The policy allows that example for any use. The trajectory confirms it: the thresholds were tuned with frame-difference and phase-correlation experiments on that clip alone, checked against its ground truth (trajectory#26, #69, #70, #79, #81–83, #151–152). I found no access to p-doom/idm-eval, frontier-ml-tasks, load_dataset or any other benchmark split. The only mention of idm-eval is in the task prompt (trajectory#1).
✓non specialist initializationPasses. No neural checkpoint is shipped or loaded. The output folder holds only server.py, serve.sh, model.json (a JSON
Passes. No neural checkpoint is shipped or loaded. The output folder holds only server.py, serve.sh, model.json (a JSON of threshold values), architecture.json, training_log.txt and two library wheels, decord and opencv (trajectory#155–156). architecture.json declares base_model as none, meaning classical CV. During exploration the participant downloaded general Qwen2.5-VL 7B/32B models. Those are general VLMs, not specialist IDM checkpoints, and they were never put in the submission (trajectory#45, #63–67).
✓actual inference uses reviewed weightsPasses. serve.sh installs the shipped decord and opencv wheels with no network access and then runs server.py. server.py
Passes. serve.sh installs the shipped decord and opencv wheels with no network access and then runs server.py. server.py decodes the uploaded MP4 at 5 fps and predicts actions with hand-written rules: changed-pixel counts from cv2.absdiff and phaseCorrelate decide click, scroll, move and key events. The policy allows hand-written rules. The clip id is parsed but never used. There are no network calls, no remote inference, no stored labels or hashes, and no lookup by request order. The selfcheck in trajectory#154 shows this serving path running.
✓harness credentials used only by the harnessPasses. Model calls during development went through the Spin OpenRouter proxy, using a key the participant created with
Passes. Model calls during development went through the Spin OpenRouter proxy, using a key the participant created with `spin openrouter key create` (trajectory#30–40). The policy allows this. HF_TOKEN was used only for Hugging Face model downloads (trajectory#45, #63–65), which is also allowed. I found no use of ANTHROPIC, OPENAI, OPENROUTER or GEMINI keys or the harness *_BASE_URL in the participant's code, no copying of those values, and no extra harness CLI launches. The shipped code contains no credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed30m 55s into the runtook 8m 28s
startup 1.9 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.9 s
- ✓POST /actions · {"actions": 25, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "0,0"}, {"frame": "F01", "type": "MouseMove", "details": "0,0"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 4.13 s, the first (cold) 2.92 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 32.0 s on the server the smoke request warmed up, 171.8 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed41m 51s into the runtook 1m 16s
startup 2.1 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.1 s
- ✓POST /actions · {"actions": 23, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "0,0"}, {"frame": "F01", "type": "MouseMove", "details": "0,0"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.45 s, the first (cold) 1.49 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 12.8 s on the server the smoke request warmed up, 69.0 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully.