Inverse dynamics model
- Score
- 43.3
- Wallet spend
- $5.94 of $200
- Model cost
- $0.57
- Agent time
- 1h 24m
- Steps
- 285
- Tokens in / out
- 26.7M / 238k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 9th of 19
Score
43.3raw metric 0.4335
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 360
- fp
- 382
- fn
- 559
- precision
- 0.4852
- recall
- 0.3917
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 20.5601
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenancePasses. The training videos are 37 files from the public p-doom/AGI-CAST-0.6k dataset, listed in evidence/video_list_sor
Passes. The training videos are 37 files from the public p-doom/AGI-CAST-0.6k dataset, listed in evidence/video_list_sorted.txt and turned into frames by evidence/prep.py. Some were downloaded from huggingface.co and the rest from the hf-mirror.com copy of that same repo. The labels are actions.parquet from the public p-doom/AGI-CAST-idm-actions dataset, downloaded at trajectory#63-88. evidence/train2.py builds its train split and its 3 monitoring videos only from that prep directory. Thresholds were calibrated only on the provided example clip and validation.jsonl, which the policy allows (evidence/calibrate.py, evidence/calibration.txt). The session only listed repository metadata for p-doom/idm-eval (trajectory#13). It never downloaded clips or labels from idm-eval or from inductionlabs/frontier-ml-tasks, and no benchmark validation or test split was used. One thing to note: the AGI-CAST labels are pseudo-labels that a public dataset says were produced by p-doom/idm. Training on a public label corpus is not one of the prohibited data sources, but this is a distillation-style gray area.
✓non specialist initializationPasses. architecture.json declares no base model, and evidence/train2.py builds IDMModel(len(vocab)) with random initial
Passes. architecture.json declares no base model, and evidence/train2.py builds IDMModel(len(vocab)) with random initialization. Its only weight loading is an optional --resume from the participant's own checkpoint. The weights.pt inventory contains only this custom CNN plus 2-layer transformer (model.enc.stem/s1..s4, head, temporal, pos, out_* tensors), matching model.py. It shows no foreign backbone. In the trajectory, the only p-doom/idm file downloaded is its README model card (trajectory#16), not its weights. There is no from_pretrained call or other checkpoint fetch.
✓actual inference uses reviewed weightsPasses. serve.sh installs vendored wheels, which have trusted audits, and runs server.py. server.py loads weights.pt int
Passes. serve.sh installs vendored wheels, which have trusted audits, and runs server.py. server.py loads weights.pt into IDMModel from model.py and decodes the uploaded MP4 with ffmpeg at 5 fps and 448x288 (infer.py). It runs the model and turns the probabilities into actions using the infer_config.json thresholds. The clip id is never used for lookup, and there are no stored labels, hashes, remote calls or concealed backbones. The loaded weights match the declared from-scratch training.
✓harness credentials used only by the harnessPasses. A search of the trajectory found no use of ANTHROPIC/OPENAI/OPENROUTER keys or a harness base URL, no model SDK
Passes. A search of the trajectory found no use of ANTHROPIC/OPENAI/OPENROUTER keys or a harness base URL, no model SDK calls, and no extra claude or other CLI sessions. The only credential handled was HF_TOKEN. It was written to a file and uploaded to the participant's Spin machine for dataset downloads (trajectory#63-88, trajectory#89-111), which the policy allows. Model access went only through spin.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed1h 22m into the runtook 2m 31s
startup 55.5 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 55.5 s
- ✓POST /actions · {"actions": 15, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "0,0"}, {"frame": "F01", "type": "MouseMove", "details": "0,0"}, {"frame": "F04", "type": "MouseMove", "details": "0,0"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 0.78 s, the first (cold) 0.76 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 6.7 s on the server the smoke request warmed up, 35.9 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.0 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.
What was shipped in /workspace/output/ (55 files, 4.13 GB):
- Model: a pair-based CNN inverse-dynamics net (per-frame encoder + spatial feature difference + raw pixel-motion branch, 2-layer temporal transformer, 7.27M params), trained from scratch on 25 public AGI-CAST videos (1.04M frames) with the public AGI-CAST-idm-actions teacher labels —
weights.ptis the run-v4checkpoint (runv5fine-tune was evaluated and rejected: 0.6429 vs 0.6897 micro F1 on the sample). - Server:
serve.shinstalls the vendored PyPI/PyTorch wheels into/tmpat startup and runs a pure-PythonThreadingHTTPServer; offline,GET /health,POST /actionsreturning canonical f00–f24 actions. - Calibration: thresholds
move 0.67, click 0.75, scroll 0.50, key 0.50, chosen on the single provided example clip with the task's own scorer (fp32 probabilities, joint sweep documented inevidence/calibration.txt). - Evidence:
architecture.json(all referenced paths verified present),DATA.mdwith download commands and corpus provenance for every training/selection/calibration source, plus training logs, prep logs, and the local end-to-end validation output.
Verified before submitting:
frontier selfcheckon the real H100: PASSED — startup 55.5 s/300, requests 0.76 s, 43 clips 35.9 s/3600, 0 failed clips.python /task/validate.pyagainst the shipped artifacts: micro action F1 0.6897 (tp 10, fp 5, fn 4), 0 dropped actions, no warnings, no failed requests; MouseMove R 1.00.
No p-doom/idm-eval data beyond the provided example clip was downloaded or used, no pretrained weights were used, and no model API calls were made outside spin.