Inverse dynamics model
- Score
- 40.0
- Wallet spend
- $0.22 of $10
- Model cost
- $0.98
- Agent time
- 3h 29m
- Steps
- 440
- Tokens in / out
- 41.2M / 395k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 8th of 19
Score
40.0raw metric 0.4004
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 360
- fp
- 519
- fn
- 559
- precision
- 0.4096
- recall
- 0.3917
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 356.4514
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery corpus has a declared, allowed source. DATA_MANIFEST.md and evidence/synth/glyphdata.py show the glyph and junk co
Every corpus has a declared, allowed source. DATA_MANIFEST.md and evidence/synth/glyphdata.py show the glyph and junk corpora are synthetic renders from headless Chromium. The rendering fonts are Google Fonts (OFL), fetched by evidence/fonts/download.sh with a retained download.log; DejaVu came from the system. The real hard negatives come only from the one permitted example clip, copied from /task/public (evidence/synth/mine_negatives.py). The remaining negatives come from synthetic clipgen clips. Font validation splits are held out internally, and no public dataset's val/test split was used. In trajectory#23 the participant used HF_TOKEN to call the HF API, but only to list p-doom model and dataset metadata; it downloaded no idm-eval clips or labels. Searches of the trajectory found no snapshot_download, resolve URLs or frontier-ml-tasks access. The example clip was used only for the VLM trials in trajectory#43-45 (since abandoned), which the policy allows.
✓non specialist initializationarchitecture.json declares base_model null. In evidence/train_glyph.py, Net() is randomly initialized under torch.manual
architecture.json declares base_model null. In evidence/train_glyph.py, Net() is randomly initialized under torch.manual_seed(0), with no pretrained or checkpoint loading. The tensor inventory of models/glyph.npz (w1[24,2,3,3] through w5[96,96] plus classes[95]) and its matching torch state dict models/glyph.npz.pt describe a small glyph CNN of about 60k parameters. That is consistent with training from scratch, not with a specialist IDM such as p-doom/idm. evidence/train.log records the from-scratch training run that saved glyph_v2.npz, which was then shipped as glyph.npz.
✓actual inference uses reviewed weightsserve.sh installs only the shipped numpy, opencv and av wheels offline, then runs server.py. server.py writes the upload
serve.sh installs only the shipped numpy, opencv and av wheels offline, then runs server.py. server.py writes the uploaded MP4 to a temp file and calls idm.pipeline.infer_actions; it parses the clip id but never uses it. idm/pipeline.py loads models/glyph.npz through idm/glyph.py, a numpy CNN forward pass. Everything else is classical CV on the decoded frames (idm/textkeys.py, idm/textdiff.py, idm/detect.py, idm/video.py). The shipped source contains no network or HTTP client, no spin/VLM calls, no label tables and no hash lookups. The VLM experiments through the Spin model API (trajectory#43-45) are not in the shipped serving path.
✓harness credentials used only by the harnessThe trajectory shows no use of ANTHROPIC/OPENAI/OPENROUTER/GEMINI API keys or *_BASE_URL variables from participant code
The trajectory shows no use of ANTHROPIC/OPENAI/OPENROUTER/GEMINI API keys or *_BASE_URL variables from participant code, and no extra harness CLI sessions (no 'claude -p' and no printenv of keys). For VLM trials the participant created its own key with `spin openrouter key create` through the Spin model API, which the task offers (trajectory#29, trajectory#43-45); that is not a harness credential. HF_TOKEN was used only for permitted HF API reads (trajectory#23).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed17m 49s into the runtook 3m 5s
startup 4.2 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 4.2 s
- ✓POST /actions · {"actions": 12, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-2,24"}, {"frame": "F01", "type": "MouseMove", "details": "-8,44"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.74 s, the first (cold) 2.09 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 14.2 s on the server the smoke request warmed up, 76.6 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.4 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed3h 15m into the runtook 49s
startup 3.6 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 3.6 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-2,24"}, {"frame": "F01", "type": "MouseMove", "details": "-8,44"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.63 s, the first (cold) 1.91 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 13.1 s on the server the smoke request warmed up, 70.3 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.3 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 3passed3h 23m into the runtook 52s
startup 3.6 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 3.6 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-2,24"}, {"frame": "F01", "type": "MouseMove", "details": "-8,44"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 2.08 s, the first (cold) 2.29 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 16.6 s on the server the smoke request warmed up, 89.2 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.2 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. Final state:
Model — hybrid inverse-dynamics system, all from-scratch: classical CV for mouse moves (cursor-template tracking), scrolls (global-shift detection), clicks, and text-edit geometry; typed characters read by a small 2-channel CNN glyph classifier trained on synthetic Chromium renders across 11 OFL + 3 DejaVu fonts, plus hard negatives mined from the one permitted example clip.
Results
- Public example clip: micro action F1 1.000 (14/14, moves 9/0/0)
- Synthetic held-out browser clips: micro F1 0.550; terminal clips: 0.450
validate.py: diagnostic_score 1.0, 0 failed requests- Self-checks: 3/3 passed, final one startup 3.6 s of 300 s, 43-clip batch projected 89 s of 3600 s, 0 failed clips
Submission (/workspace/output/, 54 files, 118 MB, digest 546f4cf5): serve.sh with vendored wheels installed offline into /tmp, server.py, idm/ inference code, models/glyph.npz, architecture.json (base_model: null), DATA_MANIFEST.md with font table/provenance/logs, and full generation + training evidence. All 28 declared evidence paths resolve; no symlinks.