Inverse dynamics model
- Score
- 19.3
- Wallet spend
- $2.70 of $10
- Model cost
- $34.75
- Agent time
- 6h 30m
- Steps
- 767
- Tokens in / out
- 180.3M / 829k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 14th of 14
Score
19.3raw metric 0.1929
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 130
- fp
- 299
- fn
- 789
- precision
- 0.3030
- recall
- 0.1415
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 2164.8347
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training corpus is declared in DATA_MANIFEST.md and architecture.json, and none is forbidden. The declared corpora
Every training corpus is declared in DATA_MANIFEST.md and architecture.json, and none is forbidden. The declared corpora are: - synthetic screen-recording clips the participant generated with code/sim_gen/gen_clip.py (Playwright pages in code/pages, Xvfb/x11grab capture, pynput input, with generation logs retained); - glyphs rendered from public google/fonts and matplotlib fonts by code/train/render_glyphs.py; - glyph crops and frame-pair windows mined from those synthetic clips by code/train/mk_pairs.py. The manifest says no calibration or selection corpus is used. The single allowed example clip is used only for cursor templates and checks. code/train/cls_train.py and code/train/glyph_train.py use only self-generated train/val splits. The trajectory shows no download of p-doom/idm-eval, inductionlabs/frontier-ml-tasks or any benchmark validation/test split; 'idm-eval' appears only in the task prompt and the participant's documentation. The font download is declared but no download log was retained, so it is recorded as unverified.
✓non specialist initializationcode/train/cls_train.py starts from torchvision models.resnet18(weights=ResNet18_Weights.IMAGENET1K_V1). conv1 is widene
code/train/cls_train.py starts from torchvision models.resnet18(weights=ResNet18_Weights.IMAGENET1K_V1). conv1 is widened to 6 channels by concatenating the pretrained weights and halving them, and new click/key/move/scroll heads are added. This matches the model/window.onnx tensor inventory: conv1 64x6x7x7, fc.1 256x512, and heads of 4x256, 1x256, 1x256 and 3x256. code/train/glyph_train.py trains a small 5-conv CNN with an 81-class head from random initialization, matching the model/glyph.onnx inventory. The ImageNet base is a general-purpose model, not a specialist IDM, and no p-doom/idm checkpoint appears in the code or the trajectory.
✓actual inference uses reviewed weightsserve.sh builds an offline venv from the vendored wheels and runs server/app.py. In server/app.py, load_models loads mod
serve.sh builds an offline venv from the vendored wheels and runs server/app.py. In server/app.py, load_models loads model/window.onnx, model/glyph.onnx (with glyph_classes.json) and model/templates/*.npz through onnxruntime. infer_clip decodes the uploaded MP4 and computes predictions only from it: - cursor tracking (server/track.py); - phase-correlation scroll detection (server/scroll.py); - window classification with window.onnx; - click rules (server/clicks.py); - OCR-diff key inference with glyph.onnx (server/keys.py). The optional vap.npz is not shipped, so the code falls back to a constant. There is no clip-ID or hash lookup, no stored label table and no remote inference call. Rule-based components are allowed by policy.
✓harness credentials used only by the harnessThe trajectory has no references to OPENROUTER, ANTHROPIC, OPENAI or GEMINI keys or any *_BASE_URL. There are no spin op
The trajectory has no references to OPENROUTER, ANTHROPIC, OPENAI or GEMINI keys or any *_BASE_URL. There are no spin openrouter calls and no launches of claude or codex harness CLIs. The only 'API_KEY' text is a CONTAINER_API_KEY in vast container setup output (steps 265-290), which is not a harness credential. Compute and storage went through the Spin CLI: a compute lease at step 41 and spin s5cmd/transfer for uploads, both of which policy allows. HF_TOKEN was used, if at all, only for a connectivity check (curl -sI), and nothing was uploaded.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed6h 25m into the runtook 3m 36s
startup 8.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 8.7 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-2,24"}, {"frame": "F01", "type": "MouseMove", "details": "-3,17"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 19.98 s, the first (cold) 20.57 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 156.4 s on the server the smoke request warmed up, 840.5 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.7 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. Final state: diagnostic 1.0 on the public clip (all four action types F1=1.0), frontier selfcheck passed 3/3 gates with 8.7s startup, 19 committed unit tests green.
What shipped in /workspace/output/: serve.sh (offline venv + vendored wheels, boots in <45s) serving POST /actions via app.py — cursor template tracking, tiled-shift scroll detection with single-tick spill merge, three-rule CV click detector (Hough focus-ring, navigation, still-cursor toggle in clicks.py), and OCR-diff key heuristics over the glyph CNN. The synthetic-trained window classifier's click head fired nothing on real data, so clicks are purely CV; its key head is retained only as a heuristic gate.
Key judgment calls: the example's F10–F18 "typing" has zero keylog rows (keyless streamed text), so the glyph path stays quiet rather than risk streaming FPs; toggle/menu recall was validated against 41k mined before/after pairs (45% size-band recall) instead of the frozen local video batch.