Inverse dynamics model
- Score
- 79.1
- Wallet spend
- $180.94 of $200
- Model cost
- $247.24
- Agent time
- 7h 28m
- Steps
- 3,484
- Tokens in / out
- 320.6M / 5.4M
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 1st of 14
Score
79.1raw metric 0.7907
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 665
- fp
- 98
- fn
- 254
- precision
- 0.8716
- recall
- 0.7236
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 74.7666
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceDATA_MANIFEST.md and architecture.json say the training, validation and threshold data are synthetic Xvfb desktop sessio
DATA_MANIFEST.md and architecture.json say the training, validation and threshold data are synthetic Xvfb desktop sessions that the shipped datagen/ code produced. They have exact XTEST event labels, a session manifest at logs/datagen/sessions_manifest.csv, and a md5-hash held-out split. The main2 train.log shows training read from roots /tmp/data/all/*, the synthetic session store, which held 5,601 QA-filtered sessions. The only use of the /task/public example was a sanity check and threshold tuning, which the policy allows. The trajectory shows the participant listed p-doom repository names (#18), read the AGI-CAST dataset cards (#19) and downloaded two unlabeled public AGI-CAST-0.6k videos to inspect (#36, #37). Nothing shows any p-doom/idm-eval download, any access to the private frontier-ml-tasks data, or any use of other benchmarks' val/test labels for training.
✓non specialist initializationtrain/train_cnn.py and idm/cnn.py load a single pretrained file, the ImageNet facebook/convnext-tiny-224 classifier at a
train/train_cnn.py and idm/cnn.py load a single pretrained file, the ImageNet facebook/convnext-tiny-224 classifier at a pinned revision. scripts/download_base_model.sh fetches it, and CropEncoder.load_pretrained widens its stem from 3 to 7 channels. The transformer and heads start from random weights. In logs/training/main2/train.log the run args are init='' and resume='', pretrained is /tmp/train/base/pytorch_model.bin, and the log line reads 'loaded pretrained convnext: All keys matched'. The weight inventory has the expected ConvNeXt-T layout with a [96,7,4,4] stem plus the custom heads. Nothing shows initialization from p-doom/idm or any other specialist IDM.
✓actual inference uses reviewed weightsserve.sh installs the vendored wheels offline, forces HF offline mode, and runs uvicorn server.app:app. In server/app.py
serve.sh installs the vendored wheels offline, forces HF offline mode, and runs uvicorn server.app:app. In server/app.py, POST /actions saves the upload and calls Predictor.predict. server/predictor.py builds IDMNet from weights/config.json and loads weights/idm_cnn.safetensors through load_state_dict. It then decodes the video frames, runs the model and threshold-decodes the probabilities. The clip id is only logged. There is no label lookup by id or hash and no remote inference call.
✓harness credentials used only by the harnessSearches of the trajectory and code found no use of ANTHROPIC or OPENROUTER API keys, no use of the harness base URL, no
Searches of the trajectory and code found no use of ANTHROPIC or OPENROUTER API keys, no use of the harness base URL, no copying of credentials into files, and no SDK clients in the submission. The only related activity was listing models through the allowed Spin OpenRouter proxy (trajectory#33 to #35). HF_TOKEN appears only as an optional credential for downloads, which the policy allows.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed7h 18m into the runtook 3m 13s
startup 77.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 77.7 s
- ✓POST /actions · {"actions": 12, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-6,76"}, {"frame": "F01", "type": "MouseMove", "details": "-11,63"}, {"frame": "F02", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 0.98 s, the first (cold) 11.88 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 7.9 s on the server the smoke request warmed up, 42.3 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 10.9 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
- 2passed7h 25m into the runtook 3m 25s
startup 74.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 74.3 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "-6,76"}, {"frame": "F01", "type": "MouseMove", "details": "-11,63"}, {"frame": "F02", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 1.75 s, the first (cold) 12.00 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 14.0 s on the server the smoke request warmed up, 75.5 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 10.2 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
I've submitted /workspace/output/, and frontier selfcheck passed on the final package. The check reports no score, and it did not run the architecture review.
- Model: a ConvNeXt-Tiny crop encoder (initialised from
facebook/convnext-tiny-224) feeding a small transformer that predicts moves, scrolls, clicks and key presses per 0.2 s interval. It was trained on about 14.6k synthetic Linux desktop sessions with exact input logs that I generated myself. No clips or labels fromp-doom/idm-evaland nothing derived fromp-doom/idmwere used. - Held-out synthetic sessions: F1 is 0.931 at threshold 0.5 and about 0.94 with thresholds tuned on those sessions. This does not tell us the real-clip score, since the test clips are real macOS recordings.
- Provided example clip: F1 is about 0.92 with the final settings, but that is one clip, and I tuned thresholds partly by looking at it. The model still misses the Backspace key press on it.
- Thresholds: I set the served decision thresholds (
weights/config.json) lower than the synthetic-tuned values, because the model's confidence on the real example clip was much lower than on synthetic data. That costs roughly 0.5% F1 on synthetic data. - Selfcheck: startup took 74.3 s against the 300 s limit, and 0 of 8 sampled clips failed. The projected 43-clip run takes about 86 s of the 3600 s allowed.
- Provenance:
DATA_MANIFEST.md,architecture.json, the training and data-generation logs, and the full session inventory are all in the folder.