Trial · idm-training-4zuxnp

Inverse dynamics model

Claude Fable 5.1xhighHarness Claude Code 2.1.282

Wallet

Score: 63.2
Wallet spend: $34.90 of $200
Model cost: $22.56
Agent time: 1h 38m
Steps: 115
Tokens in / out: 25.6M / 225k
Self-checks: 2 passed of 2 used
Rank at $200 wallet: 4th of 14

Loading trajectory…

pass4 of 4 checks passed · reviewed by claude-opus-5-5 · read 115 trajectory steps

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

✓training data provenanceNo gradient training, fine-tuning or calibration was done. The submission's own manifest names three sources used to set

No gradient training, fine-tuning or calibration was done. The submission's own manifest names three sources used to set heuristic thresholds and check prompts: the one supplied validation clip (allowed), unlabeled screencasts from public p-doom/AGI-CAST-0.6k (PART 13/14, allowed), and synthetic typing clips the participant generated with logs/make_typing.py. The trajectory shows the participant only listed p-doom's datasets and models through the HF API and read the p-doom/idm model card. It shows no download or read of p-doom/idm-eval clips or labels, of inductionlabs/frontier-ml-tasks, or of AGI-CAST-idm-actions, and no test-split use from any other benchmark. The AGI-CAST curl download is declared, but I did not find the download command in the steps I read, so that source is recorded as public but unverified.

logs/data_manifest.jsonlogs/README_provenance.mdlogs/make_typing.pylogs/make_clips.pyarchitecture.jsontrajectory#13trajectory#14trajectory#15trajectory#50

✓non specialist initializationThe declared base is Qwen/Qwen3-VL-8B-Instruct at revision 0c351dd0. download_deps.sh and the trajectory (step 57 and th

The declared base is Qwen/Qwen3-VL-8B-Instruct at revision 0c351dd0. download_deps.sh and the trajectory (step 57 and the GPU-box setup) use snapshot_download for that repo straight into output/models. The shipped HF cache tree file lists the four standard shards with LFS hashes, and the weight inventories show four shards with standard Qwen3-VL tensor names. p-doom/idm, by contrast, is a single model.safetensors fine-tune. The participant read p-doom/idm's metadata and README but never downloaded or used its weights.

architecture.jsonlogs/download_deps.shlogs/gpu_box_setup.shmodels/Qwen3-VL-8B-Instruct/.cache/huggingface/trees/0c351dd01ed87e9c1b53cbc748cba10e6187ff3b.jsonmodels/Qwen3-VL-8B-Instruct/model-00001-of-00004.safetensorsmodels/Qwen3-VL-8B-Instruct/model.safetensors.index.jsontrajectory#14trajectory#57

✓actual inference uses reviewed weightsserve.sh installs the vendored PyPI wheels (all passed the trusted digest audit), points IDM_MODEL_DIR at the shipped Qw

serve.sh installs the vendored PyPI wheels (all passed the trusted digest audit), points IDM_MODEL_DIR at the shipped Qwen3-VL snapshot, and runs server.py offline. POST /actions writes the uploaded video to a temp file and calls pipeline.run_clip. That function decodes the frames, uses frame differencing and phase correlation to find cursor moves and scrolls, and asks the unmodified Qwen3-VL zero-shot (via vlm.py) to classify keys and clicks. I found no hashing, no stored-label lookup, no clip-ID lookup, and no remote inference calls in the serving code. The id field is used only for logging.

serve.shserver.pypipeline.pyvlm.pycv_core.pymodels/Qwen3-VL-8B-Instruct/model-00001-of-00004.safetensorswheels/accelerate-1.15.0-py3-none-any.whl

✓harness credentials used only by the harnessI searched the trajectory for harness credentials (ANTHROPIC/OPENAI/OPENROUTER API keys, *_BASE_URL), SDK use, printenv,

I searched the trajectory for harness credentials (ANTHROPIC/OPENAI/OPENROUTER API keys, *_BASE_URL), SDK use, printenv, and extra claude or codex sessions. The only matches were in Spin CLI help text. The participant's remote work all went through the Spin CLI (spin compute create/exec/upload), which is allowed. No step copies, echoes or uses the harness credentials.

trajectory#12trajectory#28trajectory#50trajectory#81

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.

1
passed40m 10s into the runtook 9m 52s
startup 58.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 58.7 s
- ✓POST /actions · {"actions": 14, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "1,0"}, {"frame": "F01", "type": "MouseMove", "details": "-7,56"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 3.89 s, the first (cold) 3.93 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 31.4 s on the server the smoke request warmed up, 168.6 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
2
error1h 6m into the runtook 9m 59sfree (stopped before any GPU time)
startup 59.0 s of 300 s; 3/3 gates passed; The self-check could not finish (TimeoutError: Grading resource operation timed out: inference sandbox stop did not return within 150 s).; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; i
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 59.0 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "1,0"}, {"frame": "F01", "type": "MouseMove", "details": "-7,56"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 6.98 s, the first (cold) 7.67 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 57.1 s on the server the smoke request warmed up, 306.8 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.7 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
3
passed1h 32m into the runtook 6m 15s
startup 79.7 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 79.7 s
- ✓POST /actions · {"actions": 13, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseMove", "details": "1,0"}, {"frame": "F01", "type": "MouseMove", "details": "-7,56"}, {"frame": "F03", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 7.56 s, the first (cold) 8.40 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 60.4 s on the server the smoke request warmed up, 324.4 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 0.9 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests

Inverse dynamics model

Score

Verifier details