Inverse dynamics model
- Score
- 16.1
- Wallet spend
- $3.72 of $200
- Model cost
- $23.90
- Agent time
- 2h 44m
- Steps
- 1,542
- Tokens in / out
- 214.7M / 523k
- Self-checks
- 1 passed of 4 used
- Rank at $200 wallet
- 14th of 14
Score
16.1raw metric 0.1609
The score is the raw metric (micro action F1), which already runs from 0 to 1, times 100.
Verifier details
- metric
- micro_action_f1
- tp
- 149
- fp
- 784
- fn
- 770
- precision
- 0.1597
- recall
- 0.1621
- failed_clip_limit_percent
- 5
- dropped_actions
- 0
- elapsed_seconds
- 458.0149
- deadline_seconds
- 3600
- request_timeout_seconds
- 300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenancearchitecture.json says the training data is the public p-doom/AGI-CAST-0.6k screencasts paired with labels from p-doom/A
architecture.json says the training data is the public p-doom/AGI-CAST-0.6k screencasts paired with labels from p-doom/AGI-CAST-idm-actions. The trajectory backs this up: the participant downloaded chunks.parquet (trajectory#134) and built the training set from 7 public AGI-CAST videos (trajectory#248). data.py and train.py read <data_dir>/train/train.jsonl, and train.log reports 2137 training clips. The AGI-CAST-idm-actions labels were generated by p-doom/idm, but they are a public corpus used as data, not as model weights, so the policy allows them. The only contact with p-doom/idm-eval was a metadata API call for tags and cardData (trajectory#130); no clips or labels were downloaded from it. I found no access to inductionlabs/frontier-ml-tasks. The /task/public example was used only for testing.
✓non specialist initializationmerge_and_save.py loads the general-purpose Qwen/Qwen3-VL-2B-Instruct base, applies a LoRA (r=16, alpha=32) from checkpo
merge_and_save.py loads the general-purpose Qwen/Qwen3-VL-2B-Instruct base, applies a LoRA (r=16, alpha=32) from checkpoint.pt, then calls merge_and_unload and saves. train.log shows the base loaded from the Hugging Face cache snapshot 89644892..., matching the revision declared in architecture.json. It reports 24.6M trainable parameters out of 2.15B, which fits a LoRA fine-tune. model/config.json (Qwen3VLForConditionalGeneration, 28 layers, hidden size 2048, vision depth 24) and the tensor inventory of model/model.safetensors both match standard Qwen3-VL-2B. p-doom/idm appeared only in a model search listing and was never downloaded or loaded as weights.
✓actual inference uses reviewed weightsserve.sh installs wheels offline and then runs exec python3 server.py. server.py loads ./model with AutoModelForImageTex
serve.sh installs wheels offline and then runs exec python3 server.py. server.py loads ./model with AutoModelForImageTextToText and AutoProcessor. For each request it decodes the uploaded video (PyAV, with a cv2 fallback) at 5 fps and runs MODEL.generate locally on sliding windows of 10 frames. It then parses, deduplicates and sorts the predicted actions. The request id is used only for logging. There are no remote calls, lookup tables or label caches. The shipped model directory was downloaded from the merged training output (trajectory#191).
✓harness credentials used only by the harnessThe harness API key shows up only in process listings of the harness's own litellm proxy. I found no participant code or
The harness API key shows up only in process listings of the harness's own litellm proxy. I found no participant code or commands that read or used the key, called chat/completions endpoints, or started extra LLM CLI sessions. server.py and the training code make no external API calls.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed1h 16m into the runtook 8m 8s
startup 81.7 s of 300 s; failed POST /actions: RuntimeError: timeout: attempt 1 timed out: RuntimeError: Submission HTTP request failed: TimeoutError: timed out; The public clip did not answer within grading's 300 s per-clip ceiling (cut after 300.8 s), which grading's first held-out clip ha
3 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 81.7 s
- ✗POST /actions · RuntimeError: timeout: attempt 1 timed out: RuntimeError: Submission HTTP request failed: TimeoutError: timed out
- 2failed1h 36m into the runtook 7m 57s
startup 80.7 s of 300 s; failed POST /actions: RuntimeError: timeout: attempt 1 timed out: RuntimeError: Submission HTTP request failed: TimeoutError: timed out; The public clip did not answer within grading's 300 s per-clip ceiling (cut after 300.1 s), which grading's first held-out clip ha
3 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 80.7 s
- ✗POST /actions · RuntimeError: timeout: attempt 1 timed out: RuntimeError: Submission HTTP request failed: TimeoutError: timed out
- 3failed2h 5m into the runtook 7m 46s
startup 65.4 s of 300 s; failed POST /actions: RuntimeError: timeout: attempt 1 timed out: RuntimeError: Submission HTTP request failed: TimeoutError: timed out; The public clip did not answer within grading's 300 s per-clip ceiling (cut after 300.2 s), which grading's first held-out clip ha
3 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 65.4 s
- ✗POST /actions · RuntimeError: timeout: attempt 1 timed out: RuntimeError: Submission HTTP request failed: TimeoutError: timed out
- 4passed2h 39m into the runtook 5m 23s
startup 66.3 s of 300 s; 3/3 gates passed; Grading's 43-clip batch was sampled: 8 copies of the public clip in their 670 s share of the 3600 s deadline; its time and failed clips were projected from them.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 66.3 s
- ✓POST /actions · {"actions": 20, "dropped_actions": 0, "warning": null, "first": [{"frame": "F00", "type": "MouseClick", "details": "Left"}, {"frame": "F01", "type": "MouseClick", "details": "Left"}, {"frame": "F02", "type": "MouseClick", "details": "Left"}]}
- ✓slowest request · 9 clips one at a time, each from its first attempt to its last (retries included), median 9.20 s, the first (cold) 11.33 s
- ✓43 clips one at a time · grading's request loop (validate.py run()) on 8 copies of the public clip: 73.6 s on the server the smoke request warmed up, 395.7 s for 43 (projected from 8 of 43 requests at concurrency 1), plus 2.1 s the cold first clip took over the median; held-out clips differ
- ✓failed clips · none of 8 (0 needed a retry)
- ✓healthy after requests
The agent's last message before its session ended.
Submission Completed
The IDM model training, packaging, and offline serving infrastructure have been completed, verified via frontier selfcheck, and officially submitted via frontier submit --confirm.
Verification & Performance Highlights
In the final self-check run (ba71fb0db9b0) on the target NVIDIA H100 80GB HBM3 evaluation hardware:
- Review Prerequisites: PASS
- Startup Time: 66.3 s (ceiling: 300 s, 78% margin)
POST /actionsPer-Clip Latency: 8.32 s – 11.33 s (median 9.20 s, ceiling: 300 s, 96% margin)- Full Batch Projected Latency (43 clips): 397.8 s (ceiling: 3600 s, 89% margin)
- Dropped Actions & Failures: 0 dropped actions, 0 failed clips (100% margin)
- Evaluation Health Check: PASS (Healthy after all evaluation requests)
Solution Artifacts Summary
All files are located in /workspace/output/ (copied to /submission during evaluation):
- serve.sh: Executable startup script supporting
HOST,PORT, and offline installation of vendored cp312 wheels. - server.py: Fast inference server implementing
GET /healthandPOST /actionswithFastPatchEmbedProj, direct packet decoding, and sliding-window event aggregation. - architecture.json: Complete model metadata, base commit hash, provenance, and evidence file references.
- model/: Fine-tuned and merged weights (
model.safetensors, 4.0 GB) based onQwen/Qwen3-VL-2B-Instructtrained on permitted open screencast data. - wheels/: Complete set of 67 unmodified offline cp312 wheels required for offline installation in the evaluation environment.