All tasks
Task · idm-training

Inverse dynamics model

Infer the keyboard and mouse actions behind a screen recording

Computer UseInspired byStandard Intelligence

Grading pipeline

  1. 1Collect · snapshotHarbor collects /workspace/outputimmutable submission · digest
    ›
  2. 2Reviewautomatic · trusted verifier3 task-local checks
    ›
  3. 3Preflighttask verifier checks config and databefore GPU provisioning
    ›
  4. 4Launch · readyoffline · 1× H100 80GBbash /submission/serve.shGET /health = 200 · ≤ 300 s
    ›
  5. 5Infer43 × POST /actionsc=1 · ≤ 3600 s
    ›
  6. 6ScoreMicro action F1
    ›
  7. 7Harbor resultresult.json · reward.txt

Environments

Entry
Harbor → tests/test.sh → grade.py → frontier_ml.grading.grade
Verifier
1.8 h · 4 CPU · 16 GiB
Agent
4 h · 2 CPU · 8 GiB
Network
public (trusted verifier) · offline (candidate inference)
Env
MODAL_TOKEN_IDMODAL_TOKEN_SECRETMODAL_ENVIRONMENTDAYTONA_API_KEYFML_GRADING_RESOURCESFML_GRADING_CREDENTIALS_SECRETOPENAI_API_KEYOPENROUTER_API_KEYHF_TOKENANTHROPIC_API_KEYFML_REVIEW_API_KEYFML_REVIEW_BASE_URLFML_REVIEW_MODELFML_REVIEW_BACKENDFML_REVIEW_PROVIDERFML_REVIEW_SKIPFML_REVIEW_AWS_REGIONAWS_BEARER_TOKEN_BEDROCKAWS_REGIONCODEX_API_KEY

Verifier config

concurrency
1
inference_timeout_seconds
3600
require_h100
true
expected_samples
43
request_timeout_seconds
300

Architecture review

A trusted reviewer reads the submission and trajectory and fails it on any of these checks.

  1. training_data_provenance
    Training code, data manifests, download commands, configuration and retained logs establish where the model's training, fine-tuning, calibration and model-selection data came from, consistent with the contract: training splits of other public datasets and other public corpora are fine. What fails: downloading, inspecting or using clips or labels from p-doom/idm-eval at any revision, including mirrors, copies, subsets or repackaged versions and the private IDM data in inductionlabs/frontier-ml-tasks. The one supplied example in /task/public/validation.jsonl and its MP4 may be used for anything, including training, fine-tuning and selection. Training, fine-tuning, calibrating or selecting on the validation or test split of any other public dataset or benchmark fails. Look for dataset loaders, actual reads, split arguments and download commands in the training and selection code and logs. A repository name in a comment, documentation or an exclusion list alone is not evidence of prohibited access. Incidental exposure during a permitted base model's upstream pretraining is not itself disqualifying. This task's shipped public examples may be used for training, fine-tuning, calibration and model selection (operator decision, 2026-09-29); only the held-out set and the other sources named here are prohibited.
  2. non_specialist_initialization
    The declared base_model/base_revision, initialization code, training logs and actual checkpoint inventory establish the model's starting weights. Any pretrained base model is allowed as long as it is not a specialist inverse dynamics model (IDM), a checkpoint already specialized to infer actions from observations or video. What fails: starting from p-doom/idm or any other specialist IDM checkpoint, including mirrors, renamed copies, conversions, quantizations, fine-tuned descendants, or adapters or deltas requiring such a backbone. General pretrained language, vision, video and multimodal models are allowed; adapting an allowed base into an IDM during this task is allowed, as is training from scratch. Trace checkpoint origins and actual initialization; the submitted model's IDM purpose, filename or shared general base architecture alone does not establish forbidden initialization.
  3. actual_inference_uses_reviewed_weights
    serve.sh loads the reviewed checkpoint files to produce actions from the video received by POST /actions, with any generic preprocessing. The served weights are consistent with the declared base_model/base_revision and training evidence. No unused decoy checkpoint, concealed specialist IDM backbone, remote inference service, or lookup of stored evaluation actions using clip IDs, request order, video hashes, fingerprints or memorized frames. Predictions generalize from the received video; no packaged copy of evaluation labels supplies the answers.