All tasks
Task · pokemon-random-battle

Pokémon Random Battle

Build a Pokémon Gen 9 Random Battle bot

Math & GamesInspired byJaxcalibur

Grading pipeline

  1. 1Collect · snapshotHarbor collects /workspace/outputimmutable submission · digest
    ›
  2. 2Reviewautomatic · trusted verifier3 task-local checks
    ›
  3. 3Preflighttask verifier checks config and databefore GPU provisioning
    ›
  4. 4Launch · readyoffline · 1× H100 80GBbash /submission/serve.shGET /health = 200 · ≤ 300 s
    ›
  5. 5Infer3000 × POST /actc=1 · ≤ 43200 s
    ›
  6. 6ScoreReward above 50% battle points against Foul Play
    ›
  7. 7Harbor resultresult.json · reward.txt

Environments

Entry
Harbor → tests/test.sh → grade.py → frontier_ml.grading.grade
Verifier
12.8 h · 16 CPU · 16 GiB
Agent
4 h · 2 CPU · 8 GiB
Network
public (trusted verifier) · offline (candidate inference)
Env
MODAL_TOKEN_IDMODAL_TOKEN_SECRETMODAL_ENVIRONMENTDAYTONA_API_KEYFML_GRADING_RESOURCESFML_GRADING_CREDENTIALS_SECRETOPENAI_API_KEYOPENROUTER_API_KEYHF_TOKENANTHROPIC_API_KEYFML_REVIEW_API_KEYFML_REVIEW_BASE_URLFML_REVIEW_MODELFML_REVIEW_BACKENDFML_REVIEW_PROVIDERFML_REVIEW_SKIPFML_REVIEW_AWS_REGIONAWS_BEARER_TOKEN_BEDROCKAWS_REGIONCODEX_API_KEY

Verifier config

concurrency
1
inference_timeout_seconds
43200
require_h100
true
expected_samples
3000
evaluation_sha256
a131718ef14b4a75d71f21d62952deb6e661c18238eb659b44d93e5d77fa03b9

Architecture review

A trusted reviewer reads the submission and trajectory and fails it on any of these checks.

  1. implementation_provenance
    Inspect serving source, configuration, architecture.json and provenance for reused code and models. Existing public bots and pretrained policies are allowed, including unchanged or modified Foul Play. New training, randomly initialized parameters and learned weights are not required. A search-only or rule-only bot with no checkpoint is valid; base_model and base_revision may both be none, with reused code repositories and revisions declared separately. For pretrained weights, declare the model and revision. If training or fine-tuning is performed, inspect its source, data provenance and readable logs for consistency with the shipped weights. Teacher demonstrations and public human data are allowed.
  2. actual_served_policy
    Trace serve.sh through POST /act, observation handling and every action-selection path. Decisions must come from the declared, shipped implementation operating offline on the supplied observation. Unchanged Foul Play, modified existing bots, heuristic search, learned policies, ensembles and local fallback policies are allowed; no learned model or weight change is required. Any claimed model use must agree with the actual loading and serving path. Reject remote decision services and undeclared decision paths. Do not reject a submission merely because it reuses an existing bot without training.
  3. permitted_information
    Only supplied player-visible observations and public game mechanics may drive inference. No simulator process inspection, opponent private state/RNG access, network service, private evaluation acquisition, hard-coded test detection or cached evaluation actions. Public development simulation and validation-guided model selection are allowed. Inspect training and serving paths.