All tasks
Task · pokemon-random-battle
Pokémon Random Battle
Build a Pokémon Gen 9 Random Battle bot
Math & GamesInspired by
Jaxcalibur
Grading pipeline
Environments
- Entry
- Harbor → tests/test.sh → grade.py → frontier_ml.grading.grade
- Verifier
- 12.8 h · 16 CPU · 16 GiB
- Agent
- 4 h · 2 CPU · 8 GiB
- Network
- public (trusted verifier) · offline (candidate inference)
- Env
- MODAL_TOKEN_IDMODAL_TOKEN_SECRETMODAL_ENVIRONMENTDAYTONA_API_KEYFML_GRADING_RESOURCESFML_GRADING_CREDENTIALS_SECRETOPENAI_API_KEYOPENROUTER_API_KEYHF_TOKENANTHROPIC_API_KEYFML_REVIEW_API_KEYFML_REVIEW_BASE_URLFML_REVIEW_MODELFML_REVIEW_BACKENDFML_REVIEW_PROVIDERFML_REVIEW_SKIPFML_REVIEW_AWS_REGIONAWS_BEARER_TOKEN_BEDROCKAWS_REGIONCODEX_API_KEY
Verifier config
- concurrency
- 1
- inference_timeout_seconds
- 43200
- require_h100
- true
- expected_samples
- 3000
- evaluation_sha256
- a131718ef14b4a75d71f21d62952deb6e661c18238eb659b44d93e5d77fa03b9
Architecture review
A trusted reviewer reads the submission and trajectory and fails it on any of these checks.
- implementation_provenanceInspect serving source, configuration, architecture.json and provenance for reused code and models. Existing public bots and pretrained policies are allowed, including unchanged or modified Foul Play. New training, randomly initialized parameters and learned weights are not required. A search-only or rule-only bot with no checkpoint is valid; base_model and base_revision may both be none, with reused code repositories and revisions declared separately. For pretrained weights, declare the model and revision. If training or fine-tuning is performed, inspect its source, data provenance and readable logs for consistency with the shipped weights. Teacher demonstrations and public human data are allowed.
- actual_served_policyTrace serve.sh through POST /act, observation handling and every action-selection path. Decisions must come from the declared, shipped implementation operating offline on the supplied observation. Unchanged Foul Play, modified existing bots, heuristic search, learned policies, ensembles and local fallback policies are allowed; no learned model or weight change is required. Any claimed model use must agree with the actual loading and serving path. Reject remote decision services and undeclared decision paths. Do not reject a submission merely because it reuses an existing bot without training.
- permitted_informationOnly supplied player-visible observations and public game mechanics may drive inference. No simulator process inspection, opponent private state/RNG access, network service, private evaluation acquisition, hard-coded test detection or cached evaluation actions. Public development simulation and validation-guided model selection are allowed. Inspect training and serving paths.