Pokémon Random Battle
- Score
- 1.1
- Wallet spend
- $2.65 of $10
- Model cost
- $7.08
- Agent time
- 3h 32m
- Steps
- 291
- Tokens in / out
- 115.8M / 444k
- Self-checks
- 4 passed of 4 used
- Rank at $10 wallet
- 2nd of 14
Score
1.1raw metric 0.5057
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1517
- draws
- 0
- losses
- 1483
- capped_games
- 0
- opponent_forfeits
- 0
- move_timeout_forfeits
- 0
- paired_standard_error
- 0.0083
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenancearchitecture.json says this is a search-only bot with no learned weights (base_model and base_revision are both none). I
architecture.json says this is a search-only bot with no learned weights (base_model and base_revision are both none). It declares the reused code: Foul Play at revision 6c467c08 (GPL-3.0), the poke-engine 0.0.48 wheel, the organizer's adapter functions and pinned PyPI wheels. Trusted audits approve every shipped wheel, including the organizer-built poke-engine wheel. No weight inventories or uninspected files exist. The set-prior data file data/foul-play-priors.json is declared and was generated from the pinned Showdown generator with data/foul_play_priors.js; data/priors_regeneration.log records a byte-identical regeneration. The trajectory shows the file was copied from /task/public (trajectory#29-39 chunk) and later regenerated (steps-264-291 chunk). No training was performed. Config tuning used the participant's own development seeds, as declared.
✓actual served policyserve.sh installs the bundled wheels offline (--no-index) and runs server.py. On POST /act, server.py calls Dispatcher.a
serve.sh installs the bundled wheels offline (--no-index) and runs server.py. On POST /act, server.py calls Dispatcher.act in bot/pool.py, which sends observations to forked workers (bot/worker.py). Each worker rebuilds or advances a Foul Play Battle from the observation's public_log and private request, then bot/core.choose picks the move: Foul Play's prepare_random_battles samples opponent teams, poke-engine MCTS (get_result_from_mcts) searches each sample, and the choice is the highest aggregated visit share. If no search result maps to a legal action, a deterministic highest-base-power heuristic is used, or index 0 as a last resort. Priors are loaded from the local JSON, which skips Foul Play's remote set download. No remote decision service or undeclared path exists, which matches the declared design.
✓permitted informationInference uses only the observation fields player, public_log, request, legal_actions and turn, plus public game data: t
Inference uses only the observation fields player, public_log, request, legal_actions and turn, plus public game data: the moves and pokedex JSON and set priors from the public random-battle generator. Nothing inspects the simulator, reads opponent private state or RNG, makes network calls on the serving path, detects test cases or caches evaluation actions. The per-game cache holds only battle states rebuilt from the public log and is checked against the log prefix. Tuning used self-generated development schedules; NOTES.md and the trajectory say validation.jsonl was not used for selection.
✓harness credentials used only by the harnessTrajectory searches for API_KEY, BASE_URL, OPENROUTER, anthropic, printenv, environ and harness CLI launches found no us
Trajectory searches for API_KEY, BASE_URL, OPENROUTER, anthropic, printenv, environ and harness CLI launches found no use, echo or copy of harness credentials. The only matches are the Spin documentation text in the task prompt (trajectory#1) and spin compute commands that upload bot source and configs to Spin VMs (e.g. steps 57-77, 96-115). That is allowed Spin usage. The only curl calls hit PyPI reachability and local /health endpoints.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed32m 3s into the runtook 2m 28s
startup 1.7 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.7 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.85 s round trip on this side with the transport: [1, 7, 3, 7, 5, 5, 7, 11, 10, 11, 7, 0, 7, 11, 1, 11, 9, 11, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 6, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 91 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3554 observations, 24-32 per batch, up to 309 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 309 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.726 s, p95 0.735 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.728 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 2passed2h 13m into the runtook 2m 30s
startup 1.4 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.4 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.92 s round trip on this side with the transport: [5, 7, 3, 7, 5, 5, 7, 11, 9, 9, 7, 0, 9, 11, 1, 11, 9, 11, 6, 7, 5, 11, 11, 9, 0, 11, 5, 0, 5, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 85 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3533 observations, 23-32 per batch, up to 322 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (23-32 observations, up to 322 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.724 s, p95 0.730 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.724 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 3passed3h 12m into the runtook 4m 4s
startup 2.6 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.6 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 1.29 s round trip on this side with the transport: [5, 7, 3, 7, 5, 5, 7, 11, 10, 11, 7, 0, 9, 11, 1, 11, 9, 11, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 84 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3564 observations, 24-32 per batch, up to 364 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 364 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.726 s, p95 0.735 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.727 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 4passed3h 30m into the runtook 2m 30s
startup 1.8 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.8 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.83 s round trip on this side with the transport: [1, 7, 3, 7, 5, 5, 7, 11, 9, 9, 7, 0, 9, 11, 1, 5, 9, 9, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 6, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 83 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3532 observations, 23-32 per batch, up to 322 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (23-32 observations, up to 322 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.725 s, p95 0.731 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.726 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in (09:23 UTC). The bot serves Foul Play's search path, and measured against the pinned Foul Play it is at parity, about 0.498 raw points over 2400 games. That is zero reward, since the reward only starts above 50%.
What was submitted (/workspace/output, 86 files, 7.2 MB)
- The Foul Play
fppackage, vendored at the pinned revision with one behaviour-preserving edit (adefaultdictfactory changed so battle states pickle between processes). - The organizer's poke-engine wheel and build receipt, plus unmodified PyPI wheels for Foul Play's runtime dependencies. The three adapter functions are copied verbatim from the organizer's worker.
- Eight persistent worker processes, each running the upstream sampled-state Monte Carlo search, with the visit-share argmax and the upstream early-game rule. A per-game state cache in the parent is always checked against the complete log prefix, so answers depend only on the request.
- A deadline design: workers stop starting searches at 0.72 s, and the server answers by 0.86 s no matter what the workers are doing. Any unmappable choice falls back to a legal heuristic.
- No learned weights.
architecture.jsondeclaresbase_model: none. The set-prior file was regenerated from the task's generator against the pinned Showdown runtime and matches byte for byte.
Validation
- Four selfchecks passed; the last one ran on the exact submitted folder (digest
eb7dc8d3…). The H100 measured startup 1.8 s, slowest in-sandbox response 0.751 s (median 0.725 s), and a projected execution total of 2702 s against the 18000 s limit. - Development games: parity with Foul Play. Pooled over 2400 games on development seeds 101–124, the score is 0.498 (95% interval about 0.478 to 0.518).
- No late responses, decision exceptions, opponent forfeits, or capped games in the 2400 shipped-config games beyond one 200-turn game.
What did not help
- Value-based ranking, sample count (4×50 ms, 8×25 ms, one 160 ms sample), search time (0.80 s deadline), turning the early-game rule off, generator-weighted belief sampling, the expectiminimax search, and banning Protect or hazard moves all landed within noise of the shipped setting or were worse.
- Banning voluntary switches (−16.5 points) and banning Terastallization (−6.5) were clearly worse, which shows those decisions carry real value. They are not levers.
Things to know
- Development measurements came from rented 16- and 32-vCPU machines running 8 Foul Play workers and 8 candidate workers, not the grader's 16-worker layout. Foul Play's search is time-based per state, so the per-decision budgets are unchanged, but this is a deviation from the grader's setup.
- The organizer's opponent worker crashed once during development on a Terapagos Stellar form change (an assertion in Foul Play's form refresh). That is an organizer-side issue, not the candidate's, but it can end an evaluation run.
- I did not run
validate.py. Its fresh-seed games were not used fo