Pokémon Random Battle
- Score
- 0.0
- Wallet spend
- $1.65 of $10
- Model cost
- $2.93
- Agent time
- 6h 18m
- Steps
- 1,248
- Tokens in / out
- 127.1M / 1.1M
- Self-checks
- 4 passed of 4 used
- Rank at $10 wallet
- 6th of 19
Score
0.0raw metric 0.469
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1407
- draws
- 0
- losses
- 1593
- capped_games
- 0
- opponent_forfeits
- 0
- move_timeout_forfeits
- 62
- paired_standard_error
- 0.0082
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenancearchitecture.json sets base_model and base_revision to none and training.performed to false. The policy allows a search-
architecture.json sets base_model and base_revision to none and training.performed to false. The policy allows a search-only bot with no checkpoint, and no weight inventories or uninspected files were shipped. The reused code is declared: Foul Play is vendored under fp/ with its GPL-3.0 LICENSE and pinned to revision 6c467c08… from the organizer archive, and its five modifications are documented. Those modifications are the adapters in bot/engine_env.py, weighted sampling in fp/search/random_battles.py and the hand-written copy in fp/battle/fastcopy.py. The poke-engine 0.0.48 wheel passed a trusted binary audit (approved), and its sha256 matches the one declared in architecture.json. The priors data comes from the organizer path /task/public/foul-play-priors.json, and the cp command is declared. The trajectory backs this up: trajectory#275-298 compares sha256 hashes of the shipped priors file against /task/public/foul-play-priors.json.
✓actual served policyserve.sh installs the bundled wheel offline (pip --no-index) and then runs server.py. Each POST /act request goes Server
serve.sh installs the bundled wheel offline (pip --no-index) and then runs server.py. Each POST /act request goes Server.decide_batch → Pool.run (sticky per-game worker processes) → bot.policy.decide. decide() rebuilds the battle from public_log and request using the vendored Foul Play protocol code, then checks for a forced decision or a revival decision. Otherwise it samples battles from the belief model (prepare_battles), converts each to a poke-engine state and runs the native monte_carlo_tree_search. It picks the playable move with the highest aggregated visit share and maps it to a legal index. The only fallbacks are local: a legal fallback move, and clamping out-of-range indices to 0. The server makes no remote decision-service calls and reads no cached evaluation actions. This matches the design declared in architecture.json.
✓permitted informationThe policy reads only the observation's public_log, the player request and legal_actions. setup_engine in bot/engine_env
The policy reads only the observation's public_log, the player request and legal_actions. setup_engine in bot/engine_env.py loads the local priors JSON straight into RandomBattleMode datasets. This bypasses the remote fetch from pkmn.github.io in fp/data/sets/randbats.py (initialize/_load_raw_sets), which the serving path never calls. I found no simulator internals, opponent private state, network services, test detection or cached evaluation actions in server.py or the bot code.
✓harness credentials used only by the harnessI searched the trajectory for model API keys, anthropic/claude -p/codex/openrouter calls, env dumping and curl usage. Th
I searched the trajectory for model API keys, anthropic/claude -p/codex/openrouter calls, env dumping and curl usage. The only API_KEY and openrouter hits are spin help text. The curl calls are localhost health checks and public package downloads on spin machines. remote_run.sh (trajectory#309) uses spin session storage credentials for s5cmd transfers, which is permitted spin usage. I found no model API calls with harness credentials, no copying of those credentials and no extra harness CLI sessions.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed2h 13m into the runtook 2m 46s
startup 1.0 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.0 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.92 s round trip on this side with the transport: [1, 7, 3, 7, 5, 5, 7, 11, 10, 11, 7, 0, 7, 11, 7, 5, 9, 5, 0, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 87 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3562 observations, 23-32 per batch, up to 321 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (23-32 observations, up to 321 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.723 s, p95 0.738 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.725 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 2passed5h 20m into the runtook 7m 9s
startup 2.2 s of 300 s; 1/1 gates passed; WARNING move response within 1 s: 51 of 124 POST /act responses (24-32 observations, up to 177 KB; the first a cold batch of 32) took longer than 1 s on the trusted in-sandbox move timer; median 0.949 s, p95 1.113 s. Grading forfeits every game in a
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.2 s
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 1509 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3653 observations, 24-32 per batch, up to 177 KB per request, every answered action legal; not scored
- ✓move response within 1 s · 51 of 124 POST /act responses (24-32 observations, up to 177 KB; the first a cold batch of 32) took longer than 1 s on the trusted in-sandbox move timer; median 0.949 s, p95 1.113 s. Grading forfeits every game in a late batch (each counts as a loss) and plays on, so this slice lost 1509 games that way. A grade sends 3,722 requests and plays whole games, whose late turns carry longer public logs
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.853 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 3passed5h 53m into the runtook 1m 52s
startup 1.2 s of 300 s; 1/1 gates passed; WARNING move response within 1 s: 1 of 124 POST /act responses (24-32 observations, up to 341 KB; the first a cold batch of 32) took longer than 1 s on the trusted in-sandbox move timer; median 0.368 s, p95 0.478 s. Grading forfeits every game in a l
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.2 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.55 s round trip on this side with the transport: [1, 7, 2, 7, 5, 5, 7, 11, 10, 9, 7, 0, 7, 11, 1, 5, 9, 11, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 6, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 108 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3567 observations, 24-32 per batch, up to 341 KB per request, every answered action legal; not scored
- ✓move response within 1 s · 1 of 124 POST /act responses (24-32 observations, up to 341 KB; the first a cold batch of 32) took longer than 1 s on the trusted in-sandbox move timer; median 0.368 s, p95 0.478 s. Grading forfeits every game in a late batch (each counts as a loss) and plays on, so this slice lost 27 games that way. A grade sends 3,722 requests and plays whole games, whose late turns carry longer public logs
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.381 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 4passed6h 13m into the runtook 2m 33s
startup 1.6 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.6 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.39 s round trip on this side with the transport: [5, 7, 3, 7, 5, 5, 7, 11, 10, 9, 7, 0, 9, 11, 7, 5, 9, 11, 0, 9, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 74 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3568 observations, 25-32 per batch, up to 364 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (25-32 observations, up to 364 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.409 s, p95 0.777 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.442 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.
Final state
Configuration (all in /workspace/output, submitted at 01:53:59 UTC):
| knob | value | why |
|---|---|---|
| move selection | argmax over aggregated visit shares | +6.5 points paired (z = 4.60) vs Foul Play's random-among-top draw |
| search budget | 330 ms/decision (165 ms/sample; 4×82 ms while the opponent is hidden) | measured optimum — 550 ms gave no further gain (+0.016 ± 0.014) |
| workers | 32, one per active battle | each decision gets a whole worker |
| internal deadline | 0.60 s, adaptive | bounds the response independently of the batch size |
| budget controller | shrink ×0.8 above 0.70 s, grow ×1.2 below 0.50 s | self-tunes to the grading hardware |
| warm-up / slow-rebuild rules | first request ×0.15; a decision that rebuilt a slow state ≤150 ms | protects the one batch that can overrun |
Key changes this session
- Removed the per-sample cap that had collapsed the search to ~58 ms/sample. The engine's budget is wall-clock, so a capped search leaves CPU unused — with the cap the bot was searching ~60 ms of CPU per decision against the opponent's ~100 ms, i.e. materially weaker than the opponent. Restored search tripled the MCTS visits (20M → 55M per decision on the H100) and puts us ~2× ahead in CPU per decision.
- Moved the overshoot reserve so it applies only to the last sample of a decision (earlier samples' overruns are already absorbed by the remaining-time computation).
- Throttled the server's stdout to ≤200 lines (a full run sends ~3,700 requests; ~550 KB of log would have filled the harness's pipe and blocked the server mid-grade) and moved the full per-request trace to
/tmp/bot_timing.log. - Added the slow-state/rebuild guard for batches of fresh battles.
Verification: final self-check df2bed444df8 PASSED — startup 1.6 s, 124/124 requests legal, median 0.409 s, p95 0.777 s, slowest 0.9916 s (the cold batch, now protected), projected 1,647 s of the 18,000 s budget. Local replay regression: 222 decisions, 0 unmapped, 0 broken.
Expected reward: the remote measurement of this configuration (argmax + 330 ms, 2048 paired games) scored 0.5591 against the pinned Foul Play → reward ≈ 0.118.