Pokémon Random Battle
- Score
- 0.0
- Wallet spend
- $0.83 of $200
- Model cost
- $11.00
- Agent time
- 2h 50m
- Steps
- 174
- Tokens in / out
- 29.0M / 160k
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 4th of 14
Score
0.0raw metric 0.4577
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1373
- draws
- 0
- losses
- 1627
- capped_games
- 0
- opponent_forfeits
- 1
- move_timeout_forfeits
- 0
- paired_standard_error
- 0.0083
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenancearchitecture.json sets base_model and base_revision to "none". It declares the reused code separately: Foul Play at comm
architecture.json sets base_model and base_revision to "none". It declares the reused code separately: Foul Play at commit 6c467c0 (GPL-3.0, licence shipped), the poke-engine 0.0.48 wheel and the organizer adapter patterns. It also lists the changes. The only behavioural change to the vendored code is weighted sampling of hidden sets in fp/search/random_battles.py, and the code matches that description. There are no learned weights, no weight inventories and no training. The only data file, foul-play-priors.json, is task-provided (the trajectory shows it in /task/public). A trusted audit approves the poke-engine wheel as organizer-built. The trajectory shows the files being copied into the output directory, which agrees with the declared provenance.
✓actual served policyserve.sh installs the shipped wheels offline with pip --no-index and runs server.py. POST /act sends each observation to
serve.sh installs the shipped wheels offline with pip --no-index and runs server.py. POST /act sends each observation to forked local workers. Each worker rebuilds the battle with Foul Play's protocol parser (process_battle_updates/update_battle) and picks a move in one of three ways: forced_decision, revival_decision, or search_choice, which runs local poke_engine MCTS over hidden states sampled by the vendored random_battles.py. choice_to_action maps the result to a legal index. If that fails, fallback_action picks from the observation's action_features. All of these paths are local and declared. There is no remote decision service. The oracle_server.py that forwarded to foul_play_worker (trajectory#119) was a test-only dev tool and was not shipped.
✓permitted informationDecisions use only the observation fields player, request, public_log, legal_actions, turn and action_features, plus pub
Decisions use only the observation fields player, request, public_log, legal_actions, turn and action_features, plus public game data and the priors file. init_engine loads the priors locally and calls _initialize_pkmn_sets, which skips the remote randbats download in initialize(). The shipped source never references validation data, simulator processes, opponent private state or RNG, and contains no cached actions. game_key is used only to cache per-game state and route games to workers, not to detect tests. The participant ran validation and A/B runs only for development.
✓harness credentials used only by the harnessI searched the trajectory for API_KEY, BASE_URL, OPENROUTER, openai, anthropic, claude and printenv. The only hit is Spi
I searched the trajectory for API_KEY, BASE_URL, OPENROUTER, openai, anthropic, claude and printenv. The only hit is Spin CLI help text mentioning SPIN_API_KEY. The participant's commands use the Spin compute exec/upload CLI and local curl calls to the bot's own health endpoint. No step calls a model API with harness credentials, copies those credentials, or launches extra harness sessions.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 25m into the runtook 5m 20s
startup 1.9 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.9 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.94 s round trip on this side with the transport: [5, 7, 3, 7, 5, 5, 7, 11, 10, 9, 7, 0, 9, 11, 1, 5, 9, 11, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 84 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3556 observations, 23-32 per batch, up to 356 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (23-32 observations, up to 356 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.801 s, p95 0.859 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.775 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 2passed2h 43m into the runtook 4m 4s
startup 2.5 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.5 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 1.01 s round trip on this side with the transport: [5, 7, 3, 7, 5, 5, 7, 11, 9, 9, 7, 0, 9, 11, 7, 5, 9, 11, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 86 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3564 observations, 23-32 per batch, up to 323 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (23-32 observations, up to 323 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.633 s, p95 0.789 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.641 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. Summary of the final bot in /workspace/output/:
Approach — a hardened, deadline-aware derivative of the pinned Foul Play search (GPL-3.0, vendored under fp/ with the organizer poke-engine 0.0.48 wheel in wheels/), served by server.py:
- Reconstructs each battle from the private request + public log with upstream's protocol parser, samples hidden opponent states, and runs poke-engine MCTS per sample.
- Deterministic visit-weighted argmax instead of upstream's random choice among near-best moves.
- Unrevealed opponent species/sets sampled weighted by the true generator frequencies (per-set counts in the task-provided priors) instead of uniformly — a bias in upstream's sampler.
- Batch-adaptive search budget across 8 worker processes with per-game incremental state caching, least-loaded game assignment, stale-queue dropping, and a legal-move fallback so a response is never invalid or late (0.78s internal deadline vs the 1s limit).
Verification — developed and measured on a rented 16-CPU Spin VM ($0.82 of budget used, now released):
- Fixed-seed paired A/B (128 games, same teams): final package 0.523 vs upstream-mirror package 0.500 raw battle points; repeated public validations ranged 0.41–0.59 per 32 games.
- Two
frontier selfcheckpasses on the grading image; the final one: startup 2.5s/300s, worst move response 0.797s/1s (20% margin), projected total server time 2384s/18000s, 0.28% deadline fallbacks, 0 worker errors.
No learned weights were trained (base_model: "none" in architecture.json, with reused-code revisions and the two documented fp/ sampling changes recorded there and in README.md).