Pokémon Random Battle
- Score
- 11.9
- Wallet spend
- $25.37 of $50
- Model cost
- $35.14
- Agent time
- 6h 10m
- Steps
- 497
- Tokens in / out
- 50.3M / 535k
- Self-checks
- 3 passed of 4 used
- Rank at $50 wallet
- 1st of 14
Score
11.9raw metric 0.5597
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1679
- draws
- 0
- losses
- 1321
- capped_games
- 1
- opponent_forfeits
- 0
- move_timeout_forfeits
- 0
- paired_standard_error
- 0.0083
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenancearchitecture.json and PROVENANCE.md declare a search bot with no checkpoint (base_model/base_revision none). It reuses F
architecture.json and PROVENANCE.md declare a search bot with no checkpoint (base_model/base_revision none). It reuses Foul Play at pinned commit 6c467c0 (GPL-3.0, vendored with only tests removed) and poke-engine 0.0.48 rebuilt from the organizer's source tarball with one change: the UCB1 exploration constant goes from 2.0 to 0.6 (engine/poke-engine-0.0.48-explore.patch). The trajectory backs this up. Step 395 extracts the organizer tarball. Step 396 applies only the 0.6 sed edit, and the diff shows that single line, then builds. Step 406 shows the result has sha256 a78daf8d…, the same digest declared in wheels/engine-build.json and listed in the uninspected-file record for the shipped wheel. In step 398 the clean wheel exposes only the stock API, and in step 407 hasattr(p, 'set_param') returns False. So the earlier experimental engine builds with learned leaf evaluation and set_param/set_eval_weights (steps 223-304, 422-444) did not ship. No model is trained and no weights ship. One small inaccuracy: the claim that no validation split was used is contradicted by step 16-31, where the first 4 lines of public validation.jsonl were used for development runs. The policy allows validation-guided selection, so this is not a violation. The priors file is declared as the organizer's public file, copied unchanged.
✓actual served policyserve.sh installs the vendored wheels offline (pip --no-index) and starts bot/server.py. POST /act passes each observati
serve.sh installs the vendored wheels offline (pip --no-index) and starts bot/server.py. POST /act passes each observation batch to Pool.decide. Each observation is parsed with Foul Play's parser from the request plus the public log (core.sync). A single legal action is chosen directly. Otherwise Foul Play's sampler draws opponent worlds, which are converted to poke-engine states and searched locally with monte_carlo_tree_search in forked workers. The move with the largest summed visit share is picked and mapped back to the legal_actions index. A local random legal-move fallback covers errors and timeouts. configure_engine's optional BOT_ENGINE_PARAMS/BOT_EVAL_WEIGHTS hooks run only when those environment variables are set, and serve.sh sets neither. The shipped wheel also lacks those functions, so this is dead code, and no weight file ships. There is no remote decision service. Everything matches the declared Foul Play plus MCTS design.
✓permitted informationInference reads only obs['request'], obs['public_log'], obs['player'], obs['turn'] and obs['legal_actions'], plus the pu
Inference reads only obs['request'], obs['public_log'], obs['player'], obs['turn'] and obs['legal_actions'], plus the public organizer priors file and public game data in vendored Foul Play. The game key is a hash of the player's own team. The server makes no network calls, does not inspect the simulator or read opponent private state or RNG, has no test detection and no cached actions. STATE_LOG is a development-only option that writes to a log file and does not drive decisions. Development games used self-generated teams and seeds plus the public validation file, which the policy allows.
✓harness credentials used only by the harnessSearches of the trajectory for API_KEY, BASE_URL, anthropic, openai, OPENROUTER, codex, claude CLI invocations and print
Searches of the trajectory for API_KEY, BASE_URL, anthropic, openai, OPENROUTER, codex, claude CLI invocations and printenv found only Spin CLI documentation text (SPIN_API_KEY/SPIN_BASE_URL). Remote work used the participant's Spin compute machines, which is allowed. No participant command called a model API with harness credentials, copied them, or launched extra harness sessions.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed36m 29s into the runtook 2m 24s
startup 1.3 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.3 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.91 s round trip on this side with the transport: [5, 7, 3, 7, 0, 0, 7, 11, 10, 11, 7, 0, 9, 11, 1, 5, 9, 9, 5, 7, 11, 11, 11, 9, 0, 11, 5, 0, 9, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 85 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3576 observations, 24-32 per batch, up to 323 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 323 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.806 s, p95 0.813 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.802 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 2passed2h 58m into the runtook 2m 22s
startup 1.5 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.5 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.85 s round trip on this side with the transport: [1, 7, 3, 7, 0, 5, 7, 11, 10, 9, 7, 0, 9, 11, 1, 11, 9, 11, 0, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 6, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 86 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3558 observations, 25-32 per batch, up to 338 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (25-32 observations, up to 338 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.754 s, p95 0.766 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.755 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 3failed5h 57m into the runtook 6m 11s
startup 2.1 s of 300 s; failed POST /act, round 99: ConnectionError: [Errno 110] Connect call failed ('34.155.186.142', 443)
4 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.1 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 1.04 s round trip on this side with the transport: [1, 7, 2, 7, 5, 5, 7, 11, 10, 11, 7, 0, 9, 11, 1, 11, 9, 11, 6, 7, 5, 11, 11, 9, 0, 11, 5, 3, 9, 11, 8, 9]
- ✗POST /act, round 99 · ConnectionError: [Errno 110] Connect call failed ('34.155.186.142', 443)
- 4passed6h 5m into the runtook 2m 41s
startup 1.4 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.4 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.93 s round trip on this side with the transport: [1, 7, 3, 7, 5, 5, 7, 11, 10, 11, 7, 0, 7, 11, 1, 5, 9, 11, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 5, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 78 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3545 observations, 24-32 per batch, up to 335 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 335 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.752 s, p95 0.764 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.753 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
I submitted the bot at 05:18 UTC. It is unmodified Foul Play with one change to the search engine: poke-engine's UCB1 exploration constant is now 0.6 instead of 2.0. No model is trained. The final package passed frontier selfcheck: startup 1.4 s, slowest response 0.768 s of the 1 s limit, and all actions legal.
In my own development games against the pinned Foul Play, the score was about 0.57 over ~3600 games, versus about 0.50 with the stock constant of 2.0. That corresponds to a reward of roughly 0.14. These are my own estimates, so the graded result may differ.
The 0.6 value sits on a flat plateau, with arms between about 0.35 and 1.0 scoring 0.54–0.57. Every other change I tried this session scored the same or lower, so I did not adopt any:
- different exploration constants at the root or for the opponent side;
- more or fewer search worlds, and different screening and time-allocation settings;
- other Terastallization penalties and sigmoid scaling;
- halving or doubling the search time.
The adaptive search stage that was already in the bot did help: switching it off scored 0.52.
One earlier selfcheck failed with a connection timeout from the grading box (the harness could not even fetch the server log). The rerun on the identical package (same digest) passed, so I treated it as an infrastructure flake. I also saw Foul Play's own opponent engine crash once in about 12,000 development games, which looks like noise.
The package includes the patched poke-engine wheel with its patch, build recipe and receipt, plus updated provenance and architecture.json.