Pokémon Random Battle
- Score
- 0.0
- Wallet spend
- $2.06 of $10
- Model cost
- $15.02
- Agent time
- 4h 16m
- Steps
- 368
- Tokens in / out
- 74.5M / 447k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 4th of 14
Score
0.0raw metric 0.4897
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1469
- draws
- 0
- losses
- 1531
- capped_games
- 0
- opponent_forfeits
- 0
- move_timeout_forfeits
- 0
- paired_standard_error
- 0.0083
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenancearchitecture.json says this is a search bot with no checkpoint. base_model and base_revision are both 'none'. It declare
architecture.json says this is a search bot with no checkpoint. base_model and base_revision are both 'none'. It declares the reused code separately: Foul Play at revision 6c467c08… under fp/, the organizer-built poke-engine 0.0.48 wheel, the PyPI dependency wheels, and priors.json copied from /task/public/foul-play-priors.json with its sha256. No weight inventory exists and no training was claimed or done. The trusted audit approves the shipped poke-engine wheel by sha256. In the trajectory, step 175 runs a diff of the vendored fp/ against /task/public/foul-play.tar.gz and finds only pycache/cache directories that differ. The same step shows the priors sha256 matches the declared value. The runtime adapter monkeypatches are declared in architecture.json and APPROACH.md.
✓actual served policyserve.sh installs the vendored wheels offline and runs server.py. POST /act passes each observation to decide_batch. Any
serve.sh installs the vendored wheels offline and runs server.py. POST /act passes each observation to decide_batch. Any observation with a single legal action returns index 0. Every other observation goes to forked decider.worker_main workers. These track the battle with Foul Play's parser from public_log and request, sample determinizations from the shipped priors, and run poke_engine.monte_carlo_tree_search. MCTS visit mass is mapped onto the supplied legal_actions, with a modal vote across samples. All fallbacks run locally: the tactical heuristic over action_features, or index 0. No remote decision service or undeclared path was found, and the code matches the declared search-only approach.
✓permitted informationThe served decisions use only the observation fields: player, request, public_log, legal_actions and action_features. Th
The served decisions use only the observation fields: player, request, public_log, legal_actions and action_features. They also use public mechanics data (fp/data) and the organizer priors aggregate. worker_main disables socket connect and create_connection. I found no simulator inspection, opponent private-state or RNG access, test detection or cached evaluation actions. In the trajectory, tuning used development self-play and vs-Foul-Play matches on fresh seeds. The validator was used only to check the running server.
✓harness credentials used only by the harnessI searched the trajectory for API_KEY, BASE_URL, anthropic, openrouter, claude, codex, printenv and env dumps. None of t
I searched the trajectory for API_KEY, BASE_URL, anthropic, openrouter, claude, codex, printenv and env dumps. None of the participant's commands calls a model API with harness credentials, copies or echoes those credentials, or launches another harness CLI. The only API_KEY and BASE_URL hits are in help text and validator source the participant read. Step 268 inspected a server process's environment on a rented Spin machine; the output contains no harness credentials. Spin CLI compute use is permitted.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed4h 6m into the runtook 3m 20s
startup 1.5 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.5 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 1.11 s round trip on this side with the transport: [1, 7, 3, 7, 5, 5, 7, 11, 11, 9, 7, 0, 9, 11, 1, 11, 9, 11, 0, 7, 5, 11, 11, 9, 0, 11, 5, 0, 5, 11, 6, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 79 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3566 observations, 21-32 per batch, up to 353 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (21-32 observations, up to 353 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.834 s, p95 0.840 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.833 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 12:23 UTC.
Result
Final bot: Foul Play-based search with modal-vote selection, served via POST /act from /workspace/output/.
- Strength: 279W–233L (0.545 over 512 games) vs the pinned Foul Play across 16 fresh-seed blocks, zero late batches, worst response 0.851s. Expected final reward ≈ 0.09.
- Selfcheck: PASSED on the H100 — startup 1.5s, slowest of 124 responses 0.843s (16% margin), projected budget 3099s of 18000s, all 3566 actions legal.
What won
The decisive change was modal voting: each MCTS determinization votes for its own legal argmax instead of pooling visit mass (+0.05 paired, reproduced on fresh seeds). Supporting work: 8-worker fork pool with hash-routed incremental tracking, honest per-job budgets (840/920ms), small-batch scale-up, overload shed, legal-safe mapping with tera fallback, and the organizer's adapter fixes. Dead ends killed by paired tests: sample rebalancing (−0.16), early-stop (−0.07).
No training or selection on validation/test splits — all tuning used fresh development seeds. Provenance (pinned Foul Play revision, organizer engine wheel hash, priors hash) is recorded in architecture.json / APPROACH.md.