Pokémon Random Battle
- Score
- 0.1
- Wallet spend
- $1.30 of $10
- Model cost
- $35.45
- Agent time
- 3h 30m
- Steps
- 238
- Tokens in / out
- 69.8M / 257k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 3rd of 14
Score
0.1raw metric 0.5007
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1502
- draws
- 0
- losses
- 1498
- capped_games
- 1
- opponent_forfeits
- 0
- move_timeout_forfeits
- 0
- paired_standard_error
- 0.0085
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenanceThe submission has no learned weights and no training step, and architecture.json declares base_model and base_revision
The submission has no learned weights and no training step, and architecture.json declares base_model and base_revision as "none". The reused code is declared with its sources. Foul Play is vendored unmodified at revision 6c467c08…; the three organizer adapter fixes are applied at runtime by bot/fpstate.py. The poke-engine 0.0.48 wheel is the organizer build copied from /task/public/engine. Its SHA-256 (b91209b2…) matches data/engine-build.json and the trusted audit approves it, and serve.sh checks the hash at startup. The set priors are the organizer file /task/public/foul-play-priors.json. The trajectory (steps 35–47) shows the dependency wheels came from PyPI and the engine wheel was copied from /task/public/engine, which matches what was declared.
✓actual served policyserve.sh checks the wheel hash, installs the vendored wheels offline into /tmp/bot-deps, then runs exec python bot/serve
serve.sh checks the wheel hash, installs the vendored wheels offline into /tmp/bot-deps, then runs exec python bot/server.py. In bot/server.py, POST /act calls decide(). decide() sends work to forked workers in bot/worker.py, which run search_policy: the battle state is rebuilt with bot/fpstate.py, converted with State.from_string, then searched with poke-engine monte_carlo_tree_search. Results are mapped onto the legal actions and picked by choose_action. If search misses the deadline, the bot falls back to the heuristic scorer in bot/heuristic.py, with a legal-index safety net behind that. No remote model or decision service is called. The served policy matches the declared search approach.
✓permitted informationAt decision time the bot uses only obs["request"] and obs["public_log"] (build_battle and extend_battle in bot/fpstate.p
At decision time the bot uses only obs["request"] and obs["public_log"] (build_battle and extend_battle in bot/fpstate.py) plus the local organizer priors in data/foul-play-priors.json. make_mode loads the priors locally, which bypasses Foul Play's remote set fetch. The remote fetch in foul-play/fp/data/sets/randbats.py runs only in initialize, _load_raw_sets and start_battle, and the serving path calls none of them. Development and evaluation used dev-split schedules only (dev/make_schedule.py calls schedule(n, "dev"); dev/farm_eval.py evaluates against the pinned Foul Play). No held-out data, hidden opponent sets or grader internals are used.
✓harness credentials used only by the harnessThe trajectory shows no use, echo or copy of harness credentials (ANTHROPIC, OPENAI or other *_API_KEY / *_BASE_URL valu
The trajectory shows no use, echo or copy of harness credentials (ANTHROPIC, OPENAI or other *_API_KEY / *_BASE_URL values), and no claude or codex sessions were launched. The Spin commands are the participant's own compute and storage allowance. These include spin compute, spin s5cmd and the wallet transfer-credential help in step 56. The other hits are only the validator's MLE_JUDGE_API_KEY help text and a mention of HF_TOKEN in step 1; neither shows any credential being used.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed27m 56s into the runtook 3m
startup 1.5 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.5 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 1.01 s round trip on this side with the transport: [1, 7, 3, 7, 5, 0, 7, 2, 9, 11, 7, 0, 7, 11, 1, 11, 1, 11, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 8, 10]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 85 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3545 observations, 23-32 per batch, up to 340 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (23-32 observations, up to 340 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.794 s, p95 0.816 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.797 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 2passed2h 29m into the runtook 2m 58s
startup 1.8 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.8 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.95 s round trip on this side with the transport: [5, 7, 3, 7, 0, 5, 7, 11, 10, 9, 7, 0, 9, 11, 2, 9, 9, 9, 5, 7, 5, 11, 11, 9, 0, 11, 5, 3, 5, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 84 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3548 observations, 24-32 per batch, up to 326 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 326 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.702 s, p95 0.719 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.706 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 3passed2h 42m into the runtook 3m 6s
startup 1.6 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.6 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.98 s round trip on this side with the transport: [1, 7, 3, 7, 5, 5, 7, 4, 10, 9, 7, 0, 7, 11, 1, 5, 9, 11, 0, 7, 5, 11, 11, 9, 0, 11, 5, 3, 0, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 79 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3562 observations, 24-32 per batch, up to 344 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 344 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.703 s, p95 0.708 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.703 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
Compute is cheap enough that I can rent a 16+ core box later for large-scale evaluation. Now I'll set up the Foul Play environment locally, extract its source, and read the simulator.