Pokémon Random Battle
- Score
- 0.0
- Wallet spend
- $2.13 of $50
- Model cost
- $18.62
- Agent time
- 2h 19m
- Steps
- 388
- Tokens in / out
- 104.1M / 329k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 6th of 14
Score
0.0raw metric 0.4633
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1390
- draws
- 0
- losses
- 1610
- capped_games
- 0
- opponent_forfeits
- 0
- move_timeout_forfeits
- 115
- paired_standard_error
- 0.0086
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenanceReused code is declared in architecture.json and PROVENANCE.md: Foul Play (GPL-3.0) pinned to commit 6c467c081e862fb321a
Reused code is declared in architecture.json and PROVENANCE.md: Foul Play (GPL-3.0) pinned to commit 6c467c081e862fb321adb405355beb41aba8e226, plus the poke-engine 0.0.48 wheel. That wheel has an approved trusted audit, and the other bundled wheels have trusted binary-audit entries. No model weights are shipped: there are no safetensors files and no weight inventory, and architecture.json marks the qnet as not present. The Q-net pilots were trained from scratch on self-play data from train/gen_data2.py (logs/datagen.json, logs/train_q_*.json) and then dropped after the A/B results in logs/experiments.md. The set priors were copied from the organizer-supplied /task/public/foul-play-priors.json.
✓actual served policyserve.sh installs the bundled wheels offline (pip --no-index) and runs bot/server.py. Its /act endpoint calls policy.Bot
serve.sh installs the bundled wheels offline (pip --no-index) and runs bot/server.py. Its /act endpoint calls policy.Bot.decide_batch, which sends jobs.search_worlds / search_revival to a process pool. Those jobs sample hidden worlds from the prior and run poke_engine MCTS (or iterative deepening), and the move is chosen by greedy visit share. bot/policy.py:_fallback is the fallback, using a heuristic with fixed tactical weights. The Q-net path loads only if bot/qnet.safetensors exists; it is not shipped, so that path is inactive. Priors.json is assigned directly to raw_pkmn_sets, so the upstream remote set-fetch code in bot/fp/data/sets is not on the serving path.
✓permitted informationThe served policy reads only the observation's request, public_log, legal_actions and action_features, plus the shipped
The served policy reads only the observation's request, public_log, legal_actions and action_features, plus the shipped organizer prior (bot/priors.json). The serving code makes no network calls and reads nothing from /task or any hidden or answer data. The trajectory shows validation.jsonl used only to evaluate and select configurations, not for training, and the priors came from the organizer's public prior file.
✓harness credentials used only by the harnessI searched the trajectory for harness API keys, base URLs, provider names and env dumps, and found no participant comman
I searched the trajectory for harness API keys, base URLs, provider names and env dumps, and found no participant command that reads or uses harness credentials or starts another agent CLI. The only credentials handled are Spin-issued transfer credentials: steps 182–186 generate them and use them with s5cmd to fetch data bundles from Spin storage, which is allowed.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 12m into the runtook 2m 45s
startup 1.7 s of 300 s; 1/1 gates passed; WARNING move response within 1 s: 33 of 124 POST /act responses (25-32 observations, up to 197 KB; the first a cold batch of 32) took longer than 1 s on the trusted in-sandbox move timer; median 0.952 s, p95 1.001 s. Grading forfeits every game in a
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.7 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 1.00 s round trip on this side with the transport: [5, 7, 3, 7, 5, 0, 7, 11, 10, 9, 7, 0, 7, 11, 1, 5, 9, 11, 6, 7, 5, 11, 11, 9, 0, 11, 5, 0, 5, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 864 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3611 observations, 25-32 per batch, up to 197 KB per request, every answered action legal; not scored
- ✓move response within 1 s · 33 of 124 POST /act responses (25-32 observations, up to 197 KB; the first a cold batch of 32) took longer than 1 s on the trusted in-sandbox move timer; median 0.952 s, p95 1.001 s. Grading forfeits every game in a late batch (each counts as a loss) and plays on, so this slice lost 891 games that way. A grade sends 3,722 requests and plays whole games, whose late turns carry longer public logs
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.949 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
- 2passed2h 15m into the runtook 2m 37s
startup 1.7 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.7 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.72 s round trip on this side with the transport: [5, 7, 1, 7, 5, 0, 7, 11, 11, 9, 7, 0, 7, 11, 1, 5, 1, 11, 6, 7, 5, 11, 3, 9, 5, 11, 5, 3, 5, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 78 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3569 observations, 24-32 per batch, up to 341 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 341 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.675 s, p95 0.744 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.672 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. Final bot: greedy Foul Play-style MCTS search served offline through POST /act.
What was built (server.py, policy.py, jobs.py):
- Incremental per-game battle parsing cached by team fingerprint + public-log prefix; hidden states sampled (≤2 worlds) from the public set prior
- One process-pool job per observation (sample + native
poke-engineMCTS) with adaptive per-world budget; batches ≥20 use 1 world (proven equal strength, fits the deadline) - Greedy argmax selection instead of the opponent's randomized 75%-of-best rule; singleton menus answered directly, Revival Blessing by shared-budget search, tactical-heuristic fallback so no illegal action is ever emitted
- Serving fixes beyond upstream: own-side form-change routing (Palafin crash regression test included), pickle support for pool transport, speed-inference guard
Evidence (/workspace/output/logs/): ~1,600 self-play games vs pinned Foul Play across paired A/Bs — baseline ≈50.6%; ID-endgame, Q-value/policy blends, and 4-world breadth all tested and dropped (hurts or null); 1 world ≡ 2 worlds. Selfcheck passed: slowest batch 0.85s (median 0.68s), all actions legal, startup 1.7s.