Pokémon Random Battle
- Score
- 0.0
- Wallet spend
- $0.00 of $50
- Model cost
- $8.56
- Agent time
- 44m
- Steps
- 598
- Tokens in / out
- 76.6M / 257k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 6th of 14
Score
0.0raw metric 0.3377
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1013
- draws
- 0
- losses
- 1987
- capped_games
- 0
- opponent_forfeits
- 0
- move_timeout_forfeits
- 0
- paired_standard_error
- 0.0082
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenancearchitecture.json says there is no base model. It declares reused Foul Play code (revision 6c467c08…) and poke-engine 0.
architecture.json says there is no base model. It declares reused Foul Play code (revision 6c467c08…) and poke-engine 0.0.48, giving the wheel's SHA-256. That hash matches the trusted organizer audit of the poke-engine wheel. No weights ship: the weight inventory and uninspected-file lists are empty, and no training happened. The session shows where everything came from. At trajectory#209 or nearby (the excerpt in the steps-192-226 chunk), fp/ was copied from the task-provided foul-play.tar.gz, and foul-play-priors.json and foul_play_worker.py were copied from /task/public. Server-side edits (trajectory#501, trajectory#546) only change how state is rebuilt from the observation, which is allowed as a modified existing bot. The starter train.py was looked at but not used for the shipped bot.
✓actual served policyserve.sh installs the vendored wheels offline and runs server.py. In server.py, POST /act sends each observation to deci
serve.sh installs the vendored wheels offline and runs server.py. In server.py, POST /act sends each observation to decide_single_observation, either in-process or through a local ProcessPoolExecutor. That function rebuilds a Foul Play Battle from public_log and the request, then picks an action in one of three ways: forced_decision for single-option menus, revival_decision (local poke-engine MCTS) for Revival Blessing, or sampled battles via prepare_random_battles followed by local poke_engine monte_carlo_tree_search. Its local fallback is legal index 0. Every path matches the declared Foul Play plus poke-engine search, runs offline, and makes no remote calls.
✓permitted informationInference uses only the observation fields player, request, public_log, turn and legal_actions, plus local public data:
Inference uses only the observation fields player, request, public_log, turn and legal_actions, plus local public data: the Foul Play dex, mods and the task-provided random-battle set priors. Opponent sets are sampled from public priors. Nothing inspects the simulator, reads private evaluation files, detects test cases or caches actions. The network fetchers in fp/data/sets are skipped because server.py loads the priors directly from the local JSON. validation.jsonl was used only to run development evaluations, which is allowed.
✓harness credentials used only by the harnessThe trajectory has no references to API_KEY or *_BASE_URL environment variables, no printenv, and no model-API calls fro
The trajectory has no references to API_KEY or *_BASE_URL environment variables, no printenv, and no model-API calls from the participant's code. The only curl commands hit the local /health endpoint (e.g. trajectory#224, trajectory#552). The participant launched no extra harness CLI sessions. The single web search (trajectory#160) was the harness's own tool, and Spin was used only for budget and help commands.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed42m 31s into the runtook 2m 1s
startup 1.7 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.7 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.41 s round trip on this side with the transport: [5, 7, 3, 7, 5, 5, 7, 11, 10, 11, 7, 0, 7, 11, 7, 5, 9, 11, 0, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 6, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 90 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3583 observations, 24-32 per batch, up to 330 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 330 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.116 s, p95 0.132 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.117 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
Summary of Completed Work
-
Offline Dependency Vendoring & Auditing:
- Vendored all required runtime wheels into
/workspace/output/wheels/. - Verified the task-specific audited organizer
poke_engine-0.0.48-cp312-cp312-linux_x86_64.whlwith exact SHA-256 hashb91209b2738c06dc14c7b37cf592574a6262bc4001af6349aee3323d35481747. - Vendored
certifi,charset-normalizer,idna,python-dateutil,requests,six,urllib3, andwebsocketsto ensure 100% offline startup in/tmp/bot-depswithout internet access.
- Vendored all required runtime wheels into
-
Stateless Server Architecture & Protocol Engine:
- Created
/workspace/output/serve.shto handle offline wheel installation to/tmp/bot-depsand execute the server under the unprivileged container user. - Built
/workspace/output/server.pyproviding:- Health endpoint (
GET /health) returning 200 within 1.7 seconds (300-second deadline). - Decision endpoint (
POST /act) supporting batches up to 32 observations under the 1.0s response limit. - Accurate stateless battle state reconstruction: correctly infers opening lead Pokémon, skips only the initial switch in the protocol log, restores zeroed
max_hpfor fainted Pokémon, and synchronizes Zoroark Illusion swaps. - Optimized turn replay: bypasses expensive reverse damage check pruning during protocol replay (
update_dataset_possibilities), dropping turn reconstruction latency to <1 ms. - Multi-process worker pool (
ProcessPoolExecutor(8)viafork) utilizing all available CPU cores for concurrent MCTS search across batched requests. - Protocol and rule compliance: singleton action handling, Revival Blessing afterstate tree evaluation, and form transformation adapters.
- Health endpoint (
- Created
-
Search Tuning & Verification:
- Benchmarked MCTS iterations and batch parallelism. Configured 2 sampled battle states × 2,500 iterations per decision.
- Tested locally against the pinned Foul Play opponent via
/task/validate.py: 0 errors, 0 move timeouts, and 0 forfeits.
-
Grading Infrastructure Certification (
frontier selfcheck):- Executed
frontier selfcheck starton the official grading sandbox (NVIDIA H100 80GB HBM3):- Overall Status:
PASSED(Digestd3f76361017ea4f9d372262df8aed5e83f71d611988dda7c1c59adbfaed0d81b). - Review Prerequisites:
[pass] - Startup:
1.7 s(allowed 300 s, 99% margin). - Opening Batch: 32 opening decisions answered with legal actions (
0.41 sround-trip including network transport). - Battle Slice: 124 requests over 124 decision rounds across 32 concurrent games (3,583 observations, batches up to 32); every answered action was legal.
- 1-Second Move Response Gate:
0.1614 sworst-case latency (allowed 1.0 s, 84% margin, median 0.116 s, p95 0.132 s). - Execution Budget:
- Overall Status:
- Executed