Pokémon Random Battle
- Score
- 0.0
- Wallet spend
- $0.00 of $50
- Model cost
- $1.41
- Agent time
- 1h 41m
- Steps
- 268
- Tokens in / out
- 20.8M / 66k
- Self-checks
- 1 passed of 2 used
- Rank at $50 wallet
- 7th of 19
Score
0.0raw metric 0.4613
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1384
- draws
- 0
- losses
- 1616
- capped_games
- 0
- opponent_forfeits
- 0
- move_timeout_forfeits
- 0
- paired_standard_error
- 0.0084
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenancearchitecture.json declares base_model and base_revision as "none" and says no training was done. It lists the reused cod
architecture.json declares base_model and base_revision as "none" and says no training was done. It lists the reused code as Foul Play (pmariglia/foul-play at revision 6c467c08, GPL-3.0) and poke-engine 0.0.48 (MIT). It names /task/public/foul-play-priors.json as the source of the set-frequency data. The session confirms these claims. The foulplay/ tree was unpacked from the supplied /task/public/foul-play.tar.gz (trajectory#28-31). The poke-engine wheel was copied from /task/public/engine/, and a trusted audit approves that exact SHA. The other wheels were pip-downloaded from PyPI (trajectory#259), and trusted digest audits approve them. The submission ships no weights and does no training, so no checkpoint or training log is needed. A search-only reuse of Foul Play is explicitly allowed.
✓actual served policyserve.sh installs the vendored wheels into /tmp/bot-deps and runs server.py. In server.py, POST /act checks the batch si
serve.sh installs the vendored wheels into /tmp/bot-deps and runs server.py. In server.py, POST /act checks the batch size and sends each observation to one of several single-worker ProcessPoolExecutors, picked by a hash of a game key. Each worker runs decide_stateful, which rebuilds or updates a Foul Play Battle from the observation's request and public_log. It then picks an action in one of three ways: forced_decision (only one legal action), revival_decision (a local poke-engine MCTS over Revival Blessing targets), or find_best_move_inproc (Foul Play's prepare_battles followed by local poke_engine monte_carlo_tree_search and select_move_from_mcts_results). The chosen move is mapped onto the observation's legal_actions, falling back to index 0. Every path is local and matches the declared stateful Foul Play search. RandomBattleTeamDatasets.initialize, which would fetch sets remotely, is never called because the priors are loaded from the shipped JSON. No remote decision service is involved.
✓permitted informationAt inference time the bot reads only the observation's request (the player's own private request), public_log, player an
At inference time the bot reads only the observation's request (the player's own private request), public_log, player and legal_actions, plus the shipped public priors and Foul Play mechanics data. The per-game cache only holds the battle state rebuilt from earlier observations of the same game, keyed on the first public log lines. It does not detect test cases or store evaluation actions. The source contains no /proc or simulator inspection, no sockets, and no reads of /task files other than a fallback path to the public priors file. The /proc and process-listing commands in the session were for checking the local dev server's port (trajectory#246-247) and are not part of serving. Validation runs on the public data were local development simulation (trajectory#25).
✓harness credentials used only by the harnessSearching the trajectory found no use of API_KEY or *_BASE_URL variables, no model-API calls from participant code, and
Searching the trajectory found no use of API_KEY or *_BASE_URL variables, no model-API calls from participant code, and no launches of extra harness CLIs. The only harness proxy and base URL text appears in ps aux output (trajectory#247), which only lists processes and was not copied or used. The curl commands only probed the local server's /health endpoint.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed1h 13m into the runtook 19m 26s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: Traceback (most recent call last): File "/submission/server.py", line 23, in <module> from f
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: Traceback (most recent call last): File "/submission/server.py", line 23, in <module> from fp.modes.random_battle import RandomBattleMode File "/submission/foulplay/fp/modes/__init__.py", line 2, in <module> from fp.modes.base import BattleMode File "/submission/foulplay/fp/modes/base.py", line 12, in <module> from fp.data.sets import SmogonSets File "/submission/foulplay/fp/data/sets/__init__.py", line 1, in <module> from fp.data.sets.base import ( File "/submission/foulplay/fp/data/sets/base.py", line 11, in <module> import requests ModuleNotFoundError: No module named 'requests'
- 2passed1h 34m into the runtook 5m 15s
startup 1.6 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.6 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 0.94 s round trip on this side with the transport: [1, 7, 3, 7, 5, 5, 7, 11, 11, 9, 7, 0, 9, 8, 1, 5, 9, 7, 6, 9, 11, 11, 11, 9, 0, 11, 5, 0, 7, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 82 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3545 observations, 23-32 per batch, up to 337 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (23-32 observations, up to 337 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.509 s, p95 0.628 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.507 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests