Pokémon Random Battle
- Score
- 0.0
- Wallet spend
- $3.98 of $10
- Model cost
- $1.12
- Agent time
- 3h 20m
- Steps
- 569
- Tokens in / out
- 53.0M / 402k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 6th of 19
Score
0.0raw metric 0.485
The raw metric (mean battle points against Foul Play) is rescaled so that 50% of battle points against Foul Play scores 0 and 100% scores 100, clamped to that range.
Verifier details
- metric
- mean_battle_points
- samples
- 3000
- wins
- 1455
- draws
- 0
- losses
- 1545
- capped_games
- 1
- opponent_forfeits
- 0
- move_timeout_forfeits
- 0
- paired_standard_error
- 0.0083
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓implementation provenancearchitecture.json sets base_model and base_revision to none and learned_weights to none. It declares the reused code sep
architecture.json sets base_model and base_revision to none and learned_weights to none. It declares the reused code separately: Foul Play at pinned commit 6c467c08 (GPL-3.0, vendored at vendor/foul_play), the organizer-built poke-engine 0.0.48 wheel, and the organizer adapter semantics from foul_play_worker.py. There are no checkpoints and no training. A trusted audit approves the poke-engine wheel and its sha256 matches the declared hash. The trajectory shows the vendored Foul Play was copied from the task tarball and checked with a recursive diff (trajectory#284-ish chunk, trajectory#492-517 chunk), whose output shows only empty cache directories added. DATA_MANIFEST.md declares the runtime prior as the task-supplied foul-play-priors.json. Tuning used local development schedules from the task's own schedule helper, as recorded in logs/calibration.md and logs/dev_schedules.md. This search-only bot is valid under the policy.
✓actual served policyserve.sh installs the shipped wheels offline (pip --no-index) and runs server.py. POST /act first computes a cheap legal
serve.sh installs the shipped wheels offline (pip --no-index) and runs server.py. POST /act first computes a cheap legal fallback for each observation with Bot.fallback_index, a linear score over the observation's action_features using weights from config.json. It then sends each observation to pre-forked workers. Each worker runs Bot.decide: it rebuilds the battle from public_log and the request using the vendored Foul Play parser, samples opponent teams with prepare_random_battles from the shipped prior, runs poke_engine.monte_carlo_tree_search, and picks a move by greedy policy mass. Any failure or timeout falls back to the same local heuristic. There are no remote calls or undeclared decision paths, and this matches the description in architecture.json and PROVENANCE.md.
✓permitted informationbot.py reads only the observation's request, public_log, player, legal_actions and action_features fields, plus the ship
bot.py reads only the observation's request, public_log, player, legal_actions and action_features fields, plus the shipped public set prior and Foul Play game data. The code does not touch the network, inspect the simulator, read opponent private state or RNG, detect tests, or keep a cache of evaluation actions. Requests are handled statelessly. Development tuning used locally generated public dev schedules played against the public Foul Play worker, which is allowed.
✓harness credentials used only by the harnessSearching the trajectory found no use of ANTHROPIC/OPENAI/OPENROUTER keys or harness base URLs in participant commands,
Searching the trajectory found no use of ANTHROPIC/OPENAI/OPENROUTER keys or harness base URLs in participant commands, no printenv or copying of credentials, and no extra claude, codex or other harness CLI sessions. The only API_KEY hits are Spin CLI help text and an argparse default in the task's own validation code. The participant used Spin compute machines, uploading public task files to them (trajectory#27, trajectory#35), which the policy allows.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed3h 17m into the runtook 3m 25s
startup 1.3 s of 300 s; 2/2 gates passed; The speed gates were measured on a sample and the execution budget projected from it: 124 of a complete grade's 3,722 POST /act requests, from the first decision rounds of the public games rather than whole games.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.3 s
- ✓POST /act, round 1 · 32 opening decisions answered with legal actions, 1.14 s round trip on this side with the transport: [1, 7, 3, 7, 5, 5, 7, 11, 10, 9, 7, 0, 7, 11, 1, 5, 9, 11, 5, 7, 5, 11, 11, 9, 0, 11, 5, 0, 9, 11, 8, 9]
- ✓battle slice · 124 POST /act batches over 124 decision rounds, 32 games at a time: the first 32 public games (teams from development seed 91827461), 85 finished games replaced by further development-seed matchups, against the pinned Foul Play: 3545 observations, 24-32 per batch, up to 343 KB per request, every answered action legal; not scored
- ✓move response within 1 s · slowest of 124 POST /act responses (24-32 observations, up to 343 KB; the first a cold batch of 32) on the trusted in-sandbox move timer; median 0.851 s, p95 0.852 s. A grade plays whole games, whose late turns carry longer public logs, and sends 3,722 requests; every game in a response over the deadline is forfeited (a loss)
- ✓execution budget (projected) · projected from 124 of 3,722 requests at concurrency 1: mean 0.851 s per request x 3,722, as many as one complete 3,000-game grade sent; grading sums every request's in-sandbox time
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. Final state:
What was handed in (/workspace/output, 108 files, 7.6 MB)
- A stateless serving stack:
serve.shinstalls the pinned wheels offline into/tmpand startsserver.py(8 supervised worker processes, 1 s response budget, target 0.88 s, per-request deadline split across queued observations). bot.pyrebuilds each game state from the request plus the fullpublic_logusing the unmodified pinned Foul Play parser, samples opponent teams from the supplied prior, and runs poke-engine MCTS over each sampled world (~6 worlds × 40 ms per decision, chance-weighted visit-share selection; tera and switch actions included). Inference is disabled after measuring that it only hurt.- All sources documented in
architecture.json,README.md,DATA_MANIFEST.md,PROVENANCE.md, andlogs/calibration.md. No training data, no public validation/test split used; the only opponent ever faced was the pinned Foul Play worker.
Validation
frontier selfcheckpassed every gate on the grading image (H100, offline): startup 1.3 s (limit 300 s), 124 batchedPOST /actrounds with all actions legal, slowest response 0.853 s (limit 1 s), projected execution budget 3,168 s (limit 18,000 s), server healthy afterward.- Development performance with the exact shipped stack: 0.484 (484 W / 516 L, se 0.0146) over a fresh 1,000-game schedule; 0.472 (se 0.019) over 600 games; best 200-game sweeps at 0.505. A state-reconstruction cross-check against a mirror of the pinned worker's incremental battle found no reconstruction defect.