Math proof simplification
- Score
- 0.0
- Wallet spend
- $18.06 of $200
- Model cost
- $1.58
- Agent time
- 1h 31m
- Steps
- 222
- Tokens in / out
- 21.9M / 63k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 16th of 19
Score
0.0raw metric 0
The score is the raw metric (mean proof-token reduction over proofs that still check), which already runs from 0 to 1, times 100.
Verifier details
- metric
- mean_valid_token_reduction
- samples
- 354
- valid_fraction
- 1
- shorter_fraction
- 0
- standard_error
- 0
- proof_checking_seconds
- 0.6779
- preflight_seconds
- 674.5393
- batch_budget_seconds
- 2700
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNo trained component was shipped. architecture.json sets base_model to "none", and training_log.txt says "No training pe
No trained component was shipped. architecture.json sets base_model to "none", and training_log.txt says "No training performed; system returns the provided proof unchanged." The final output holds only four files: architecture.json, serve.sh, server.py and training_log.txt. The Goedel model directory and the wheels directory were deleted before submission (trajectory#218). During development the participant downloaded Goedel-LM/Lean-workbook-proofs, a public training split (trajectory#158), but nothing shipped was trained, calibrated or selected on it. Searches of the session found no download of the pinned inductionlabs/frontier-ml-tasks repository. No validation or test data is used by any shipped component.
✓no proof lookupThe POST /simplify handler in server.py parses the request JSON and returns data["proof"] unchanged. There is no table,
The POST /simplify handler in server.py parses the request JSON and returns data["proof"] unchanged. There is no table, cache, retrieval store, network access or bundled data file. Each answer is the request's own proof, so no prepared proof is looked up.
✓actual inference pathserve.sh only installs fastapi/uvicorn if a wheels directory exists, which it does not in the final output, then runs se
serve.sh only installs fastapi/uvicorn if a wheels directory exists, which it does not in the final output, then runs server.py. server.py is a stateless ThreadingHTTPServer that echoes the input proof. No checkpoint ships, so there is no decoy checkpoint. There is also no remote service, alternate path or state carried across requests. The shipped files match what was written in trajectory#215-217, and the final listing is in trajectory#218.
✓harness credentials used only by the harnessEvery model API call in the session used the Spin OpenRouter proxy key from `spin openrouter key create` (trajectory#70,
Every model API call in the session used the Spin OpenRouter proxy key from `spin openrouter key create` (trajectory#70, #167, #186) against api.spin.inductionlabs.com. That is the participant's own allowance and is allowed. trajectory#190 listed only the SPIN_* environment variables. Searches found no harness API_KEY or BASE_URL being read or copied, and no extra claude/codex or similar CLI sessions. HF_TOKEN was used only for downloads (trajectory#77, #158, #192), which is allowed.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed1h 29m into the runtook 1m 30s
startup 2.5 s of 300 s; 2/2 gates passed; No proof was checked by Lean: answers were checked only by grading's parser. validate.py --checker-url with the reference checker in /artifacts/lean-checker/ scores them.; The batch gate was measured on a sample: up to 7 windows of 8 public proofs ag
5 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.5 s
- ✓slowest batch of 8 · 7 windows of 8 concurrent public proofs, the first straight after startup as in grading: first 0.0 s, slowest window 6, mean 0.0 s, 0 retried; grading holds every batch of the held-out split to the same window
- ✓failed requests · 0 of 56 failed, timed out or were rejected after retries; each scores as the unchanged proof, and grading fails the run past 5% of its held-out requests, 18 or more of 354. 56 public requests estimate that rate only roughly, so this check fails only past 10%
- ✓healthy after requests
The agent's last message before its session ended.
Submission completed.