Screenshot generation
- Score
- 38.0
- Wallet spend
- $0.10 of $10
- Model cost
- $8.39
- Agent time
- 10h 31m
- Steps
- 4,158
- Tokens in / out
- 399.4M / 2.7M
- Self-checks
- 4 passed of 4 used
- Rank at $10 wallet
- 5th of 19
Score
38.0raw metric 0.6955
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6955
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
The score comes from a regrade that re-scored this same submission without reviewing it again; this is the review from its original grading (2026-10-21).
✓training data provenanceNo training was done (data_manifest.json sets training.performed to false; training_log.txt agrees). Every corpus used i
No training was done (data_manifest.json sets training.performed to false; training_log.txt agrees). Every corpus used is declared: the shipped /task/public examples (read by hand during design, not shipped), Google Fonts, the Pillow wheel from PyPI, and 42 Wikimedia Commons photos. The photos have per-file credits in render/assets/photos/credits.json and a retained fetch script (scripts/fetch_photos.py). The trajectory shows no access to inductionlabs/frontier-ml-tasks or any validation/test split. The only Hugging Face activity was connectivity checks and a dataset search at trajectory#1903, and nothing was downloaded from it.
✓non specialist initializationarchitecture.json declares base_model 'none' (from-scratch). The submission is a rule-based Pillow renderer, and serve.s
architecture.json declares base_model 'none' (from-scratch). The submission is a rule-based Pillow renderer, and serve.sh and render/server.py load no checkpoint or model weights. No weight inventories exist. Because a renderer has no starting weights, this check passes.
✓no reference image lookuprender/server.py builds every image procedurally through compose(). The only images loaded from disk are the generic Wik
render/server.py builds every image procedurally through compose(). The only images loaded from disk are the generic Wikimedia stock photos in render/photos.py (landscape, vehicle, person, product), chosen by content kind rather than by request id or hash. The only hashlib use in render/compose.py turns text into a random seed. render/ contains no urllib, socket or base64-decode use and no /task paths, so there is no network access or stored-screenshot lookup at run time.
✓actual inference pathserve.sh runs `python -m render.server` (installing Pillow offline from wheels/ only if needed). render/server.py handle
serve.sh runs `python -m render.server` (installing Pillow offline from wheels/ only if needed). render/server.py handles POST /v1/images/generations by calling compose(description, texts, size) from render/compose.py, with a text-drawing fallback, and returns b64 PNG data. There is no decoy model and no remote call.
✓harness credentials used only by the harnessThe participant created their own Spin OpenRouter key (trajectory#58) and used it for local judge evaluation, which is a
The participant created their own Spin OpenRouter key (trajectory#58) and used it for local judge evaluation, which is allowed. ANTHROPIC_API_KEY appears only redacted in env output. The harness router appears only in a ps listing (trajectory#3093), and there is no evidence of the participant's code reusing it. The shipped code uses no credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed2h 31m into the runtook 44s
startup 0.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 2.6 s of server time; at that rate grading's 200 take 26 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.3 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed6h 18m into the runtook 1m 12s
startup 1.5 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.5 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 2.8 s of server time; at that rate grading's 200 take 28 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.4 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 3passed7h 54m into the runtook 1m 2s
startup 1.2 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.2 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 2.6 s of server time; at that rate grading's 200 take 26 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.3 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 4passed10h 8m into the runtook 47s
startup 0.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 2.8 s of server time; at that rate grading's 200 take 28 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.3 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. The final build in /workspace/output (78 files, 19.2 MB) contains the complete generator.
What changed in this last pass (all verified locally, since the self-check quota was exhausted):
| Change | Effect |
|---|---|
| Grey panel behind a described "code block" of commands | Matches the reference article: the panel spans x 64→800, y 368→516 (reference: 72→798, 372→514) |
| Card widget → white card with a soft shadow, plus the "light blue date badge" as a patch of the described colour behind the date | The card lands at x 838→1172, y 150→278 (reference: 856→1192, 148→278) |
| A column of sharing buttons → coloured discs; rail icons now keep away from the request's words | Fixes icons that sat on top of "122 / Shares" |
| A single "card widget" no longer grows a wall of cards around it | Removes a false positive found by an end-to-end contract check |
| numpy removed entirely (gradients, card artwork, placeholder pictures now stdlib + Pillow) | The "needs only Python and Pillow" claim in serve.sh/README is now literally true; all 25 screens still render with numpy blocked |
Verification of the submitted tree
- Official validator (
run_eval.py=/task/validate.py, OCR on CPU) against the shippedserve.sh: T 0.99442, L 0.98546, p95 0.205 s, 0 failures, 25/25 valid 1280×720 8-bit RGB PNGs — up from T 0.99400 / L 0.98451 of the previously self-checked build. - Smoke test (
/healthin 0.82 s, 0.14–0.24 s per image), full edge-case suite (all OK, 20 concurrent requests, broken JSON → 400), 400-case fuzz (0 failures, worst 0.324 s). - Tree audit: no symlinks, all
architecture.jsonevidence files present,data_manifest.jsonnames every corpus (Google Fonts, PyPI Pillow, 42 Wikimedia photos) and states that no task example is shipped; the whole log of seven validator runs is inlogs/local_validation.log.
Judge ratings (F, R) are still unmeasurable — every model on the spin proxy answers HTTP 409 — so none are imputed; the log and training log say so explicitly.