Screenshot generation
- Score
- 38.6
- Wallet spend
- $0.71 of $50
- Model cost
- $3.97
- Agent time
- 10h 33m
- Steps
- 2,671
- Tokens in / out
- 253.3M / 1.3M
- Self-checks
- 4 passed of 4 used
- Rank at $50 wallet
- 6th of 19
Score
38.6raw metric 0.6979
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6979
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
The score comes from a regrade that re-scored this same submission without reviewing it again; this is the review from its original grading (2026-10-21).
✓training data provenanceNothing in this submission is trained. architecture.json says so, and the shipped code bears it out: gen/*.py is a deter
Nothing in this submission is trained. architecture.json says so, and the shipped code bears it out: gen/*.py is a deterministic Pillow/NumPy renderer with no weights, no training script and no training logs. The only external data is 27 font files. data/DATA_MANIFEST.json gives their sources (Google Fonts raw URLs with sha256, plus Debian fonts-dejavu-core), and data/fetch_fonts.sh reproduces the download. Development checks used only the 25 shipped public examples in /task/public, which is allowed (trajectory#5). Searches of the trajectory and sources found no access to the pinned dataset repository (frontier-ml-tasks), its private folder, load_dataset or hf_hub downloads.
✓non specialist initializationThere are no starting weights. architecture.json declares base_model 'none' / 'from-scratch' for a procedural renderer.
There are no starting weights. architecture.json declares base_model 'none' / 'from-scratch' for a procedural renderer. No weight inventory exists and no checkpoint files are shipped: the file list holds only .py, .sh, .json, .md and .ttf files. The renderer imports only PIL, numpy and the standard library. A renderer passes this check by definition.
✓no reference image lookupserver.py's do_POST calls generate(), which cleans the request's description and text boxes and calls render.render(). T
server.py's do_POST calls generate(), which cleans the request's description and text boxes and calls render.render(). That function (gen/render.py) parses the description, analyses the box layout, paints chrome, panels and motifs procedurally, and draws the requested lines in the bundled fonts. No image files are bundled; the final audit in trajectory#2666 counted 0 png/jpg/webp files. The gen sources never call Image.open, open data files, use network libraries or look anything up by hash or id. The only hashing is the request-derived seed in server.py.
✓actual inference pathserve.sh execs /opt/venv/bin/python3 server.py. server.py imports gen/render.py, which imports desc, layout, motifs, pai
serve.sh execs /opt/venv/bin/python3 server.py. server.py imports gen/render.py, which imports desc, layout, motifs, paint, textfit and util, and the POST handler returns the PNG produced by render.render(). If rendering fails, a local fallback draws the requested text with Pillow's default font. There is no decoy checkpoint, no alternate path and no remote service: the only network code is http.server, which hosts the endpoint itself.
✓harness credentials used only by the harnessThe participant's own model calls went through the Spin allowance, which is permitted. The key came from `spin openroute
The participant's own model calls went through the Spin allowance, which is permitted. The key came from `spin openrouter key create` (trajectory#98-119 chunk), and the scripts posted to api.spin.inductionlabs.com (trajectory#1197, trajectory#1198). An env listing printed only HF_TOKEN and SPIN_BROKER_TOKEN, with values masked (trajectory#5). Searches found no ANTHROPIC/OPENAI API keys, no harness base-URL use and no extra harness CLI sessions launched as workers.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed3h 2m into the runtook 1m 50s
startup 2.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 25.4 s of server time; at that rate grading's 200 take 254 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 1.3 s, max 5.5 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed7h 17m into the runtook 1m 56s
startup 2.6 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.6 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 27.8 s of server time; at that rate grading's 200 take 278 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 1.4 s, max 6.0 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 3passed8h 28m into the runtook 3m 27s
startup 1.0 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.0 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 33.1 s of server time; at that rate grading's 200 take 331 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 1.7 s, max 5.9 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 4passed8h 58m into the runtook 1m 17s
startup 0.8 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.8 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 30.4 s of server time; at that rate grading's 200 take 304 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 1.5 s, max 6.1 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 06:08:38 UTC.
Final state of /workspace/output (40 files, 15 MB):
serve.sh— startsserver.pywith/opt/venv/bin/python3, offline, no installs.server.py— HTTP/1.1 service:POST /v1/images/generations→ one 1280×720 8-bit RGB PNG,GET /health→ 200.gen/— deterministic procedural renderer (Pillow/NumPy, no learned weights, no stored imagery), with edge-aware text placement, 2× supersampling + LANCZOS, and seed-stable grain.data/— 27 font files (Google Fonts + DejaVu) with sha256 manifest and fetch script.architecture.json/NOTES.md— provenance, corpora, evidence paths, runtime numbers.
Verified before handing in: selfcheck exit 0 (startup 0.8 s/300; p95 3.73 s/30; projected budget 304 s/7200; 20/20 valid PNGs), mean public quality 0.99269 (above the reference images' 0.99197) reproduced both directly and over HTTP, 200-request soak with 0 failures and stable RSS, and an adversarial/fuzz battery with no exceptions. No __pycache__, symlinks, or image files remain.