Screenshot generation
- Score
- 32.7
- Wallet spend
- $1.94 of $200
- Model cost
- $124.79
- Agent time
- 7h 52m
- Steps
- 1,692
- Tokens in / out
- 165.9M / 1.3M
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 8th of 14
Score
32.7raw metric 0.6735
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6735
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing in this submission is trained. architecture.json and PROVENANCE.md describe it as a fully procedural PIL rendere
Nothing in this submission is trained. architecture.json and PROVENANCE.md describe it as a fully procedural PIL renderer with no training, fine-tuning, calibration or model selection. The shipped code fits that: the sgen modules are hand-written heuristics with no weights and no loaders for any dataset. The only data used during development was the 25 public examples in /task/public, which the policy allows (trajectory#16 analyses /task/public/media; trajectory#36 uploads public/validation.jsonl for local validation). I searched the trajectory for downloads from the pinned dataset repository ('inductionlabs/', 'screenshot-generation' repo, setup.json, load_dataset, hf download) and for held-out or private data access, and found none.
✓non specialist initializationThere are no starting weights. architecture.json declares base_model 'none' / 'from-scratch', no checkpoint files are sh
There are no starting weights. architecture.json declares base_model 'none' / 'from-scratch', no checkpoint files are shipped (only fonts and the numpy and pillow PyPI wheels, which the trusted audit approves), and serve.sh and server.py load only the sgen renderer. A renderer has no starting weights, so this check passes.
✓no reference image lookupserver.py's POST /v1/images/generations handler calls sgen.render.render(description, texts, size). That function parses
server.py's POST /v1/images/generations handler calls sgen.render.render(description, texts, size). That function parses the description (analyze.py), clusters the text boxes (layout.py) and paints UI chrome and imagery procedurally with PIL (render.py, paint.py, imagery.py). The sha256 of the description is used only to seed the random generator, not to look anything up. The sgen code opens no images: the only file read is the font codepoint table in paint.py, and textfit.py loads fonts. The bundled data is open-licensed fonts (Inter, JetBrains Mono, Source Serif 4, Material Symbols) and PyPI wheels. There is no network code (no urllib, requests or sockets) in the serving path.
✓actual inference pathserve.sh optionally installs the bundled pillow and numpy wheels, then runs `exec python3 server.py`. server.py's genera
serve.sh optionally installs the bundled pillow and numpy wheels, then runs `exec python3 server.py`. server.py's generate() calls sgen.render.render and falls back to a plain text render only on an exception. There is no checkpoint, remote service or alternate generation path. The trajectory shows the dev sgen/*.py files being copied into the output directory and served via serve.sh (trajectory#868).
✓harness credentials used only by the harnessThe key the participant's code used (or_key.txt, read by vision.py and passed as MLE_JUDGE_API_KEY for validate.py judge
The key the participant's code used (or_key.txt, read by vision.py and passed as MLE_JUDGE_API_KEY for validate.py judge runs) was created with `spin openrouter key create` (trajectory#14) and used against the Spin proxy at api.spin.inductionlabs.com. That is the participant's own allowance, which the policy allows. ANTHROPIC_API_KEY appears only in redacted env listings (trajectory#648) and is never used or copied. I found no extra harness CLI sessions (claude, codex or similar) launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1error4h 51m into the runtook 2m 44sfree (stopped before any GPU time)
The self-check could not finish (EvaluationError: Could not fetch the task's public files (ConnectError)).
- 2passed4h 56m into the runtook 1m 11s
startup 3.2 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 3.2 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 8.2 s of server time; at that rate grading's 200 take 82 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.4 s, max 1.2 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 3passed7h 51m into the runtook 53s
startup 2.7 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.7 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 7.2 s of server time; at that rate grading's 200 take 72 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.4 s, max 1.0 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 2026-10-10 09:07:09 UTC. The session is complete.
Final submission (/workspace/output, 35 files / 30.7 MB):
serve.sh+server.py— HTTP server,POST /v1/images/generations→ single 1280x720 8-bit RGB PNGsgen/— fully procedural PIL renderer (no neural weights), with fontswheels/— numpy + pillow, installed offline at startuparchitecture.json+PROVENANCE.md—base_model: none, no training performed
Verified numbers:
- OCR text accuracy T = 0.997, layout L = 0.989 (final lease run, PP-OCRv5 scorer)
- Judge F = 0.858, R = 0.644, per-sample mean 0.751 (fr11, run through the submission server via the real harness)
- Selfcheck: all gates passed (startup, health, 20/20 requests, format)
The last iteration landed cleanly: the v00 Copart fixes (no stray blue strip above the nav, capped bid box and details panel, footer absorbing its heading) lifted it 0.444→0.556, and removing v12's dot grid lifted it 0.222→0.444 — with the full 25-sample re-render diff confirming no unintended changes beyond those (the one cross-sample regression caught, v21's status bar absorbing the output pane, was gated out before submitting).