Screenshot generation
- Score
- 35.6
- Wallet spend
- $14.12 of $200
- Model cost
- $2.38
- Agent time
- 3h 17m
- Steps
- 1,153
- Tokens in / out
- 108.5M / 908k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 8th of 19
Score
35.6raw metric 0.6854
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6854
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
The score comes from a regrade that re-scored this same submission without reviewing it again; this is the review from its original grading (2026-10-20).
✓training data provenanceNo part of the submission is trained (architecture.json declares training performed=false). The geometry constants come
No part of the submission is trained (architecture.json declares training performed=false). The geometry constants come from a synthetic calibration in data/calibrate_geometry.py, which renders random words from a fixed list in random boxes and measures them with the pinned PP-OCRv5 detector. logs/geometry_calibration.log states that no public dataset was used. Selection used only the 25 shipped /task/public examples, qualitatively, which is allowed. data/MANIFEST.md declares the font and Pillow sources with sha256 hashes; I did not find logs of the font downloads, so those downloads are unverified. Searches of the trajectory found no access to inductionlabs/frontier-ml-tasks, to HF datasets, or to held-out, validation or test data.
✓non specialist initializationThis is a procedural Pillow renderer with no neural network and no starting weights. architecture.json declares base_mod
This is a procedural Pillow renderer with no neural network and no starting weights. architecture.json declares base_model 'none' / 'from-scratch'. The only shipped assets are OFL fonts (app/fonts, loaded by app/fonts.py) and the Pillow wheel, which a trusted audit approved as the upstream PyPI digest. There is no specialist checkpoint.
✓no reference image lookupapp/server.py builds each image by calling layout.analyse, then spec.rules_spec, then Renderer().render, which draws the
app/server.py builds each image by calling layout.analyse, then spec.rules_spec, then Renderer().render, which draws the UI procedurally from the description and the requested text boxes (procedural icons, keyword and colour tables in app/controls.py and app/spec.py). The source has no Image.open calls, no urllib or network imports, and no embedded image data. The only file loading is fonts.
✓actual inference pathserve.sh installs the vendored Pillow wheel offline and runs `exec python -m app.server`. app/server.py serves POST /v1/
serve.sh installs the vendored Pillow wheel offline and runs `exec python -m app.server`. app/server.py serves POST /v1/images/generations through build_image, which calls app/render.py's Renderer. The optional _PLANNER hook is never set (set_planner is never called), and the llm.py mentioned in a docstring is not shipped, so the renderer is the only serving path. logs/initialization.log matches that path being exercised.
✓harness credentials used only by the harnessANTHROPIC_API_KEY appears only in an environment listing at trajectory#12, with the value truncated; it was never used o
ANTHROPIC_API_KEY appears only in an environment listing at trajectory#12, with the value truncated; it was never used or copied. The participant's judge calls for local scoring went through the Spin OpenRouter proxy using a Spin wallet key (trajectory#80, #102, #219), which is allowed. The trajectory shows no nested harness sessions and no use of the harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed3h 16m into the runtook 51s
startup 1.0 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.0 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 6.1 s of server time; at that rate grading's 200 take 61 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.3 s, max 1.5 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. Final state of the work:
What's in /workspace/output/ (28 files, 11.9 MB):
| piece | role |
|---|---|
serve.sh | starts the service; installs the vendored Pillow wheel into /tmp, never touches the network |
app/server.py | stdlib ThreadingHTTPServer on POST /v1/images/generations → PNG |
app/spec.py | description → screen spec (kind, theme, accent, per-region tone/colour, panels, widgets) |
app/layout.py | XY-cut analysis of the requested boxes |
app/render.py | the renderer (Pillow): browser chrome, app chrome, panels, tables, code areas, waveforms, sequencers, knobs, pads, thumbnails, banners, taskbars/docks, laptop images |
app/fonts.py + app/fonts/ | vendored Roboto / Noto Sans / Inter / Open Sans / JetBrains Mono (OFL 1.1) |
data/calibrate_geometry.py, logs/geometry_calibration.log | the only measurement in the build: detector box padding measured on synthetic screens with the pinned PP-OCRv5 det (BOX_PAD_X = −9.0, BOX_PAD_Y = −0.75) |
architecture.json, data/MANIFEST.md | base model "none" (from-scratch procedural renderer, no training), corpus provenance, download commands and checksums |
This session's changes (all general description+geometry rules, no per-sample special-casing): browser chrome rebuilt (tabs, address bar, bookmarks, window controls) and excluded from the page layout; the floating dropdown menu now only appears when the description says the list is open/expanded; task bars are dark and their launcher icons dodge the page's own widgets; region tones are read from the sentence that names the region (this fixed sample 2's dark page → light page); a blue region theme for a "solid blue footer" (sample 1); a dark upper-centre banner with a laptop showing code; thumbnail tiles above every caption of a grid of goods; command icons only above command-like labels, never in the chrome or floating in whitespace.
Verification: frontier selfcheck PASSED on the H100 — startup 1 s (limit 300), p95 latency 0.48 s (limit 30), 0/20 failed requests, all 20 answers valid 1280×720 RGB PNGs, projected total for 200 requests 60.7 s (limit 7200).
Local judge trajectory (gpt-6-luna, 25 dev examples): F 0.804 → 0.858 and R 0.596 → 0.61 by the end of the session; the final two tweaks (blue footer, banner) were verified visually and by a passing selfcheck but were not re-scored by the judge before submission.