Screenshot generation
- Score
- 0.0
- Wallet spend
- $14.07 of $50
- Model cost
- $66.21
- Agent time
- 5h 28m
- Steps
- 958
- Tokens in / out
- 93.2M / 619k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 11th of 14
Score
0.0raw metric 0.4627
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.4627
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing is trained, fine-tuned or calibrated. architecture.json, evidence/data-manifest.json and evidence/no-training.md
Nothing is trained, fine-tuned or calibrated. architecture.json, evidence/data-manifest.json and evidence/no-training.md all say the training and calibration corpora are 'none'. The only bundled data assets are these: 442 decorative photos generated offline with FLUX.1-schnell at a pinned revision, open fonts, Lucide icons and Chromium/Ubuntu libraries. The FLUX photos were made from hand-written prompts; the parameters are logged in evidence/photo_generation.log, and the prompts and generation script appear at trajectory#124 and trajectory#183. The provided /task/public examples were used only to measure metrics, which the policy allows. I found no download of a forbidden dataset, and no lookup in a benchmark repo or setup file, in the code or the trajectory.
✓non specialist initializationThe serving system contains no learned model. It builds HTML/CSS/SVG in code and renders it with headless Chromium. The
The serving system contains no learned model. It builds HTML/CSS/SVG in code and renders it with headless Chromium. The only weight-inventory entry, chrome/shell/v8_context_snapshot.bin, is a V8 JavaScript-engine snapshot, not ML weights. FLUX.1-schnell is an allowed general-purpose base. It was downloaded from Hugging Face with HF_TOKEN (trajectory#155) and used only offline to generate decorative assets. Nothing was initialized from a specialist checkpoint.
✓no reference image lookupserver.py passes the request's description and texts to gen/generate.py. generate.py builds the page from Style(descript
server.py passes the request's description and texts to gen/generate.py. generate.py builds the page from Style(description), Layout(texts) and a seed derived by hashing the request, then renders local HTML through gen/render.py with background networking turned off. The bundled photo library holds FLUX-generated images with generic subjects, listed in gen/photos/manifest.json. No reference or target images are bundled or fetched, and there are no network calls.
✓actual inference pathserve.sh installs local wheels, runs a warmup and then execs server.py. server.py's POST /v1/images/generations handler
serve.sh installs local wheels, runs a warmup and then execs server.py. server.py's POST /v1/images/generations handler calls generate_png, which runs the Style/Layout/PageBuilder pipeline in gen/generate.py and renders with the bundled chrome-headless-shell via gen/render.py. If that fails, it falls back to drawing the texts with PIL. I found no decoy checkpoints, remote inference services or unused models; the declared architecture matches the code that actually serves requests.
✓harness credentials used only by the harnessThe judging during development used a Spin OpenRouter key the participant created with `spin openrouter key create` (tra
The judging during development used a Spin OpenRouter key the participant created with `spin openrouter key create` (trajectory#42, trajectory#43, used at trajectory#199), which the policy allows. HF_TOKEN was used only to download FLUX on the Spin machine (trajectory#155), which is also allowed. Credential-like strings showed up only in `ps aux` process-listing output (trajectory#196), and those values were never reused. I found no use of the harness's own credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed5h 19m into the runtook 3m 2sfree (stopped before any GPU time)
failed snapshot: Unsafe or unreadable submission archive: './chrome/syslibs/etc/fonts/conf.d/10-hinting-slight.conf' is a link to an absolute path
1 gates
- ✗snapshot · Unsafe or unreadable submission archive: './chrome/syslibs/etc/fonts/conf.d/10-hinting-slight.conf' is a link to an absolute path
- 2passed5h 23m into the runtook 1m 11s
startup 0.8 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.8 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 10.7 s of server time; at that rate grading's 200 take 107 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.5 s, max 0.8 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 3passed5h 24m into the runtook 1m 30s
startup 1.4 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.4 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 10.8 s of server time; at that rate grading's 200 take 108 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.5 s, max 0.9 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 07:56:05 UTC. Final session summary:
What was done this session
1. Tightened _right_split (the open problem from before):
- Excluded footer items from the corridor test, required a real corridor gap ≥ 40px, right-cluster span ≤ 400, cohesive left edge (x1 spread ≤ 200), rightmost valid boundary, and app/ide kinds only
- Result: fires on exactly 4 genuinely right-panel apps (0009 effects column, 0010 Blender outliner, 0013 Power BI config pane, 0023 Explorer tree) — down from 12 false positives that had tanked 0008's L to 0.928
2. Fixed a systematic layering bug: the right panel started above the header bar, and fill_empty's decorations (3D viewport, quiet panels) painted over both the decorated left sidebar and right panel. Both panels now bound decorate_main's fill region.
3. 0019 (Word, worst scorer): "solid blue background" now renders flat blue instead of a teal/gold gradient; "blue title bar" tints the title bar + ribbon.
4. 0009: imagery flag now catches "image featuring/colorful image" so the program monitor gets a real photo; decorate_form's checkbox card no longer overflows the canvas and stays inside the right pane; decorate_right_panel gained per-row controls (sliders/dropdowns/toggles) matching the description's "sliders, checkboxes, and dropdown menus".
Final verified numbers (eval25, official judge + pinned OCR)
- T = 0.9966, L = 0.9868 (baseline 0.9958/0.9873)
- R = 0.533, F = 0.747 (baseline 0.444/0.631 — +0.09 R, +0.12 F)
- Biggest sample gains: 0010 (2,6)→(5,9), 0019 (2,3)→(6,8), 0014 (3,6)→(5,8), 0013 (3,7)→(5,8), 0017 (3,2)→(6,7)
Submission hygiene
- Dereferenced 296 absolute symlinks in the chrome syslibs (selfcheck had failed on them)
- fonts.conf now written to the writable cache dir (robust to a read-only submission mount)
- Chrome render retries once in an isolated cache dir with stderr logging
frontier selfcheck: PASSED — 0% failed requests, p95 latency 0.83s (limit 30s), all responses 1280x720 8-bit RGB PNGs- Server verified end-to-end with the exact contract payload (0.5–0.8s per image)