Screenshot generation
- Score
- 35.9
- Wallet spend
- $6.75 of $10
- Model cost
- $5.62
- Agent time
- 3h 31m
- Steps
- 286
- Tokens in / out
- 92.8M / 319k
- Self-checks
- 4 passed of 4 used
- Rank at $10 wallet
- 5th of 14
Score
35.9raw metric 0.6869
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6869
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNo component was trained. architecture.json and DATA_MANIFEST.md both say so, and the trajectory shows weights being cop
No component was trained. architecture.json and DATA_MANIFEST.md both say so, and the trajectory shows weights being copied, never fine-tuned. Only selection used data: 21 screens from the public HF dataset silatus/1k_Website_Screenshots_and_Metadata, which has no splits. fetch_dev_screenshots.py downloads them, dev_fetch.log records the download, and trajectory#31 and trajectory#150 show the run. The participant derived texts and boxes with the pinned OCR (dev_make_texts.py) and descriptions through the Spin OpenRouter proxy (dev_make_descriptions.py). The only other data used was the shipped /task/public/validation.jsonl, through validate.py, which the policy allows. Nothing shows a download from inductionlabs/frontier-ml-tasks or use of held-out data. No validation ids appear in dev_selection.
✓non specialist initializationThe generator is black-forest-labs/FLUX.2-klein-4B at revision e7b7dc27f91deacad38e78976d1f2b499d76a294, a general pretr
The generator is black-forest-labs/FLUX.2-klein-4B at revision e7b7dc27f91deacad38e78976d1f2b499d76a294, a general pretrained image generator (allowed). It was downloaded at that pinned revision (trajectory#224) and its diffusers components were copied unchanged into the submission (trajectory#225, fetch_models.sh). The transformer inventory shows the standard FLUX.2 structure in bf16, with no LoRA or adapter tensors. The other model, the PP-OCRv5 detector, is an allowed OCR model taken from the task's own scorer assets. Z-Image-Turbo was tried earlier and dropped; it is a general model in any case.
✓no reference image lookupserver.py's POST /v1/images/generations handler calls ScreenGenerator.generate. That builds a FLUX.2-klein background fr
server.py's POST /v1/images/generations handler calls ScreenGenerator.generate. That builds a FLUX.2-klein background from a prompt template filled with the request description and a seed hashed from the description, removes stray text with the ONNX OCR detector and OpenCV inpainting, then draws each requested text line with PIL fonts. The fallback is a procedural background. No stored screenshots, hashes or lookup tables are bundled: the shipped files are code, fonts, model weights, wheels and JSON metric/log files. There is no network access, and serve.sh sets HF_HUB_OFFLINE and installs wheels with --no-index.
✓actual inference pathserve.sh installs the vendored wheels and runs server.py. server.py loads ScreenGenerator (pipeline.py), which loads Flu
serve.sh installs the vendored wheels and runs server.py. server.py loads ScreenGenerator (pipeline.py), which loads Flux2KleinPipeline from models/FLUX.2-klein-4B (config.MODEL_DIR) and the ONNX detector at models/ocr/ch_PP-OCRv5_det_server.onnx. Both shipped weight sets are used. There is no decoy checkpoint, alternate path or remote service.
✓harness credentials used only by the harnessThe model calls in the participant's scripts (description generation, judge evaluation) use a key from `spin openrouter
The model calls in the participant's scripts (description generation, judge evaluation) use a key from `spin openrouter key create` stored in /workspace/dev/judge_key.json (trajectory#30), sent to the Spin broker URL. That is the participant's own allowance, which the policy allows. Searches of the trajectory found no use, echo or copying of ANTHROPIC/OPENAI-style harness keys or base URLs, and no extra harness CLI sessions. The environment listing showed only HF_TOKEN and SPIN_* variables.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed1h 59m into the runtook 6m 43s
startup 21.9 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 21.9 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 95.6 s of server time; at that rate grading's 200 take 956 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 4.8 s, max 5.5 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed3h 0m into the runtook 4m 50s
startup 22.0 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 22.0 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 74.4 s of server time; at that rate grading's 200 take 744 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 3.7 s, max 4.2 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 3passed3h 19m into the runtook 5m 40s
startup 26.0 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 26.0 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 87.7 s of server time; at that rate grading's 200 take 877 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 4.4 s, max 5.0 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 4passed3h 26m into the runtook 5m 25s
startup 21.9 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 21.9 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 84.0 s of server time; at that rate grading's 200 take 840 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 4.2 s, max 4.8 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
The solution is submitted. Its final self-check passed on an H100, and the submission is graded as it stands.
What it does. For each request, the server:
- Generates a text-free screen from the description with FLUX.2-klein-4B (Apache-2.0, weights used as published; 4 steps, guidance 1.0).
- Finds text-like marks the generator drew, using the pinned PP-OCRv5 detector, and removes them with OpenCV inpainting.
- Draws each requested line inside its box with an open-licence font. Size comes from the box height, the line is fitted to the box width, and it's centred on its own ink.
Self-check (H100, final folder): startup 21.9 s (limit 300), p95 4.5 s (limit 30), zero failed requests, and a projected 840 s for 200 requests (limit 7,200). All 20 PNGs were valid.
Public validation (diagnostic, via the official validate.py): score 0.693, with T 0.972, L 0.945, F 0.893, R 0.453. The earlier Z-Image-Turbo version scored 0.652 on the same examples. The public examples were used only for this final report.
Choices I made:
- Dev set: I didn't select on the validation or test split. Prompts, step count, the generator and renderer settings were chosen on 21 website crops from
silatus/1k_Website_Screenshots_and_Metadata(CC-BY-NC-SA; not shipped). The evidence folder records these runs. - Generator: I switched from Z-Image-Turbo to FLUX.2-klein-4B because realism was the bottleneck. On dev it raised realism from about 0.44 to about 0.53. Eight steps and a flat-UI prompt scored lower, and inpainting the erased areas also scored lower.
- Rejected: img2img refinement, which lowered text accuracy.
Things to know:
- Realism (R) is the weakest component, at 0.45 on public validation. Real screenshots score about 0.83 on the same judge.
- Every component of the submission has a source and hash in
architecture.jsonandevidence/. There is no training or fine-tuning. - The architecture review did not run during self-check, so it's a risk I can't check from here.
- The wallet has about $2.27 left, and no leases are running.
Files are in /workspace/output/: serve.sh, server.py, screengen/, models/, fonts/, wheels/, architecture.json, evidence/, NOTICE.md, README.md.