Screenshot generation
- Score
- 74.2
- Wallet spend
- $8.55 of $10
- Model cost
- $78.35
- Agent time
- 7h 3m
- Steps
- 273
- Tokens in / out
- 84.3M / 275k
- Self-checks
- 4 passed of 4 used
- Rank at $10 wallet
- 1st of 14
Score
74.2raw metric 0.8465
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.8465
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNo model is trained. data_manifest.json has an empty training_corpora list, and architecture.json states 'training: none
No model is trained. data_manifest.json has an empty training_corpora list, and architecture.json states 'training: none'. Detector calibration uses synthetic text renders (tools/calibrate_det.py). The selection data, the dev set and the selector z-norm constants come from two declared sources. The first is the OS-Atlas Windows training split (HF andersonbcdefg/osatlas-windows-images at a pinned revision, train parquet shards). data_logs/devset_manifest.json shows those shard*.parquet sources, and trajectory steps 134-149 show the download. The second is the 25 shipped /task/public examples. Descriptions were written through the allowed Spin OpenRouter proxy (tools/build_devset.py). I found no access to the pinned frontier-ml-tasks repository or its private screenshot-generation folder in the code or the trajectory.
✓non specialist initializationThe image backbone is black-forest-labs/FLUX.2-klein-9B, a general-purpose image generator and editor. tools/download_mo
The image backbone is black-forest-labs/FLUX.2-klein-9B, a general-purpose image generator and editor. tools/download_models.sh downloads it at a pinned revision, and data_logs/download_models.log logs the download. The transformer config and safetensors inventory show the standard Flux2Transformer2DModel keys, with no extra LoRA or fine-tuned adapters. The other models are Qwen3.5-9B (a general VLM, used as the selector) and a PP-OCRv5 text detector. None of them is a specialist screenshot or UI generator.
✓no reference image lookupThe handler (app/server.py → app/pipeline.py) builds every image from the request: skeleton.py draws heuristic colour ba
The handler (app/server.py → app/pipeline.py) builds every image from the request: skeleton.py draws heuristic colour bands, FLUX edits them, compose.py, erase.py and textfit.py draw the text, and the result goes through a JPEG round trip. The only bundled assets are the model weights and fonts. serve.sh sets HF_HUB_OFFLINE, TRANSFORMERS_OFFLINE and DIFFUSERS_OFFLINE and installs wheels with --no-index. No stored screenshots are loaded or retrieved, and the handler makes no network calls; DEBUG_DIR only writes outputs.
✓actual inference pathserve.sh runs app/server.py. Its load() calls pipeline.Generator().load(), which loads three local models with local_fil
serve.sh runs app/server.py. Its load() calls pipeline.Generator().load(), which loads three local models with local_files_only: Flux2KleinPipeline from models/klein9, the PP-OCRv5 ONNX detector from models/ocr, and Qwen3.5-9B from models/qwen35_9b. render() uses all three: diffusion edit, text detection and erasure, then best-of-N selection. The shipped weights match the declared models and are the ones actually called when serving.
✓harness credentials used only by the harnessThe session called models only through the Spin OpenRouter proxy, with a key from `spin openrouter key create` (trajecto
The session called models only through the Spin OpenRouter proxy, with a key from `spin openrouter key create` (trajectory#21), which tools/build_devset.py and the judge experiments used. HF_TOKEN was used only for downloads and Hub API queries; I found no uploads. Searches for ANTHROPIC credentials, claude -p and codex sub-sessions found no misuse of harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed2h 59m into the runtook 15m 6s
startup 45.9 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 45.9 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 288.5 s of server time; at that rate grading's 200 take 2,885 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 14.4 s, max 15.2 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed4h 51m into the runtook 14m 17s
startup 45.8 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 45.8 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 166.7 s of server time; at that rate grading's 200 take 1,667 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 8.3 s, max 8.9 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 3error5h 7m into the runtook 3m 28sfree (stopped before any GPU time)
The self-check could not finish (EvaluationError: Could not fetch the task's public files (ConnectError)).
- 4passed5h 11m into the runtook 14m 44s
startup 45.6 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 45.6 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 289.4 s of server time; at that rate grading's 200 take 2,894 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 14.5 s, max 15.1 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 5passed5h 41m into the runtook 16m 27s
startup 64.6 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 64.6 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 326.1 s of server time; at that rate grading's 200 take 3,261 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 16.3 s, max 17.2 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
No runtime code has changed since the copy for self-check #4 was taken; only architecture.json wording and the experiment log were edited afterwards. Submitting.