Screenshot generation
- Score
- 59.3
- Wallet spend
- $22.70 of $50
- Model cost
- $163.70
- Agent time
- 5h 47m
- Steps
- 785
- Tokens in / out
- 102.7M / 680k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 3rd of 14
Score
59.3raw metric 0.7844
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.7844
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training, selection and diagnostic corpus is declared. They are synthetic cases written by openai/gpt-6-luna throu
Every training, selection and diagnostic corpus is declared. They are synthetic cases written by openai/gpt-6-luna through the Spin OpenRouter proxy (training/generate_cases.py, the training manifests, diagnostics/data marked 'excluded from training') and recorded in architecture.json data_sources. Neither the trajectory nor the code shows access to held-out benchmark data. The public validation examples were used only for validation runs. In any case, architecture.json declares the final selected candidate to be the base checkpoint, so no fine-tuned weights are served.
✓non specialist initializationThe planner is Qwen/Qwen3.6-27B-FP8, a general-purpose LLM pinned at a revision in architecture.json and download_models
The planner is Qwen/Qwen3.6-27B-FP8, a general-purpose LLM pinned at a revision in architecture.json and download_models.py. Its shipped shards are FP8 layer tensors consistent with that base. Photo regions use Z-Image-Turbo, a general text-to-image model, and text inpainting uses generic PP-OCRv5 ONNX models. No specialist UI or screenshot generator is used for initialization.
✓no reference image lookupserver.py builds each image from the planner's drawing program. It executes the program with renderer.py/vector_backend.
server.py builds each image from the planner's drawing program. It executes the program with renderer.py/vector_backend.py, which rasterize SVG via resvg and refuse href, file: and url() references. Z-Image generates photo regions. No raster images are shipped, serve.sh runs offline (HF_HUB_OFFLINE), and no stored-screenshot lookup exists. Image.open calls in training/refiner_experiment are abandoned experiments that are not on the serving path.
✓actual inference pathserve.sh starts planner_runtime.py (vLLM) with --model models/planner and then server.py. server.py calls the local plan
serve.sh starts planner_runtime.py (vLLM) with --model models/planner and then server.py. server.py calls the local planner endpoint, loads ZImagePipeline from models/imagery and PhotoTextCleanup from models/photo_ocr, and renders through renderer.py. The shipped weights in models/planner, models/imagery and models/photo_ocr match this call graph.
✓harness credentials used only by the harnessModel calls during development used a Spin OpenRouter key created with `spin openrouter key create` (/workspace/spin-key
Model calls during development used a Spin OpenRouter key created with `spin openrouter key create` (/workspace/spin-key.json), passed explicitly to training/generate_cases.py. The trajectory shows no use of the harness's own credentials or base URL, and no extra agent sessions launched with them.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed35m 43s into the runtook 20m 7s
startup 188.6 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 188.6 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 292.0 s of server time; at that rate grading's 200 take 2,920 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 14.6 s, max 20.2 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed4h 37m into the runtook 21m 20s
startup 218.5 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 218.5 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 330.7 s of server time; at that rate grading's 200 take 3,307 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 16.5 s, max 26.4 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/.
- Offline self-check passed: startup 218.5s, p95 26.3s.
- Reliability test: 200/200 valid images, zero failures.
- Public validation: score 0.762, OCR accuracy 0.998, layout 0.972.