Screenshot generation
- Score
- 40.5
- Wallet spend
- $13.98 of $50
- Model cost
- $5.16
- Agent time
- 1h 42m
- Steps
- 198
- Tokens in / out
- 20.7M / 204k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 4th of 14
Score
40.5raw metric 0.7058
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.7058
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing in the submission is trained. architecture.json and evidence/initialization.json say there is no training or fin
Nothing in the submission is trained. architecture.json and evidence/initialization.json say there is no training or fine-tuning. Development and selection used only author-written synthetic cases (tools/synthetic_cases.py, evidence/synthetic_cases.jsonl), plus the allowed public /task/public examples for interface and latency checks. Those checks ran with --predictions-only after the code was frozen (evidence/quality_freeze.json). The trajectory shows no download of a benchmark dataset or the held-out split. The other assets come from declared sources: lucide icons from npm, Liberation and DejaVu fonts from Debian, and the PP-OCR ONNX model from ModelScope with a sha256 check. Their download scripts and manifests are kept.
✓non specialist initializationEvery model is a frozen general-purpose model used as published. The planner is Qwen3-Coder-30B-A3B-Instruct-FP8; models
Every model is a frozen general-purpose model used as published. The planner is Qwen3-Coder-30B-A3B-Instruct-FP8; models/coder/config.json shows Qwen3MoeForCausalLM with FP8 quantization. The image model is Z-Image-Turbo, and the text detector is the PaddleOCR PP-OCRv5 detector in ONNX form. tools/download_models.py pins the Qwen and Z-Image revisions, evidence/model_download.log confirms both downloads, and tools/download_text_detector.py checks the detector's sha256. The weight inventories match these models. No screenshot-specialist checkpoint was loaded or trained. The Qwen3-4B model used in early development was replaced and is not shipped.
✓no reference image lookupEach image is built at request time. renderer.py turns the LLM-planned SVG into pixels with resvg, using only whiteliste
Each image is built at request time. renderer.py turns the LLM-planned SVG into pixels with resvg, using only whitelisted tags and with external, data: and @import references blocked. It embeds only the photos generated for that request and draws the requested text with the bundled fonts. server.py runs offline with no network calls. evidence/artifact_audit.log reports no stored PNG screenshots, and the trajectory shows no code that copies or retrieves reference images.
✓actual inference pathThe serving chain is serve.sh, which installs the local wheels offline and runs server.py. server.py calls model.py: the
The serving chain is serve.sh, which installs the local wheels offline and runs server.py. server.py calls model.py: the Planner loads vLLM on models/coder and the Imagery class loads ZImagePipeline on models/imagery with local_files_only. photo_cleaner.py then runs models/text_detector.onnx through ONNX Runtime with cv2 inpainting, and renderer.py rasterizes the scene and draws the text. All of these weights are in the inventories, and there is no remote service or decoy path. The offline H100 self-check passed, starting in 192.1 s (trajectory#195).
✓harness credentials used only by the harnessThe only credential the participant used was their own Spin OpenRouter key, created with `spin openrouter key create` (t
The only credential the participant used was their own Spin OpenRouter key, created with `spin openrouter key create` (trajectory#12) and passed to the synthetic scorer as --judge-key-file. This is allowed. The env listing at trajectory#10 showed variable names only, and no harness key value was used. The harness auth.json was only seen in a directory listing (trajectory#16), never read or used. The participant also scanned the output for leaked hf_ and sk- tokens before submitting (trajectory#187).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed1h 24m into the runtook 18m 7s
startup 192.1 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 192.1 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 290.6 s of server time; at that rate grading's 200 take 2,906 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 14.5 s, max 16.5 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with serve.sh, generator code, model weights, offline wheels, and provenance logs.
- Public interface validation: 25/25 valid responses, no failures.
- Offline H100 selfcheck: passed.
- Startup: 192.1 seconds.
- p95 latency: 16.36 seconds, with zero failures across 20 checked requests.
Quality development uses handcrafted synthetic cases only; no public benchmark quality tuning.