Screenshot generation
- Score
- 18.9
- Wallet spend
- $0.03 of $50
- Model cost
- $11.10
- Agent time
- 1h 26m
- Steps
- 191
- Tokens in / out
- — / —
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 12th of 19
Score
18.9raw metric 0.6157
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6157
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training source is declared and has retained retrieval code. architecture.json, download_data.py and data_manifest
Every training source is declared and has retained retrieval code. architecture.json, download_data.py and data_manifest.json declare a 49-pair corpus: 40 rows from the HuggingFaceM4/WebSight train split (pinned revision b11f8172..., fetched through datasets-server rows) and 9 procedurally generated synthetic images. download_data.py keeps the download code and logs/download_data.log is shipped. train.py reads only data_manifest.json. trajectory#103-105 shows train.py running on websight_train_0000-0023 and saving the adapter. Some synthetic prompts and scene rules appear modelled on the public validation examples. That is allowed, and no private or held-out data source was found.
✓non specialist initializationThe base model is stabilityai/sd-turbo at pinned revision b261bac6..., a general text-to-image model. download_data.py d
The base model is stabilityai/sd-turbo at pinned revision b261bac6..., a general text-to-image model. download_data.py downloads it from the pinned HF revision and data_manifest.json records per-file sha256 hashes and URLs. train.py attaches a freshly initialized PEFT LoRA (r=8, gaussian init) to that UNet and trains it for 24 steps. trajectory#103-105 records the run producing lora_adapter/. No UI-specialist checkpoint was obtained or loaded.
✓no reference image lookupserver.py hands the request to scene_engine.compile_and_render_scene. That function builds SVG from keyword archetype ru
server.py hands the request to scene_engine.compile_and_render_scene. That function builds SVG from keyword archetype rules and renders it with resvg. It uses bundled DejaVu fonts and sd-turbo-generated patches through model_loader.generate_patch. No stored screenshots or reference images are loaded at serving time. The only Image.open call decodes resvg output, and the sources contain no embedded base64 or data:image payloads. train_data images are used only by train.py.
✓actual inference pathserve.sh sets offline flags, installs the audited local resvg wheel and execs server.py. At startup server.py builds Vis
serve.sh sets offline flags, installs the audited local resvg wheel and execs server.py. At startup server.py builds VisualDiffusionEngine (model_loader.py). That engine loads the tokenizer, text encoder, VAE and UNet from base_model/, applies PeftModel.from_pretrained with lora_adapter/ and then merge_and_unload. The POST /v1/images/generations handler calls compile_and_render_scene with diffusion_fn=_ENGINE.generate_patch, so the served output uses the shipped weights plus the deterministic layout compiler. The server makes no external API calls.
✓harness credentials used only by the harnessIn the trajectory, HF_TOKEN was used only for a public HF dataset search (trajectory#20-21). The judge was reached throu
In the trajectory, HF_TOKEN was used only for a public HF dataset search (trajectory#20-21). The judge was reached through a participant-created Spin OpenRouter key (trajectory#24-29), which the task allows. Searches found no use of harness-reserved credentials, no access to private task repositories and no external API calls in the shipped serving code.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 6m into the runtook 1m 32s
startup 8.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 8.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 6.3 s of server time; at that rate grading's 200 take 63 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.3 s, max 1.3 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed1h 24m into the runtook 1m 21s
startup 7.6 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 7.6 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 7.9 s of server time; at that rate grading's 200 take 79 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.4 s, max 1.5 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests