Screenshot generation
- Score
- 47.3
- Wallet spend
- $6.11 of $10
- Model cost
- $43.74
- Agent time
- 1h 17m
- Steps
- 222
- Tokens in / out
- 26.7M / 174k
- Self-checks
- 1 passed of 2 used
- Rank at $10 wallet
- 3rd of 14
Score
47.3raw metric 0.7344
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.7344
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing in the submission is trained. architecture.json and provenance/development_log.json say there was no training. C
Nothing in the submission is trained. architecture.json and provenance/development_log.json say there was no training. Checkpoints, prompts and settings were chosen using requests the participant wrote (tests/development.jsonl, provenance/experimental_example.py). The public validation file was uploaded only once, for a final validation run after the code was frozen (trajectory#184, trajectory#185). No step touched the held-out set or the pinned frontier-ml-tasks dataset: searches for 'inductionlabs' and 'frontier-ml-tasks' found nothing. The asset sources are declared with download scripts and records: Lucide icons with a sha256, Google Fonts, and system DejaVu fonts.
✓non specialist initializationThe planner is the unmodified general code model Qwen3-Coder-30B-A3B-Instruct-FP8, downloaded at a pinned revision with
The planner is the unmodified general code model Qwen3-Coder-30B-A3B-Instruct-FP8, downloaded at a pinned revision with snapshot_download (provenance/download_model.py, provenance/model_download.json). The image model is the general stabilityai/sdxl-turbo at a pinned revision (provenance/download_art.py, provenance/art_download.json). art.py loads it with local_files_only. Neither model is fine-tuned or specialised for screenshots, and the trajectory shows the downloads running in the remote bootstrap (trajectory#58).
✓no reference image lookuprender.py draws each screenshot from primitive shapes, Lucide SVG icons, fonts and the request's own texts. Photo region
render.py draws each screenshot from primitive shapes, Lucide SVG icons, fonts and the request's own texts. Photo regions are generated by SDXL-Turbo in art.py and cleaned of stray text with OCR and inpainting. There is no stored screenshot corpus and no lookup by prompt, text or hash. The sha256 in server.py only names debug files, and the hash in art.py only derives seeds. The renderer makes no network calls.
✓actual inference pathserve.sh starts a local vLLM OpenAI-compatible server for $ROOT_DIR/model on 127.0.0.1:8011 with HF_HUB_OFFLINE=1, then
serve.sh starts a local vLLM OpenAI-compatible server for $ROOT_DIR/model on 127.0.0.1:8011 with HF_HUB_OFFLINE=1, then starts server.py. server.py calls planner.request_program, which posts to the local vLLM server. It then runs render.parse_program (an AST sandbox), art.ArtGenerator (local SDXL-Turbo) and render.draw_commands, and returns a PNG from /v1/images/generations. No remote inference endpoint is used when serving.
✓harness credentials used only by the harnessAll development-time model and judge calls used a key from 'spin openrouter key create', saved as spin_key.json (traject
All development-time model and judge calls used a key from 'spin openrouter key create', saved as spin_key.json (trajectory#6 and the uses in steps 18–32 and 33–48). The same Spin key was copied as judge_key.json for the participant's own validation run (trajectory#184, trajectory#185). Remote compute went through Spin (trajectory#56, trajectory#57). No step uses OPENAI_API_KEY or a harness *_BASE_URL in participant code: the only '_BASE_URL' match is validate.py's help text. No extra codex or claude CLI sessions were started.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed18m 21s into the runtook 12m 1s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: (empty)
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: (empty)
- 2passed1h 5m into the runtook 13m 13s
startup 111.1 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 111.1 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 200.0 s of server time; at that rate grading's 200 take 2,000 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 10.0 s, max 14.9 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with serve.sh.
- Offline H100 self-check passed: startup 111.1s, p95 14.80s, zero failures.
- Public validation: 25/25 valid images, OCR accuracy 99.79%, layout F1 96.28%.
- Combined public diagnostic score: 0.735.