Screenshot generation
- Score
- 39.2
- Wallet spend
- $0.07 of $200
- Model cost
- $9.28
- Agent time
- 1h 52m
- Steps
- 236
- Tokens in / out
- 26.5M / 193k
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 6th of 19
Score
39.2raw metric 0.7005
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.7005
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing in this submission was trained, fine-tuned or calibrated. It is a hand-written procedural renderer, and architec
Nothing in this submission was trained, fine-tuned or calibrated. It is a hand-written procedural renderer, and architecture.json, evidence/no_training.json and assets/ASSETS.md all say so. The only data used to develop and select the renderer were the 25 example screenshots shipped in /task/public, and the policy allows those; the 'validation-00xx' ids in the trajectory are those public examples. The session shows no download of the pinned dataset repository (inductionlabs/frontier-ml-tasks) or its private folder, and no use of any other benchmark's validation or test split. The only network downloads were Google Fonts files from a pinned commit (trajectory#9), the Pillow wheel from PyPI, and the scorer's OCR models fetched with validate.py --prepare-models (trajectory#7), which were used only for local evaluation.
✓non specialist initializationThere are no starting weights. architecture.json declares base_model 'none', and the shipped files are only Python code,
There are no starting weights. architecture.json declares base_model 'none', and the shipped files are only Python code, TrueType fonts and an unmodified Pillow wheel whose digest a trusted audit approved. No checkpoint is shipped or loaded. A renderer has no initialization, so it passes this check.
✓no reference image lookupThe request handler in server.py calls renderer.generate(description, texts, size), which builds a Renderer object. That
The request handler in server.py calls renderer.generate(description, texts, size), which builds a Renderer object. That object draws the page with PIL vector shapes (rectangles, polygons, icons, seeded geometric illustrations) and then draws only the request's own strings into their boxes with text_render.draw_text_box. The only files the code reads are the bundled fonts. No screenshots, image data, embeddings or hashes of reference pages are bundled. The SHA-256 call is used only to seed the random generator. The code makes no network calls. Some layout templates were tuned to descriptions of the public examples (for example 'online auction' or 'backup'), but they are procedural drawing code, not stored pixels.
✓actual inference pathserve.sh installs the vendored Pillow wheel only if PIL is missing, then runs server.py. server.py imports renderer.gene
serve.sh installs the vendored Pillow wheel only if PIL is missing, then runs server.py. server.py imports renderer.generate and text_render, and its fallback also uses only draw_text_box on a blank canvas. There are no model weights, decoy checkpoints, alternate code paths or remote services. The reviewed code is exactly what produces the PNG.
✓harness credentials used only by the harnessThe participant called the judge model with keys from `spin openrouter key create` (trajectory#25, and the run_judge.py
The participant called the judge model with keys from `spin openrouter key create` (trajectory#25, and the run_judge.py wrapper in trajectory#147-157), sent to the Spin proxy at api.spin.inductionlabs.com. That is the participant's own allowance, which the policy permits. No step uses the harness's own API keys or base URL, writes them to files, or launches another coding-agent CLI. The participant also grepped the output folder for credential strings before submitting (trajectory#130-139, trajectory#207-224).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 17m into the runtook 36s
startup 0.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 1.6 s of server time; at that rate grading's 200 take 16 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.1 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed1h 49m into the runtook 52s
startup 0.8 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.8 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 1.4 s of server time; at that rate grading's 200 take 14 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.1 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Final development aggregate: T 0.99636, L 0.95842, R 0.72889, F 0.95556, for 0.84941 overall—up 0.01405 from v4. All required checks are complete, and /workspace/output/ remains the exact selfcheck-passed package. I’m starting the requested two-step submission.