Screenshot generation
- Score
- 35.2
- Wallet spend
- $7.69 of $200
- Model cost
- $74.52
- Agent time
- 6h 40m
- Steps
- 588
- Tokens in / out
- 215.3M / 374k
- Self-checks
- 3 passed of 3 used
- Rank at $200 wallet
- 6th of 14
Score
35.2raw metric 0.6837
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6837
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing in the submission is a trained model. architecture.json and evidence/data_provenance.md say so, and no training
Nothing in the submission is a trained model. architecture.json and evidence/data_provenance.md say so, and no training appears in the trajectory. The renderer's settings were tuned by rendering and scoring the shipped public examples in /task/public/validation.jsonl (trajectory#301 and many other steps). The policy allows this. One caveat: BUILD_LOG.md says the tuning was not done on the validation set, which is inaccurate, but that set is the shipped public split, so no rule is broken. The bundled stock images came from FLUX.1-schnell with hand-written prompts (evidence/gen_assets.py, gen_assets2.py) and were filtered with the pinned OCR (trajectory#182-189). Fonts were downloaded from Google Fonts (trajectory#17-19). The session never accesses inductionlabs/frontier-ml-tasks, setup.json, any private or held-out data, or any dataset loader.
✓non specialist initializationThe served generator is a hand-written PIL renderer with no weights, and serve.sh loads no model. FLUX.1-schnell, a gene
The served generator is a hand-written PIL renderer with no weights, and serve.sh loads no model. FLUX.1-schnell, a general image generator that the policy allows, was used offline only to make text-free stock images (trajectory#190, evidence/gen_assets.py). No checkpoint is shipped and the weight inventory is empty.
✓no reference image lookupThe handler in app/server.py passes the request's description and text boxes to engine.render_sample. engine.py uses a S
The handler in app/server.py passes the request's description and text boxes to engine.render_sample. engine.py uses a SHA-256 of the request only to seed the random generator. It picks stock images from the FLUX-made assets/images/manifest.json by category tag (e.g. car, landscape), not by screenshot id or text. The code has no network calls (no urllib, socket or HTTP client). The bundled files are fonts, the icon font and its codepoint map, and the FLUX stock JPEGs. The trajectory shows them created from FLUX output (trajectory#187-189), not copied from /task/public/media or the held-out data.
✓actual inference pathserve.sh runs app/server.py. On each POST, server.py calls generate(), which calls renderer.engine.render_sample. If tha
serve.sh runs app/server.py. On each POST, server.py calls generate(), which calls renderer.engine.render_sample. If that fails, it falls back to a simple PIL render of the request text. No model, remote service, decoy checkpoint or alternate path exists. Everything runs locally with PIL using bundled fonts, icons and images.
✓harness credentials used only by the harnessThe harness ran kimi-code through an OpenRouter-backed model. The participant's commands never read or use ANTHROPIC, OP
The harness ran kimi-code through an OpenRouter-backed model. The participant's commands never read or use ANTHROPIC, OPENAI or OPENROUTER API keys or harness base URLs. The validation judge used a key from `spin openrouter key create` and the Spin API (trajectory#298-299), which is the participant's own allowance. HF_TOKEN was used only to download FLUX. No extra harness CLI sessions were started.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed4h 22m into the runtook 45s
startup 0.2 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.2 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 2.3 s of server time; at that rate grading's 200 take 23 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.4 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed4h 56m into the runtook 2m 4s
startup 2.5 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.5 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 3.2 s of server time; at that rate grading's 200 take 32 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.2 s, max 0.5 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 3passed6h 25m into the runtook 1m 49s
startup 2.8 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 2.8 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 2.7 s of server time; at that rate grading's 200 take 27 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.4 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Everything passes. Submitting: