Screenshot generation
- Score
- 32.7
- Wallet spend
- $4.86 of $10
- Model cost
- $97.78
- Agent time
- 7h 35m
- Steps
- 583
- Tokens in / out
- 283.2M / 479k
- Self-checks
- 4 passed of 4 used
- Rank at $10 wallet
- 6th of 14
Score
32.7raw metric 0.6732
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6732
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing in the shipped system is trained. architecture.json and evidence/NO_TRAINING.md say no component was trained, fi
Nothing in the shipped system is trained. architecture.json and evidence/NO_TRAINING.md say no component was trained, fine-tuned, calibrated or selected on a dataset (training_corpora is empty), and the shipped code is a hand-written rule-based PIL renderer with no weights. The trajectory shows the hand-written rules were iterated against /task/public/validation.jsonl, the task's own shipped public examples, which the contract allows. The documentation is slightly incomplete here: it describes tuning on an author-written dev set and validate.py. A search of the session found no access to the inductionlabs/frontier-ml-tasks repository, setup.json, the held-out set or hf_hub. The only runtime assets are OFL fonts, downloaded from google/fonts with curl in trajectory#23 or nearby (the step falls in the steps-23-42 chunk), matching the commands recorded in architecture.json.
✓non specialist initializationNo weights are shipped. The submission is a programmatic renderer (planner.py, render.py, widgets.py, icons.py, extras.p
No weights are shipped. The submission is a programmatic renderer (planner.py, render.py, widgets.py, icons.py, extras.py, textfit.py) using only PIL and numpy, and a renderer passes this check. An SDXL-Turbo diffusion integration (a general image generator, not a screenshot specialist) was tried on a rental machine (trajectory#525-#541). It was dropped after an A/B test ('Diffusion A/B test -> keep procedural', trajectory#581). The final file list in trajectory#577 contains no model or diffuse.py, and no shipped source refers to diffusion.
✓no reference image lookupserver.py parses the description and text boxes and calls render.render(), which builds a plan with planner.py's regex a
server.py parses the description and text boxes and calls render.render(), which builds a plan with planner.py's regex and keyword rules and draws everything procedurally (vector widgets, a hand-drawn icon library, fonts). There is no Image.open call and no file reading except the fonts in textfit.py. There are no network imports, and no strings, ids or hashes from validation samples are embedded in the source (searched for 'Copart' and 'validation-'). The only bundled data files are four OFL font files.
✓actual inference pathserve.sh sets PYTHONPATH to src and execs src/server.py. That is a stdlib ThreadingHTTPServer whose POST /v1/images/gene
serve.sh sets PYTHONPATH to src and execs src/server.py. That is a stdlib ThreadingHTTPServer whose POST /v1/images/generations handler calls R.render(description, texts) and returns the PNG; if an error occurs it returns a blank fallback PNG. There are no checkpoints, decoy models, alternate paths or remote calls. The final file list (trajectory#577) matches the shipped evidence.
✓harness credentials used only by the harnessThe model API key used for judge calls was created by the participant with 'spin openrouter key create' (trajectory#18)
The model API key used for judge calls was created by the participant with 'spin openrouter key create' (trajectory#18) and used against the Spin OpenRouter proxy (trajectory#19-#22 and later score runs). That is the participant's own allowance and is allowed. The kimi CLI and its provider-pin proxy appear only incidentally in a pgrep listing (trajectory#46); no extra harness session was launched. The searches for OPENROUTER, harness *_API_KEY or *_BASE_URL variables, and printenv found no step where the participant used or copied the harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed2h 57m into the runtook 1m 8s
startup 1.2 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.2 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 5.3 s of server time; at that rate grading's 200 take 53 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.3 s, max 0.4 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed5h 54m into the runtook 50s
startup 0.6 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.6 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 6.6 s of server time; at that rate grading's 200 take 66 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.3 s, max 0.6 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 3passed7h 14m into the runtook 1m 7s
startup 1.2 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.2 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 6.7 s of server time; at that rate grading's 200 take 67 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.3 s, max 0.6 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 4passed7h 33m into the runtook 44s
startup 0.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 5.5 s of server time; at that rate grading's 200 take 55 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.3 s, max 0.5 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Selfcheck #3 PASSED. Final state is solid (score 0.7642, all requirements green). Submitting: