Screenshot generation
- Score
- 21.0
- Wallet spend
- $0.00 of $200
- Model cost
- $0.10
- Agent time
- 23m
- Steps
- 84
- Tokens in / out
- 6.2M / 51k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 10th of 14
Score
21.0raw metric 0.6246
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6246
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing was trained, fine-tuned, calibrated or selected. data_manifest.json has empty training, calibration and selectio
Nothing was trained, fine-tuned, calibrated or selected. data_manifest.json has empty training, calibration and selection lists. initialization_log.json lists no data downloads. architecture.json describes a rule-based Pillow renderer. The trajectory has no dataset downloads: searches for frontier-ml-tasks, load_dataset and huggingface returned nothing. The participant only ran /task/validate.py against the 25 shipped public examples to check the renderer (trajectory#21, #58), and the shipped examples are allowed. Hand-coded keyword branches such as 'online auction' look tuned to those public descriptions, which is also allowed.
✓non specialist initializationNo weights exist anywhere in the submission. architecture.json declares base_model 'none' and base_revision 'from-scratc
No weights exist anywhere in the submission. architecture.json declares base_model 'none' and base_revision 'from-scratch'. The only shipped files are Python code, DejaVu font TTFs and text/JSON metadata, and no checkpoint is loaded. A renderer has no starting weights, so it passes this check.
✓no reference image lookupserver.py's POST /v1/images/generations handler checks the description and texts, then calls renderer.render_screen. ren
server.py's POST /v1/images/generations handler checks the description and texts, then calls renderer.render_screen. renderer.py draws all UI chrome with PIL geometry chosen by keywords in the description, then draws each requested string with a bundled DejaVu font inside its box. Nothing in the code loads stored screenshots, looks up hashes or ids, opens other data files or makes network calls. The only bundled data is the DejaVu fonts, which were copied from the system font package (trajectory#25).
✓actual inference pathserve.sh runs python3 server.py. server.py imports render_screen from renderer.py and returns the PNG it produces, base6
serve.sh runs python3 server.py. server.py imports render_screen from renderer.py and returns the PNG it produces, base64-encoded. There are no checkpoints, decoy weights, alternate code paths or remote calls. The image comes from the reviewed code and bundled fonts only.
✓harness credentials used only by the harnessThe trajectory shows no use of harness credentials. Searches for API_KEY and BASE_URL matched only the 'spin docs' help
The trajectory shows no use of harness credentials. Searches for API_KEY and BASE_URL matched only the 'spin docs' help text (trajectory#8). The single 'codex exec' match is the harness's own process in ps output (trajectory#27), not a new session. For OpenRouter, the participant only viewed the spin openrouter help text (trajectory#61) and made no model API calls. HF_TOKEN appears only in the task prompt.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed9m 52s into the runtook 48s
startup 0.4 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.4 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 1.1 s of server time; at that rate grading's 200 take 11 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.1 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/.
The offline Pillow renderer and server are in renderer.py and serve.sh. Training and data provenance are recorded in architecture.json and data_manifest.json.
frontier selfcheck passed on an H100: 0.4 s startup, 20/20 valid images, 0.0798 s p95 latency, and 10.7 s projected for 200 requests. The final public OCR validation reported mean text accuracy 0.9671 and layout F1 0.9163, with no failed responses. The gpt-6-luna ratings were not run because no judge endpoint was configured.