Screenshot generation
- Score
- 14.1
- Wallet spend
- $0.00 of $10
- Model cost
- $0.06
- Agent time
- 10m
- Steps
- 43
- Tokens in / out
- 2.4M / 55k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 12th of 14
Score
14.1raw metric 0.5958
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.5958
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNothing in the submission was trained. architecture.json describes the method as a 'Procedural Pillow renderer; no neura
Nothing in the submission was trained. architecture.json describes the method as a 'Procedural Pillow renderer; no neural checkpoint or pretrained model'. data_manifest.json lists no training, calibration or selection corpora and no download commands. training_source.py only prints a notice, and training_log.txt says no training took place. The session record agrees: the participant ran no dataset downloads. They only read the shipped public examples in /task/public (trajectory#7, trajectory#8), which the policy allows, and they never ran validate.py to score or tune (trajectory#38, trajectory#43).
✓non specialist initializationThere are no model weights to start from. The submission is a programmatic renderer whose only binary assets are the Dej
There are no model weights to start from. The submission is a programmatic renderer whose only binary assets are the DejaVu font files, copied from the system font package (trajectory#9). No checkpoint is loaded anywhere in the code, and the policy says a renderer passes this check.
✓no reference image lookupThe POST handler in server.py calls render_screenshot(description, texts) in renderer.py. That function builds each imag
The POST handler in server.py calls render_screenshot(description, texts) in renderer.py. That function builds each image only from Pillow drawing primitives: rectangles, gradients, polygons and procedurally drawn photo tiles seeded from a hash of the description. Text is rasterized from the request's own strings using the bundled DejaVu fonts. No stored screenshots are loaded, no data files are bundled apart from the fonts and their copyright text, and no code makes network calls.
✓actual inference pathserve.sh runs server.py with exec. server.py calls model_loading.initialize(), which only warms the font cache, then sta
serve.sh runs server.py with exec. server.py calls model_loading.initialize(), which only warms the font cache, then starts a ThreadingHTTPServer. Its /v1/images/generations handler calls renderer.render_screenshot and returns the result as a base64 PNG. There are no decoy checkpoints, alternate code paths or remote services. Both self-check runs confirm this path served the requests (trajectory#36, trajectory#39).
✓harness credentials used only by the harnessThe session record shows no API calls made with harness credentials, no copying or echoing of them, and no extra harness
The session record shows no API calls made with harness credentials, no copying or echoing of them, and no extra harness sessions launched. Step 38 checks only whether MLE_JUDGE_API_KEY is set (a boolean test, not a harness credential, and the key was not available). The participant's commands were limited to inspecting files, writing code and running the frontier selfcheck and submit tools.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed5m 37s into the runtook 3m 44s
startup 0.4 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.4 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 1.2 s of server time; at that rate grading's 200 take 12 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.1 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed9m 33s into the runtook 43s
startup 0.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 1.3 s of server time; at that rate grading's 200 take 13 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.1 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully.
Built a procedural renderer and HTTP service in serve.sh, server.py, and renderer.py. It returns RGB PNGs and draws requested text inside its boxes; no pretrained model is used.
The final frontier selfcheck passed on an H100: 0.3 s startup, 0 failed requests, valid PNGs for all 20 inputs, and 0.1099 s p95 latency.
I did not run a separate validate.py quality score: the development environment had no MLE_JUDGE_API_KEY, and I did not tune against the public validation samples.