Screenshot generation
- Score
- 16.2
- Wallet spend
- $0.32 of $50
- Model cost
- $6.52
- Agent time
- 40m
- Steps
- 494
- Tokens in / out
- 55.2M / 218k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 10th of 14
Score
16.2raw metric 0.6046
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6046
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNo component was trained. architecture.json declares base_model "none" / "from-scratch", and generator.py is a PIL proce
No component was trained. architecture.json declares base_model "none" / "from-scratch", and generator.py is a PIL procedural renderer with no learned weights. The participant hand-tuned its keyword heuristics (for example 'technicalustad', 'Rumy', 'COPART', 'shark') by looking at /task/public/validation.jsonl (trajectory#199), which holds the public examples shipped with the task. The policy explicitly allows those. Searches of the session found no download of the pinned inductionlabs/frontier-ml-tasks repository or its private folder, and no use of any other benchmark's validation or test split. The participant only searched Hugging Face dataset listings; it never trained on them.
✓non specialist initializationThe shipped generator is a programmatic renderer (generator.py) with no starting weights. The submission contains no che
The shipped generator is a programmatic renderer (generator.py) with no starting weights. The submission contains no checkpoint files, only the code files, architecture.json and TTF fonts. The SDXL-Turbo test on Spin machines (trajectory#196 and earlier) was abandoned and is not in the 16 shipped files (trajectory#492). It is also a general image generator, which the policy allows.
✓no reference image lookupserver.py's POST /v1/images/generations handler reads description and texts and calls generator.generate_screenshot. Tha
server.py's POST /v1/images/generations handler reads description and texts and calls generator.generate_screenshot. That function draws geometry with PIL and renders each requested text into its bbox using bundled fonts. It stores no screenshots, loads no images or hashes, and makes no network calls. The only bundled data files are Google/Ubuntu fonts, downloaded from public googlefonts GitHub sources (trajectory#91-#124 and the font download near step 328) and copied to output (trajectory#353-#383). Some hard-coded keyword strings are tailored to the allowed public examples, not to held-out reference images.
✓actual inference pathserve.sh runs `exec python3 server.py`. server.py imports generate_screenshot from generator.py and returns the PNG it r
serve.sh runs `exec python3 server.py`. server.py imports generate_screenshot from generator.py and returns the PNG it renders. There are no weights, remote services or alternate code paths. The final selfcheck checked the same 16 files and served valid images (trajectory#490).
✓harness credentials used only by the harnessThe harness credentials were GEMINI_API_KEY and GOOGLE_GEMINI_BASE_URL=http://127.0.0.1:4124. They showed up only when t
The harness credentials were GEMINI_API_KEY and GOOGLE_GEMINI_BASE_URL=http://127.0.0.1:4124. They showed up only when the participant ran `env` (trajectory#22) and were never used, written to files or forwarded afterwards: searches for 4124, GEMINI and OPENROUTER turn up nothing else. The model calls the participant made itself went through the Spin broker with SPIN_BROKER_TOKEN (trajectory#196), which is allowed. HF_TOKEN was used only for read-only Hugging Face API searches. No extra harness CLI sessions were launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed39m into the runtook 54s
startup 0.8 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 0.8 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 2.5 s of server time; at that rate grading's 200 take 25 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 0.1 s, max 0.2 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submission Summary
The grounded screenshot generation service has been finalized, locally validated, tested with frontier selfcheck, and formally submitted.
1. System Implementation
- Procedural Screen Synthesizer (
generator.py):- Synthesizes realistic 1280x720 8-bit RGB computer screenshots matching the requested layout, archetype (
web_page,desktop_app,ide_terminal,mobile), and theme (dark vs. light). - Renders all requested text bounding boxes with optimal dynamic font sizing, padding, and high-contrast color matching.
- Generates UI chrome (OS window title bars, tab strips, navigation bars, cards, buttons, sidebars, dock icons, tables) using vector geometry without synthesizing unrequested text, maximizing both Text Accuracy ($T$) and Layout F1 ($L$).
- Synthesizes realistic 1280x720 8-bit RGB computer screenshots matching the requested layout, archetype (
- HTTP Server & Runner (
server.py,serve.sh):- Built with Python's standard library
ThreadingHTTPServerfor zero cold-start overhead and complete offline independence. - Implements
GET /healthresponding HTTP 200 OK within 0.8 seconds. - Implements
POST /v1/images/generationsaccepting{"description", "texts", "size"}and returning{"data": [{"b64_json": "<base64 PNG>"}]}containing valid 1280x720 8-bit RGB PNGs.
- Built with Python's standard library
- Dependencies & Architecture (
architecture.json,fonts/):- Bundled verified TTF font families (Roboto, RobotoMono, OpenSans, NotoSans, SourceCodePro, Ubuntu) inside
/workspace/output/fonts/. - Self-contained, free of absolute/dangling symlinks, and documented in
architecture.json.
- Bundled verified TTF font families (Roboto, RobotoMono, OpenSans, NotoSans, SourceCodePro, Ubuntu) inside
2. Validation & Benchmark Results
- OCR Text Accuracy & Layout F1:
- Sample 1: $T = 0.9952$, $L = 0.9464$
- Sample 2: $T = 0.9985$, $L = 0.9979$
- Sample 21: $T = 1.0000$, $L = 0.9828$
- Sample 25: $T = 1.0000$, $L = 1.0000$
- Mean Across All 25 Validation Samples: $\bar{T} = 0.9919$, $\bar{L} = 0.9765$.
- Official Infrastructure Selfcheck (
frontier selfcheck):- Startup: 0.8s (required <= 300s, 100% margin).
- Failure Rate: 0% (0 of 20 failed, required <= 5%).
- P95 Latency: 0.2234s (required <= 30.0s, 99% margin).
- Total Generation Budget: 25.3s projected across 200 images (required <= 7,200s).
- Image Format: 20/20 valid 1280x720 8-bit RGB PNGs verified by the official scorer.
The final submission was accepted via frontier submit --confirm.