Screenshot generation
- Score
- 22.4
- Wallet spend
- $14.03 of $50
- Model cost
- $13.50
- Agent time
- 2h 20m
- Steps
- 175
- Tokens in / out
- 34.3M / 140k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 6th of 14
Score
22.4raw metric 0.6303
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6303
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceNo component was trained, fine-tuned or calibrated. architecture.json, docs/PROVENANCE.md and logs/no_training.log all s
No component was trained, fine-tuned or calibrated. architecture.json, docs/PROVENANCE.md and logs/no_training.log all state that the only weight change was an FP8 dtype cast of public Z-Image-Turbo. The only task data used was the 25 shipped public examples in /task/public (trajectory#3), and only for qualitative and judge checks during development. Searches of the trajectory found no access to the forbidden inductionlabs/frontier-ml-tasks repository and no other undeclared corpus.
✓non specialist initializationThe weights come from the general text-to-image model Tongyi-MAI/Z-Image-Turbo, pinned to a declared revision. model_pre
The weights come from the general text-to-image model Tongyi-MAI/Z-Image-Turbo, pinned to a declared revision. model_preparation.py loads it with ZImagePipeline.from_pretrained and only casts the transformer and text_encoder to float8_e4m3fn before save_pretrained. model/model_index.json lists the standard Z-Image components (ZImageTransformer2DModel, Qwen3Model, AutoencoderKL). The transformer weight inventory has the expected Z-Image tensor layout (cap_embedder, context_refiner, all_x_embedder), stored in F8_E4M3. The trajectory shows no screenshot-specialist checkpoint. The general base Z-Image was downloaded once for an experiment (trajectory#157), and the shipped model remains Turbo (trajectory#169).
✓no reference image lookupapp.py builds every output from the request's description and text boxes. layout.make_canvas draws a procedural PIL canv
app.py builds every output from the request's description and text boxes. layout.make_canvas draws a procedural PIL canvas, Z-Image img2img refines it with a seed derived by hashing the request, detector.py uses RapidOCR only to find and erase invented text, and render.py draws the requested strings with the bundled fonts. serve.sh sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, the server makes no network calls, and no bundled screenshots or image index are loaded or retrieved.
✓actual inference pathserve.sh installs the bundled wheels offline and runs exec python3 app.py with MODEL_DIR=$DIR/model, MODEL_KIND=zimage a
serve.sh installs the bundled wheels offline and runs exec python3 app.py with MODEL_DIR=$DIR/model, MODEL_KIND=zimage and GEN_MODE=i2i. app.py loads ZImageImg2ImgPipeline from the shipped model/ directory onto CUDA. The POST /v1/images/generations handler runs canvas, then diffusion, then OCR erase, then text render, and returns a base64 PNG. The final self-check server log shows 'loading model kind=zimage mode=i2i ... dir=/submission/model' and weights being loaded (trajectory#171).
✓harness credentials used only by the harnessDevelopment judge calls used a key created with `spin openrouter key create` (trajectory#24) and pointed at the Spin pro
Development judge calls used a key created with `spin openrouter key create` (trajectory#24) and pointed at the Spin proxy, api.spin.inductionlabs.com (trajectory#47, trajectory#161), which the policy allows. Model downloads used the permitted HF_TOKEN. The trajectory shows no misuse of harness-only credentials, and the shipped serving code uses no credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 24m into the runtook 4m 47s
startup 30.0 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 30.0 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 43.7 s of server time; at that rate grading's 200 take 437 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 2.2 s, max 2.4 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed2h 14m into the runtook 4m 44s
startup 33.4 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 33.4 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 45.3 s of server time; at that rate grading's 200 take 453 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 2.3 s, max 2.6 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests