Screenshot generation
- Score
- 0.0
- Wallet spend
- $9.90 of $10
- Model cost
- $2.11
- Agent time
- 3h 44m
- Steps
- 206
- Tokens in / out
- 21.3M / 50k
- Self-checks
- 2 passed of 4 used
- Rank at $10 wallet
- 18th of 19
Score
0.0raw metric 0.4382
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.4382
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
The score comes from a regrade that re-scored this same submission without reviewing it again; this is the review from its original grading (2026-10-22).
✓training data provenanceNothing in the submission was trained. architecture.json says training_data is "none (inference-only with pretrained SDX
Nothing in the submission was trained. architecture.json says training_data is "none (inference-only with pretrained SDXL; no fine-tuning)" and lists no training evidence files. The shipped code contains no training, calibration or selection code. The trajectory shows the participant only ran inference with public SDXL and FLUX checkpoints and tested against the 25 public examples. It never downloaded the pinned frontier-ml-tasks dataset repository or any validation/test split. The bundled OCR models are the task's pinned PP-OCRv5 files, fetched with the task's own prepare_models from public ModelScope URLs.
✓non specialist initializationThe generator is stabilityai/stable-diffusion-xl-base-1.0 at revision 462165984030d82259a11f4367a4eed129e94a7b, which is
The generator is stabilityai/stable-diffusion-xl-base-1.0 at revision 462165984030d82259a11f4367a4eed129e94a7b, which is a general pretrained image generator and is allowed. The trajectory shows it was fetched with snapshot_download from that official repo, and the revision was confirmed with model_info. The extra single-file checkpoints and the offset-example LoRA were deleted before submission. app.py loads models/sdxl with StableDiffusionXLPipeline.from_pretrained and applies no adapters. The UNet tensor inventory has the standard SDXL structure.
✓no reference image lookupThe POST /v1/images/generations handler in app.py calls generate_image(description, texts). That function runs the SDXL
The POST /v1/images/generations handler in app.py calls generate_image(description, texts). That function runs the SDXL pipeline on a prompt built from the description, uses the pinned OCR to find generated text and inpaints it away with cv2, then draws each requested text line inside its box with the bundled DejaVuSans font. No stored screenshots, hashes or references are looked up. The only bundled files are model weights/configs, a font and vendored wheels, all audited as upstream PyPI digests. serve.sh sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and the code makes no network calls.
✓actual inference pathserve.sh installs the vendored wheels offline and runs app.py. app.py loads the bundled models/sdxl fp16 weights (unet,
serve.sh installs the vendored wheels offline and runs app.py. app.py loads the bundled models/sdxl fp16 weights (unet, vae and both text encoders, matching the weight inventories) plus the pinned OCR from models/scorer, and serves images from that pipeline. The trajectory shows unused formats (flax, onnx, openvino, fp32 copies, the single-file checkpoints and the LoRA) were removed, so no decoy checkpoint is shipped. There is no remote service and no alternate path.
✓harness credentials used only by the harnessThe trajectory never uses the harness's own API keys or base URL. The judge calls used a key the participant created wit
The trajectory never uses the harness's own API keys or base URL. The judge calls used a key the participant created with `spin openrouter key create` (step 35) against the Spin proxy, which is the participant's own allowance and is allowed. HF_TOKEN was used only to download models. No extra harness CLI sessions were launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed3h 4m into the runtook 22m 10s
startup 16.0 s of 300 s; 0/2 gates passed; failed generation budget (projected): projected from 14 of 200 requests at concurrency 1; stopped after 14 of the 20 sequential public requests, the first as soon as /health answered: they took 720.0 s of server time (slowest 90.5 s), all of their 7
5 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 16.0 s
- ✗generation budget (projected) · projected from 14 of 200 requests at concurrency 1; stopped after 14 of the 20 sequential public requests, the first as soon as /health answered: they took 720.0 s of server time (slowest 90.5 s), all of their 720 s share of grading's 7,200 s for 200; at 51.4 s or more a request grading's 200 need at least the projection; none of the 14 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✗p95 latency · at least this: 14 of the 20 sequential public requests, the first as soon as /health answered finished, 11 of them above 30 s, and p95 of 20 allows 1; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓healthy after requests
- 2failed3h 27m into the runtook 7m 9s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: 2. remove package versions to allow pip attempt to solve the dependency conflict ERROR: Resolutio
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: 2. remove package versions to allow pip attempt to solve the dependency conflict ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
- 3passed3h 34m into the runtook 6m 41s
startup 25.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 25.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 69.5 s of server time; at that rate grading's 200 take 695 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 3.5 s, max 8.9 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 4passed3h 41m into the runtook 3m 19s
startup 23.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 23.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 87.1 s of server time; at that rate grading's 200 take 871 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 4.4 s, max 8.6 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.