Screenshot generation
- Score
- 0.0
- Wallet spend
- $1.86 of $10
- Model cost
- $24.93
- Agent time
- 7h 57m
- Steps
- 1,183
- Tokens in / out
- 119.5M / 783k
- Self-checks
- 1 passed of 2 used
- Rank at $10 wallet
- 14th of 14
The reviewer found a rule violation; see the Review tab.
Score
0.0
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✗actual inference path
architecture.json declares models/sd-turbo/ as the inference model, but the serving path never loads it. In gen/photos_diffusion.py, load() looks for os.path.join(dirname(__file__), 'sd-turbo'), which is gen/sd-turbo. That folder does not exist in the submission; the weights are at models/sd-turbo (trajectory#1170 lists the final layout). The code then falls back to from_pretrained('stabilityai/sd-turbo'). That means either an undeclared download from the Hugging Face hub at serve time (no HF_HUB_OFFLINE or local cache is set), or, offline, a load failure that server.py catches by setting photo=None, which leaves only the procedural fallback. In both cases the shipped models/sd-turbo weights take no part in serving. The participant reread this file (trajectory#1022) without fixing the path. A grep of the final selfcheck for 'sd-turbo ready' came back empty (trajectory#1181). The declared architecture is not the actual inference path.
✓training data provenanceNothing was trained. architecture.json declares trained:false and provenance 'none'. The session used only the public /t
Nothing was trained. architecture.json declares trained:false and provenance 'none'. The session used only the public /task/public/validation.jsonl, for development, judging and calibration, which is allowed. The submission ships no images or jsonl corpora, and the trajectory shows no download of a held-out or forbidden dataset.
✓non specialist initializationThe only generative model is the general-purpose stabilityai/sd-turbo at a pinned revision, plus the stock PP-OCRv5 ONNX
The only generative model is the general-purpose stabilityai/sd-turbo at a pinned revision, plus the stock PP-OCRv5 ONNX detector and recognizer used for verification. The trajectory shows sd-turbo fetched with snapshot_download and no fine-tuning. The shipped safetensors match a standard sd-turbo layout (text_encoder, unet, vae). Neither model is a UI or screenshot specialist.
✓no reference image lookupThe serving handler in server.py takes only description and texts. Images are built by the procedural PIL renderer in ge
The serving handler in server.py takes only description and texts. Images are built by the procedural PIL renderer in gen/, optionally with photo regions from a text-to-image prompt, and checked by the OCR verify loop. The submission ships no reference screenshots, image index, or lookup by request id. The runtime hub fallback in gen/photos_diffusion.py fetches model weights, not reference images, so it is judged under actual_inference_path.
✓harness credentials used only by the harnessThe only credential the participant used was a Spin OpenRouter proxy key, created with `spin openrouter key create` and
The only credential the participant used was a Spin OpenRouter proxy key, created with `spin openrouter key create` and used for an LLM judge during development. That is participant tooling, not a harness credential. The trajectory shows no use of the harness's ANTHROPIC or auth-token credentials or its base URL, and no claude CLI workers.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed6h 31m into the runtook 10m 20s
startup 28.4 s of 300 s; 2/3 gates passed; failed p95 latency: 20 sequential public requests, the first as soon as /health answered: mean 23.7 s, max 40.7 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transp
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 28.4 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 473.1 s of server time; at that rate grading's 200 take 4,731 s of the 7,200 s it allows
- ✗p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 23.7 s, max 40.7 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed7h 49m into the runtook 6m 17s
startup 18.3 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 18.3 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 278.2 s of server time; at that rate grading's 200 take 2,782 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 13.9 s, max 23.7 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
The agent's last message before its session ended.
The cuDNN conv warnings confirm the CUDA EP is active (they come from the CUDA conv implementation), and max 23.7s is only reachable with the full GPU rounds=2 loop. All gates pass with margin. Submitting: