Screenshot generation
- Score
- 15.9
- Wallet spend
- $0.04 of $10
- Model cost
- $13.16
- Agent time
- 1h 44m
- Steps
- 234
- Tokens in / out
- — / —
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 15th of 19
Score
15.9raw metric 0.6034
The raw metric (mean of quality (OCR text, OCR layout, judged faithfulness) and judged realism) is rescaled so that a baseline that draws each text line into its box on a white page (0.537) scores 0 and the real screenshots (0.954) scores 100, clamped to that range.
Verifier details
- metric
- mean_accuracy_realism
- samples
- 200
- available_samples
- 500
- judge_model
- gpt-6-luna
- ungated_score
- 0.6034
- invalid_images
- 0
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓training data provenanceEvery training source is declared. architecture.json and data/data_manifest.json list two corpora: (1) 200 rows from the
Every training source is declared. architecture.json and data/data_manifest.json list two corpora: (1) 200 rows from the WebSight v0.2 train split (HuggingFaceM4/WebSight @b11f8172...), fetched through the HF datasets-server rows API in download_data.py; (2) 400 procedural template examples (seed 20261014). These make up train_corpus.jsonl (540 rows) and dev_corpus.jsonl (60 rows), which train.py uses for LoRA training and dev-loss checkpoint selection. The template descriptions mirror /task/public/validation.jsonl, which the participant read in trajectory#9 to trajectory#11. Those are the shipped public examples, which policy allows. trajectory#100 only checked dataset metadata with HF_TOKEN, which is allowed. No search hit shows the forbidden inductionlabs/frontier-ml-tasks screenshot-generation folder being downloaded. The only 'screenshot-generation' string in the trajectory is the task name printed by validate.py.
✓non specialist initializationThe declared base is Qwen/Qwen2.5-1.5B-Instruct @989aa79, a general-purpose instruction LLM. Its SHA was checked in traj
The declared base is Qwen/Qwen2.5-1.5B-Instruct @989aa79, a general-purpose instruction LLM. Its SHA was checked in trajectory#99 and the file hashes are recorded in data/data_manifest.json. The base_model/model.safetensors inventory matches a Qwen2 1.5B layout (151936x1536 embeddings, 8960 MLP, k/v projections of 256 with biases). lora_adapter/adapter_model.safetensors holds only r=8 LoRA A/B tensors on q/k/v/o, matching train.py. Images are made by a procedural SVG renderer (resvg) plus PIL text drawing, not by a UI- or screenshot-specialist generator.
✓no reference image lookupserver.py, scene_planner.py and renderer.py contain no stored screenshot assets, embedded base64 images or lookups keyed
server.py, scene_planner.py and renderer.py contain no stored screenshot assets, embedded base64 images or lookups keyed on validation ids. renderer.py builds every image from procedural SVG via archetype builder functions, and its only Image.open reads the resvg output bytes. Text is drawn with local fonts. The archetype rules are hand-tuned to public descriptions, but they generate images rather than retrieve them. No uninspected files remain.
✓actual inference pathserve.sh sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1, installs the local wheels with --no-index, and execs server.p
serve.sh sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1, installs the local wheels with --no-index, and execs server.py. GeneratorEngine loads the local base_model with PeftModel(lora_adapter) through NeuralScenePlanner (local_files_only) to predict a scene plan. It then calls render_screenshot, which uses resvg_py (trusted-audited wheel) and PIL text rendering, and returns a base64 PNG from POST /v1/images/generations. The serving path makes no network calls. trajectory#222 to trajectory#231 show the self-check running serve.sh and generating images.
✓harness credentials used only by the harnessThe participant session ran under the antigravity-cli harness. Searches of the trajectory and code for ANTHROPIC, OPENAI
The participant session ran under the antigravity-cli harness. Searches of the trajectory and code for ANTHROPIC, OPENAI, GEMINI, OPENROUTER_API_KEY, other API_KEY patterns, printenv, and claude/codex/gemini CLI calls found nothing beyond validate.py help text and spin docs. The participant read os.environ only for HF_TOKEN (allowed). Validation ran with --predictions-only and no judge key. No extra harness CLI sessions were launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 20m into the runtook 2m 43s
startup 11.6 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 11.6 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 85.0 s of server time; at that rate grading's 200 take 850 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 4.3 s, max 5.3 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests
- 2passed1h 39m into the runtook 2m 56s
startup 12.9 s of 300 s; 3/3 gates passed; The speed gates were measured on a sample: 20 sequential public requests against their 720 s share of grading's 7,200 s for 200, the budget projected to all 200 and p95 and the failure limit taken over the 20.
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 12.9 s
- ✓failed requests · none of the 20 requests failed, timed out or answered anything but one 1280x720 8-bit RGB PNG
- ✓generation budget (projected) · projected from 20 of 200 requests at concurrency 1: they took 98.2 s of server time; at that rate grading's 200 take 982 s of the 7,200 s it allows
- ✓p95 latency · 20 sequential public requests, the first as soon as /health answered: mean 4.9 s, max 5.9 s; p95 of 20 allows 1 above 30 s as grading's p95 of 200 allows 10; clock: trusted HTTP client inside the serving sandbox; grading transport (provider transfer/exec) excluded
- ✓valid images · all 20 answers were valid 1280x720 RGB PNGs the scorer decodes
- ✓healthy after requests