SVG generation
- Score
- 0.0
- Wallet spend
- $9.81 of $10
- Model cost
- $0.89
- Agent time
- 3h 28m
- Steps
- 368
- Tokens in / out
- 35.2M / 282k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 5th of 19
Score
0.0raw metric 0.11
The raw metric (judged drawing quality) is rescaled so that untouched gpt-oss-20b (0.188) scores 0 and the reference drawings (0.753) scores 100, clamped to that range.
Verifier details
- metric
- macro_graded_drawing_quality
- protocol
- svg-generation-v5-graded
- samples
- 254
- judge_model
- gpt-6-sol
- rubric_sha256
- 9cafaad1e10f5a288c5d2f84db71778fd31de1fd72dc6bb0e2415b81bd35c0af
- renderer_version
- rsvg-convert version 2.54.7
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
The score comes from a regrade that re-scored this same submission without reviewing it again; this is the review from its original grading (2026-10-20).
✓finetuned from gpt oss 20bStage 1 started from openai/gpt-oss-20b at the pinned revision 6cee5e81. fetch_hf.py and fetch_base.sh download it with
Stage 1 started from openai/gpt-oss-20b at the pinned revision 6cee5e81. fetch_hf.py and fetch_base.sh download it with snapshot_download (HF_TOKEN, download only), with Spin R2 as a fallback. The base was loaded as MXFP4, dequantized to bf16 and checked against the expected tensor list. Training attached PEFT LoRA to attention q/k/v/o plus custom per-expert LoRA (train_sft.py, expert_lora.py). train.log records 104 optimizer steps with loss falling from about 1.31 to about 0.76, followed by a merge into 10 shards. Stage 2 continued from the stage-1 merged checkpoint (/home/sesterce/run2/base was symlinked to run1/merged, trajectory#255) for 85 more steps (run2_train.log). That is 189 steps total, matching training_manifest.json. The stage-2 merge was downloaded as model2 (download2.log) and copied into /workspace/output/model at trajectory#322. verify_model reported 411/411 expected tensors, a tokenizer identical to the base, and MODEL_OK. build_output.py then regenerated the manifest hashes from these files (trajectory#343-358). No evidence shows a different base or substituted weights. I did not compare the manifest hashes against the weight inventories directly, but nothing contradicts them.
✓no benchmark tuning or lookupAll training data provenance is declared. data_manifest.json, architecture.json and training_manifest.json describe corp
All training data provenance is declared. data_manifest.json, architecture.json and training_manifest.json describe corpora a, b and c, distilled in-session from the teacher openai/gpt-6-luna through the Spin OpenRouter endpoint, using topics the teacher itself generated (gen_topics.py, gen_pairs.py). The 30 public dev examples from /task/public/validation.jsonl are also used, which the policy allows. Teacher distillation is likewise allowed. Searching the submission and trajectory found no reference to SVGenius, no load_dataset calls and no download of any public text-to-SVG benchmark or dataset. Topic categories are generic lists of everyday objects, food and animals.
✓harness credentials used only by the harnessThe trajectory shows only these credentials in use: - A Spin OpenRouter key from `spin openrouter key create` (trajector
The trajectory shows only these credentials in use: - A Spin OpenRouter key from `spin openrouter key create` (trajectory#15), used against api.spin.inductionlabs.com through orc.py (trajectory#16). - Spin wallet R2 session credentials for file transfer (upload_merged.py, fetch_r2.sh). - HF_TOKEN, used only to download the pinned base (fetch_hf.py). Every upload in the trajectory goes to Spin compute or R2; nothing is pushed to HF. Searches found no use of ANTHROPIC or other harness keys, no printenv dumps, no direct api.openrouter.ai calls, and no claude or codex CLI launched as a worker.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1failed1h 45m into the runtook 4m 45sfree (stopped before any GPU time)
failed training evidence: Training evidence scan failed: Missing or escaping evidence path
2 gates
- ✓review prerequisites
- ✗training evidence · Training evidence scan failed: Missing or escaping evidence path
- 2passed1h 54m into the runtook 12m 20s
startup 239.9 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 239.9 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 1024 1024\" width=\"512\" height=\"512\"><g fill=\"white\" stroke=\"black\" stroke-width=\"24\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M320 350C320 320 345 300 375 300C405 3 ...[clipped]
- ✓valid SVG · 789 of 32,768 bytes, 8 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 27.4 s, about 436 s for 254, plus 2.8 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
- 3passed3h 3m into the runtook 13m 52s
startup 229.5 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 229.5 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 1024 1024\" width=\"512\" height=\"512\"><g fill=\"white\" stroke=\"black\" stroke-width=\"24\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M320 350C320 320 345 300 375 300C405 3 ...[clipped]
- ✓valid SVG · 1,176 of 32,768 bytes, 11 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 50.7 s, about 804 s for 254, plus 3.6 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
- 4passed3h 18m into the runtook 11m 32s
startup 235.4 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 235.4 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 1024 1024\" width=\"512\" height=\"512\"><g fill=\"white\" stroke=\"black\" stroke-width=\"24\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><path d=\"M320 350C320 320 345 300 375 300C405 3 ...[clipped]
- ✓valid SVG · 1,176 of 32,768 bytes, 11 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 50.7 s, about 805 s for 254, plus 3.6 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.
What was delivered — /workspace/output/ (62 files, 41.87 GB):
model/— the stage-2 fine-tuned checkpoint: 10 bf16 safetensors shards holding exactly gpt-oss-20b's 411 tensors (names/shapes/dtypes verified against/task/public/serving/tensors.json),model.safetensors.index.json, andtokenizer.jsoncopied byte-identical from/artifacts/gpt-oss-20b/. Weights were confirmed to differ from the base (e.g.model.layers.0.self_attn.q_proj.weight).training_manifest.json— base model + pinned revision6cee5e81…,optimizer_steps: 189, 6 training sources each with a 64-hex digest of the corresponding corpus file, and evidence incode/config/log/data/data_logs/environment/post_training_checks; 12 weight entries with matching sha256.architecture.json— base model/revision, weight format, method, 46 training-evidence paths, training-data provenance (no public benchmark splits), and the base checkpoint's shard digests.training/— the actual training code, config, per-stage logs (train.log,run2_train.log), the raw and validated corpora, and the probe/stress generations.
How it was trained — two LoRA SFT stages (r=32 on q/k/v/o + custom per-expert low-rank adapters, 192.9 M trainable params = 0.91%) on teacher-distilled (instruction → SVG) pairs generated in-session with gpt-6-luna through spin and validated with the task's own renderer, plus the 30 public dev examples. Stage 1: 104 steps on 1665 examples; stage 2: 85 steps on 2019 examples including 177 many-element documents added to fix the repetition-loop failures seen on detailed prompts.
Verification — final selfcheck 60140f386656 PASSED every gate on the exact submitted bytes (digest 7bce6b82…): training evidence, weight audit, startup 235.4 s of 900 s, valid SVG rendered by rsvg-convert 2.54.7, 254-request batch projected at 809 s of 10,800 s, all 16 public answers parsed and schema-valid, server healthy afterwards.