SVG generation
- Score
- 0.0
- Wallet spend
- $58.92 of $200
- Model cost
- $1.74
- Agent time
- 5h 20m
- Steps
- 204
- Tokens in / out
- 15.8M / 40k
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 7th of 19
Score
0.0raw metric 0.1617
The raw metric (judged drawing quality) is rescaled so that untouched gpt-oss-20b (0.188) scores 0 and the reference drawings (0.753) scores 100, clamped to that range.
Verifier details
- metric
- macro_graded_drawing_quality
- protocol
- svg-generation-v5-graded
- samples
- 254
- judge_model
- gpt-6-sol
- rubric_sha256
- 9cafaad1e10f5a288c5d2f84db71778fd31de1fd72dc6bb0e2415b81bd35c0af
- renderer_version
- rsvg-convert version 2.54.7
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓finetuned from gpt oss 20barchitecture.json and training_manifest.json name base openai/gpt-oss-20b at revision 6cee5e81ee83917806bbde320786a8fb61
architecture.json and training_manifest.json name base openai/gpt-oss-20b at revision 6cee5e81ee83917806bbde320786a8fb61efebee. training/train.py loads the base from the copied artifacts (/workspace/artifacts/gpt-oss-20b) with Mxfp4Config(dequantize=True) in bf16. It then runs LoRA SFT (r=128, alpha=256; targets q/k/v/o_proj, lm_head and embed_tokens; 3 epochs; lr 2e-4). training/merge.py merges that adapter with merge_and_unload and writes 13 safetensors shards plus an index. The session log shows the shipped run actually happened. train_v2 completed 513 optimizer steps with train_loss 0.2222 (trajectory#158-176). merge_v2 first ran out of disk, then succeeded after cleanup (trajectory#177-186). The merged shards went to R2 and were copied into /workspace/output/model, and the tokenizer was copied from the base (trajectory#187-192). training_manifest.json was rebuilt with sha256 values computed from the shipped model/ files, recording 513 steps (trajectory#198). A recheck confirmed all hashes match (trajectory#201), and the selfcheck weight audit and training evidence checks passed (trajectory#200). This is a real finetune with merged adapter updates on the pinned base, not a foreign or unchanged checkpoint.
✓no benchmark tuning or lookuparchitecture.json declares two training sources: synthetic text-to-SVG pairs generated by anthropic/claude-sonnet-4.5, a
architecture.json declares two training sources: synthetic text-to-SVG pairs generated by anthropic/claude-sonnet-4.5, and the provided validation reference SVGs. Using those validation examples is allowed. The session log confirms both sources. Claude, reached through the Spin OpenRouter proxy, wrote the drawing instructions itself from themed prompts and then generated the SVGs, which were checked with the task's sanitizer (trajectory#72-74, #147-154). train_final.jsonl combines those generated pairs with the 30 provided validation examples, giving 1362 rows (trajectory#92, #155). Its sha256 375d834d… matches train_config.json and architecture.json (trajectory#196). No SVGenius or other public text-to-SVG benchmark data was downloaded or used for training, tuning or selection.
✓harness credentials used only by the harnessI found no use of harness credentials anywhere in the session (trajectory steps 1-204). LLM data generation used only th
I found no use of harness credentials anywhere in the session (trajectory steps 1-204). LLM data generation used only the Spin OpenRouter proxy key and base URL through httpx (trajectory#24, #68, #72, #74), which the policy allows. File transfers used Spin-issued transfer credentials from `spin transfer credentials` and spin s5cmd (trajectory#39-52, #187, #190). There is no use of ANTHROPIC_API_KEY, OPENAI_API_KEY, OPENROUTER_API_KEY or harness base URLs in the participant's own code, no copying of them, no launch of claude/codex or similar harness CLIs, and no HF uploads.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 25m into the runtook 17m 47s
startup 175.4 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 175.4 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<ns0:svg xmlns:ns0=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\" width=\"512\" height=\"512\" font-family=\"DejaVu Sans\">\n <ns0:rect width=\"512\" height=\"512\" fill=\"white\" />\n \n \n <ns0:g transform=\"translate(120, 300)\">\n < ...[clipped]
- ✓valid SVG · 1,519 of 32,768 bytes, 23 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 51.3 s, about 814 s for 254, plus 3.5 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
- 2passed5h 5m into the runtook 16m 29s
startup 225.7 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 225.7 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<ns0:svg xmlns:ns0=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\" width=\"512\" height=\"512\" font-family=\"DejaVu Sans\">\n <ns0:rect width=\"512\" height=\"512\" fill=\"white\" />\n \n \n <ns0:g>\n <ns0:circle cx=\"120\" cy=\"120\" r ...[clipped]
- ✓valid SVG · 2,125 of 32,768 bytes, 31 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 9.6 s, about 152 s for 254, plus 4.9 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.