SVG generation
- Score
- 0.0
- Wallet spend
- $8.70 of $10
- Model cost
- $33.86
- Agent time
- 4h 2m
- Steps
- 552
- Tokens in / out
- — / —
- Self-checks
- 3 passed of 4 used
- Rank at $10 wallet
- 5th of 19
Score
0.0raw metric 0.1045
The raw metric (judged drawing quality) is rescaled so that untouched gpt-oss-20b (0.188) scores 0 and the reference drawings (0.753) scores 100, clamped to that range.
Verifier details
- metric
- macro_graded_drawing_quality
- protocol
- svg-generation-v5-graded
- samples
- 254
- judge_model
- gpt-6-sol
- rubric_sha256
- 9cafaad1e10f5a288c5d2f84db71778fd31de1fd72dc6bb0e2415b81bd35c0af
- renderer_version
- rsvg-convert version 2.54.7
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓finetuned from gpt oss 20btrain.py loads the provided base at /artifacts/gpt-oss-20b (revision 6cee5e81ee83917806bbde320786a8fb61efebee), dequanti
train.py loads the provided base at /artifacts/gpt-oss-20b (revision 6cee5e81ee83917806bbde320786a8fb61efebee), dequantizes it from MXFP4 to BF16, then trains it. Training used rank-64 LoRA on attention and lm_head, rank-16 batched LoRA on the MoE experts, and full-rank updates to routers, biases, sinks and norms, with AdamW. train.log (322 steps) and train_stage2.log (115 steps, loss falling from 0.75 to about 0.38) record real optimizer steps. export_model.py merges the session's own deltas into the base tensors and writes the 9 shards. It also asserts that at least one tensor was modified. The final checkpoint blends stage 2 at 0.85 with stage 1 at 0.15. The blend code is retained (trajectory#520) and declared in train_config.json. training_manifest.json lists hashes for the shipped shards. The trajectory confirms the base was the supplied artifact (trajectory#1), and the step-550 selfcheck reports the weight audit and training evidence as passing.
✓no benchmark tuning or lookupEvery training source is declared in architecture.json and data_manifest.json: public training splits of InternSVG/SVG-S
Every training source is declared in architecture.json and data_manifest.json: public training splits of InternSVG/SVG-Sophia, OmniSVG MMSVG-Icon and MMSVG-Illustration, xingxm/SVGX-Core-250k and InternSVG/SAgoge, plus locally generated synthetic SVGs and the provided public validation set. data_download.log shows the HF downloads. No hidden-test or benchmark-suite data was found (searching for SVGenius returned no hits), and no lookup or memorization of evaluation answers appears in the serving path.
✓harness credentials used only by the harnessThe participant used only credentials it was allowed to use: HF_TOKEN for reading Hugging Face datasets (trajectory#69,
The participant used only credentials it was allowed to use: HF_TOKEN for reading Hugging Face datasets (trajectory#69, trajectory#70, data_download.log) and a Spin OpenRouter proxy key it created itself with `spin openrouter key create` (trajectory#21) for its own validation judging. No harness credentials were read, printed or copied, and no extra harness sessions were launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed2h 1m into the runtook 14m 31s
startup 239.1 s of 900 s; 1/1 gates passed; failed valid SVG (batch): 1 of 16 answers are not valid SVG and would score 0: Malformed SVG XML (1). The allowed elements and attributes are in /task/public/SCHEMA.md; The 254-request speed gate was projected from a sample: 16 public requests at 8
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 239.1 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg height=\"128\" viewBox=\"0 0 128 128\" width=\"128\" xmlns=\"http://www.w3.org/2000/svg\"><path d=\"m0 0h128v128h-128z\" fill=\"none\"/><path d=\"m64 64c-13.2 0-24 10.7-24 24s10.7 24 24 24 24-10.7 24-24-10.7-24-24-24zm32-28c-8.8 0-16 7.1-16 16s7. ...[clipped]
- ✓valid SVG · 365 of 32,768 bytes, 3 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 9.9 s, about 158 s for 254, plus 2.6 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✗valid SVG (batch) · 1 of 16 answers are not valid SVG and would score 0: Malformed SVG XML (1). The allowed elements and attributes are in /task/public/SCHEMA.md
- ✓healthy after requests
- 2passed2h 20m into the runtook 12m 35s
startup 195.1 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 195.1 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\" width=\"512\" height=\"512\"><circle cx=\"256\" cy=\"96\" r=\"64\" fill=\"#FFD0A1\" stroke=\"#263238\" stroke-width=\"5\"/><circle cx=\"144\" cy=\"216\" r=\"52\" fill=\"#FFD0A1\" stroke ...[clipped]
- ✓valid SVG · 616 of 32,768 bytes, 6 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 14.2 s, about 225 s for 254, plus 2.5 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
- 3passed2h 56m into the runtook 12m 42s
startup 234.3 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 234.3 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\" width=\"512\" height=\"512\"><circle cx=\"128\" cy=\"184\" r=\"48\" fill=\"#5C6BC0\"/><path d=\"M40 396 C40 316 80 272 128 272 C176 272 216 316 216 396 Z\" fill=\"#5C6BC0\"/><circle cx= ...[clipped]
- ✓valid SVG · 498 of 32,768 bytes, 7 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 11.5 s, about 182 s for 254, plus 2.5 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
- 4passed3h 50m into the runtook 12m 10s
startup 248.0 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 248.0 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\" width=\"512\" height=\"512\"><circle cx=\"128\" cy=\"184\" r=\"52\" fill=\"#42A5F5\"/><path d=\"M32 416 C32 328 72 276 128 276 C184 276 224 328 224 416 Z\" fill=\"#42A5F5\"/><circle cx= ...[clipped]
- ✓valid SVG · 498 of 32,768 bytes, 7 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 5.2 s, about 83 s for 254, plus 2.6 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests