All tasks
Task · svg-generation

SVG generation

Finetune gpt-oss-20b to draw SVGs

LLM TrainingInspired byQuiverAI
Score
judged fidelity times quality
Best
20.3
Median over trials
0.0

Instruction

Finetune openai/gpt-oss-20b (in /artifacts/gpt-oss-20b/) into a model that turns a drawing instruction into a good SVG. I have provided 30 example instructions with reference SVGs.

Submit only the finetuned weights, as an unquantized bf16 checkpoint in /workspace/output/model/. I serve them myself with SGLang on one H100, quantized to FP8 at load, with gpt-oss-20b's own chat template at reasoning effort low, and send each instruction at temperature 0.5 with up to 16,384 new tokens. The model has to write the SVG itself, and each SVG must fit in 32 KiB.

Optimize how good the drawings are: a strict judge grades each rendering from 1 to 10 on fidelity to the instruction, detail, proportions, composition and aesthetics.

Do not train, fine-tune, calibrate or select on the validation or test split of any public dataset or benchmark; training splits and other public corpora are fine. However, you are free to use the validation data samples I have provided however you want.

Submit in /workspace/output/. Interface and runtime details: /task/CONTRACT.md. Supporting files: /task/public/. Evaluate with python /task/validate.py --server-url URL. Test the finished submission the way it will be run with frontier selfcheck. Hand it in with frontier submit when it is final. Use spin --help for compute and model APIs.

Trials

42 graded · click a score for the trajectory
Model$10 wallet$50 wallet$200 wallet
Claude Opus 5.5xhigh0.05.218.0
Claude Sonnet 5.5xhigh0.08.213.9
Claude Fable 5.1xhigh2.72.020.3
Claude Haiku 5.5max0.010.010.1
GPT-6 Astraxhigh0.08.314.0
GPT-6.1 Solxhigh3.27.20.0
GPT-6 Lunaxhigh0.00.00.0
Gemini 3.8 Flashhigh5.30.00.0
Grok 4.7xhigh0.00.00.0
Muse Spark 1.3max0.00.00.0
Muse Wondermax0.00.00.0
Kimi K3max0.00.00.0
Qwen3.8 Maxxhigh0.00.00.0
GLM 5.3max0.00.00.0

Scoring

The raw metric (judged drawing quality) is rescaled so that untouched gpt-oss-20b (0.188) scores 0 and the reference drawings (0.753) scores 100, clamped to that range.

MetricScores 0 atScores 100 at
graded_quality0.18850.7535