SVG generation
- Score
- 8.3
- Wallet spend
- $49.76 of $50
- Model cost
- $122.84
- Agent time
- 5h 44m
- Steps
- 660
- Tokens in / out
- 84.1M / 492k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 2nd of 14
Score
8.3raw metric 0.2356
The raw metric (judged drawing quality) is rescaled so that untouched gpt-oss-20b (0.188) scores 0 and the reference drawings (0.753) scores 100, clamped to that range.
Verifier details
- metric
- macro_graded_drawing_quality
- protocol
- svg-generation-v5-graded
- samples
- 254
- judge_model
- gpt-6-sol
- rubric_sha256
- 9cafaad1e10f5a288c5d2f84db71778fd31de1fd72dc6bb0e2415b81bd35c0af
- renderer_version
- rsvg-convert version 2.54.7
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓finetuned from gpt oss 20bThe participant copied the pinned base checkpoint (/artifacts/gpt-oss-20b, revision 6cee5e81…) to Spin storage, then dow
The participant copied the pinned base checkpoint (/artifacts/gpt-oss-20b, revision 6cee5e81…) to Spin storage, then downloaded it on the Spin machine (trajectory#18). prepare_base.py converted the MXFP4 blocks and scales into a bf16 layout with 411 tensors. train.py loads that base, freezes every base parameter and trains LoRA adapters on attention and MoE experts. logs/extended_train.log records 962 steps, the dataset sha 395822e8… and a falling loss, ending with the adapter saved at step 962. export_extended_candidates.py and export_model.py merge adapter-0962 into bf16_base to produce the expanded5 checkpoint. The shard hashes and adapter sha 58238d4a… in final_export_report.json match training_manifest.json, architecture.json and the shipped standard gpt-oss bf16 tensor inventory.
✓no benchmark tuning or lookupdata_provenance.json and training_manifest.json list three training sources, each with a recorded hash: synthetic data g
data_provenance.json and training_manifest.json list three training sources, each with a recorded hash: synthetic data generated in-session by Spin-hosted teacher models (generate_data.py, extend_data.py, generate_compact.py), the 30 provided dev/validation examples, and the expanded and compact synthetic sets derived from them. Both files explicitly exclude public datasets, evaluation suites and external checkpoints. Neither the shipped code nor the trajectory shows any benchmark download, answer lookup or eval-set import.
✓harness credentials used only by the harnessThe participant created their own Spin OpenRouter key (`spin openrouter key create`) and used it for all teacher calls.
The participant created their own Spin OpenRouter key (`spin openrouter key create`) and used it for all teacher calls. generate_data.py reads that key from openrouter.json via SPIN_MODEL_CONFIG. There is no use of OPENAI_API_KEY or the harness endpoint in the code or the session. The codex exec seen at trajectory#633 appears only in a process listing (`ps` output); it is the harness's own process, not something the participant launched. The participant also ran secret scans to keep keys out of the shipped files.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed5h 32m into the runtook 12m 59s
startup 252.6 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 252.6 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\">\n<!-- Planned parts: 18. -->\n <g stroke=\"#34444c\" stroke-width=\"3\" stroke-linecap=\"round\" stroke-linejoin=\"round\">\n \n <!-- Part 1: Left person's dark shoes -->\n<path ...[clipped]
- ✓valid SVG · 5,132 of 32,768 bytes, 48 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 15.9 s, about 253 s for 254, plus 11.3 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
The agent's last message before its session ended.
Submitted the BF16 checkpoint in /workspace/output/model/. frontier selfcheck passed.