SVG generation
- Score
- 0.0
- Wallet spend
- $9.89 of $10
- Model cost
- $24.19
- Agent time
- 3h 10m
- Steps
- 195
- Tokens in / out
- 52.2M / 416k
- Self-checks
- 1 passed of 2 used
- Rank at $10 wallet
- 4th of 14
Score
0.0raw metric 0.1196
The raw metric (judged drawing quality) is rescaled so that untouched gpt-oss-20b (0.188) scores 0 and the reference drawings (0.753) scores 100, clamped to that range.
Verifier details
- metric
- macro_graded_drawing_quality
- protocol
- svg-generation-v5-graded
- samples
- 254
- judge_model
- gpt-6-sol
- rubric_sha256
- 9cafaad1e10f5a288c5d2f84db71778fd31de1fd72dc6bb0e2415b81bd35c0af
- renderer_version
- rsvg-convert version 2.54.7
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓finetuned from gpt oss 20bThe shipped weights come from the task-provided gpt-oss-20b base, finetuned in this session. The task gave the base at /
The shipped weights come from the task-provided gpt-oss-20b base, finetuned in this session. The task gave the base at /artifacts/gpt-oss-20b (trajectory#1). The pinned revision 6cee5e81 was dequantized to bf16 on the Spin H100 box (trajectory#38, #43). LoRA SFT used rank 32, alpha 64, on q/k/v/o, run by training/box/train.py through training/box/run_pipeline.sh. Phase 1 ran 1384 steps with falling loss and ended with TRAIN_DONE (trajectory#108, #109, #140; logs/train.log). Phase 2 continued from the phase-1 adapter via training/box/run2.sh and stopped at 500 steps on its time budget (trajectory#155-#157, #165-#171; logs/train2.log). That gives 1384 + 500 = 1884, matching optimizer_steps in training_manifest.json. The participant downloaded the trained adapter2, which is distinct from the smoke adapter (sha256 checked at trajectory#146 and #173). training/merge_export.py merged it on CPU against /artifacts/gpt-oss-20b with PeftModel merge_and_unload (trajectory#174, #175, #183; logs/merge_export.log). The merge's sanity check showed the adapter-affected tensor changed and the embeddings did not. finalize.py then hashed the final model files into training_manifest.json (trajectory#122, #183), and nothing changed the weights between that and submission (trajectory#187-#194). The weight inventory shows the expected bf16 gpt-oss tensor layout (411 tensors). I found no other checkpoint or substituted weights.
✓no benchmark tuning or lookupThe training corpus is declared in architecture.json, training_manifest.json and training/train_config.json. It has 11,7
The training corpus is declared in architecture.json, training_manifest.json and training/train_config.json. It has 11,707 rows: 11,587 from procedural generators (training/gen/*.py via prep_data.py's template REGISTRY) and the 30 task-provided public dev examples copied 4 times, which is allowed. The trajectory shows no download of SVGenius or any other public text-to-SVG eval suite. Probes were hand-written. Model selection between adapter 1 and adapter2 used only the public validation examples and those hand-written probes (trajectory#117, #168, #177-#180). An attempt at local judging through the Spin proxy also ran only on public validation outputs, and it failed for lack of budget (trajectory#159-#165).
✓harness credentials used only by the harnessI found no participant use of the harness's API keys or base URL. The local judge calls went to api.spin.inductionlabs.c
I found no participant use of the harness's API keys or base URL. The local judge calls went to api.spin.inductionlabs.com with the Spin proxy key stored in .judge_key (trajectory#41, #160-#164), which the policy allows. Data and adapter transfers used Spin compute and Spin s5cmd transfer credentials (trajectory#125-#127, #140, #145, #172). Delegation used only the harness's built-in Agent subagents (trajectory#28). No external harness CLIs (claude, codex, gemini, etc.) were launched as workers.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed1h 49m into the runtook 4m 39sfree (stopped before any GPU time)
failed training evidence: Training evidence scan failed: Missing or escaping evidence path
2 gates
- ✓review prerequisites
- ✗training evidence · Training evidence scan failed: Missing or escaping evidence path
- 2failed1h 56m into the runtook 11m 14s
startup 246.2 s of 900 s; 1/1 gates passed; failed valid SVG (batch): 2 of 16 answers are not valid SVG and would score 0: SVG exceeds 32,768 UTF-8 bytes (1); Unsupported SVG attribute (1). The allowed elements and attributes are in /task/public/SCHEMA.md; The 254-request speed gate was proj
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 246.2 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg viewBox=\"0 0 512 512\" width=\"512\" height=\"512\"><g transform=\"translate(256,256)\"><circle cx=\"0\" cy=\"0\" r=\"200\" fill=\"#f0f0f0\"/><circle cx=\"-80\" cy=\"-60\" r=\"60\" fill=\"#ffb6c1\"/><circle cx=\"-80\" cy=\"60\" r=\"60\" fill=\"# ...[clipped]
- ✓valid SVG · 334 of 32,768 bytes, 7 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 64.2 s, about 1,019 s for 254, plus 2.3 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✗valid SVG (batch) · 2 of 16 answers are not valid SVG and would score 0: SVG exceeds 32,768 UTF-8 bytes (1); Unsupported SVG attribute (1). The allowed elements and attributes are in /task/public/SCHEMA.md
- ✓healthy after requests
- 3passed2h 59m into the runtook 10m 15s
startup 159.4 s of 900 s; 1/1 gates passed; The 254-request speed gate was projected from a sample: 16 public requests at 8 in flight within their share of the 10,800 s deadline, scaled to 254.
10 gates
- ✓review prerequisites
- ✓training evidence
- ✓weight audit
- ✓startup · GET /health answered 200 after 159.4 s
- ✓POST /v1/chat/completions · {"id": "validation-0000", "svg": "<svg viewBox=\"0 0 512 512\" width=\"512\" height=\"512\"><ellipse cx=\"126\" cy=\"444\" rx=\"66.6\" ry=\"7\" fill=\"#000000\" fill-opacity=\"0.13\"/><rect x=\"117.6\" y=\"351.2\" width=\"16.8\" height=\"58.8\" rx=\"8\" fill=\"#f0b28c\"/><path d=\"M58 ...[clipped]
- ✓valid SVG · 1,844 of 32,768 bytes, 25 elements, rendered by rsvg-convert version 2.54.7
- ✓254 requests, 8 in flight · projected from 16 of 254 requests at concurrency 8: 16 instructions drawn from the 30 public examples took 6.2 s, about 99 s for 254, plus 4.9 s for the first request. The public examples are simple icons only; detailed and complex held-out drawings usually need longer SVGs, so keep a wide margin
- ✓batch responses · all 16 answers parsed
- ✓valid SVG (batch) · all 16 answers pass the scorer's SVG schema check (not rendered)
- ✓healthy after requests
The agent's last message before its session ended.
Selfcheck passed on all gates (startup 159s/900s, valid-SVG batch, speed projection). Submitting: