Figure to TikZ
- Score
- 36.0
- Wallet spend
- $2.70 of $10
- Model cost
- $17.69
- Agent time
- 1h 57m
- Steps
- 114
- Tokens in / out
- 28.1M / 66k
- Self-checks
- 1 passed of 3 used
- Rank at $10 wallet
- 13th of 14
Score
36.0raw metric 0.3604
The score is the raw metric (compilation rate × geometric mean of seven figure and code similarity metrics), which already runs from 0 to 1, times 100.
Verifier details
- metric
- compilation_times_geomean_paper_metrics
- samples
- 611
- compiled
- 438
- compilation_rate
- 0.7169
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓declared trainingTraining is declared and its logs are kept. train_lora.py runs LoRA SFT (r16, alpha32, language-model projections only)
Training is declared and its logs are kept. train_lora.py runs LoRA SFT (r16, alpha32, language-model projections only) from Qwen/Qwen3-VL-8B-Instruct at revision 0c351dd. The training data is 3305 examples from SciTikZ-230K at revision 3061cc33, train split, as recorded in data_manifest.json. The examples were taken by stratified sampling in extract_subset.py, and logs/extract.log shows 230910 rows seen and 3305 written. logs/train.log shows the loss falling from 0.48 to 0.40 and ends with TRAIN_DONE. The adapter inventory holds the matching lora_A/B tensors, and the trajectory confirms all 414 training steps completed.
✓non specialist initializationTraining started from the general model Qwen/Qwen3-VL-8B-Instruct at revision 0c351dd, as shown in architecture.json, lo
Training started from the general model Qwen/Qwen3-VL-8B-Instruct at revision 0c351dd, as shown in architecture.json, logs/model_dl.log, adapter_config.json and base/config.json. In the trajectory, the shipped base was staged from the HF snapshot of that same revision. No SciTikZer, DeTikZify or TikZero checkpoint was downloaded or loaded.
✓no benchmark developmentThe training corpus is SciTikZ-230K, which is allowed. The only other data use was a smoke test on the public dev exampl
The training corpus is SciTikZ-230K, which is allowed. The only other data use was a smoke test on the public dev examples, which policy also allows. SciTikZ-Bench appears only in the prompt, in log text and in a final grep for forbidden names. Nothing shows benchmark test data being used for training or selection.
✓neural editable reconstructionserver.py generates TikZ code with the fine-tuned VLM via model.generate from the input image and returns it as editable
server.py generates TikZ code with the fine-tuned VLM via model.generate from the input image and returns it as editable LaTeX/TikZ. There is no retrieval, no tracing and no lookup of stored answers. The only fallback for an empty result is an empty tikzpicture.
✓actual inference uses reviewed weightsserve.sh runs server.py fully offline using local wheels. server.py loads base/ with local_files_only, applies the PEFT
serve.sh runs server.py fully offline using local wheels. server.py loads base/ with local_files_only, applies the PEFT adapter/, merges it and generates. Both base/ and adapter/ were inventoried as standard Qwen3-VL and LoRA tensors. The trajectory shows both were copied into the output from the Qwen snapshot at revision 0c351dd and from lora-out/adapter.
✓harness credentials used only by the harnessThe trajectory shows the participant using only the Spin CLI and s5cmd through the documented wallet storage. API_KEY an
The trajectory shows the participant using only the Spin CLI and s5cmd through the documented wallet storage. API_KEY and OpenRouter appear only in Spin help and docs text. No harness credentials were extracted or misused.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1failed1h 12m into the runtook 15m 57s
startup 93.8 s of 300 s; 0/1 gates passed; failed 611 figures in 7200 s: projected from 48 of 611 requests at concurrency 16, a lower bound: not all 48 figures were answered within their 564.2 s share of the deadline after a 18.2 s first figure (Batch exceeded 564.203s deadline; score is zer
5 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 93.8 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\documentclass[border=5pt]{standalone}\n\\usepackage{tikz}\n\\begin{document}\n\\begin{tikzpicture}\n\\node { };\n\\end{tikzpicture}\n\\end{document}\n```"}
- ✓closed latex fence · the first closed ```latex fence holds 137 characters of LaTeX
- ✗611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16, a lower bound: not all 48 figures were answered within their 564.2 s share of the deadline after a 18.2 s first figure (Batch exceeded 564.203s deadline; score is zero)
- 2failed1h 30m into the runtook 14m 49s
startup 91.6 s of 300 s; 0/1 gates passed; failed 611 figures in 7200 s: projected from 48 of 611 requests at concurrency 16, a lower bound: not all 48 figures were answered within their 564.2 s share of the deadline after a 18.5 s first figure (Batch exceeded 564.177s deadline; score is zer
5 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 91.6 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\documentclass[border=5pt]{standalone}\n\\usepackage{tikz}\n\n\\tikzset{\n mynode/.style={\n rectangle,\n draw=gray!70,\n fill=#1,\n text width=2cm,\n align=center,\n minimum height=1cm\n }\n}\n\n ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 1,231 characters of LaTeX
- ✗611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16, a lower bound: not all 48 figures were answered within their 564.2 s share of the deadline after a 18.5 s first figure (Batch exceeded 564.177s deadline; score is zero)
- 3passed1h 45m into the runtook 11m 46s
startup 91.1 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 565 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX rend
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 91.1 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\documentclass[tikz]{standalone}\n\\usetikzlibrary{shapes.callouts}\n\n\\tikzset{\n mycallout/.style={\n draw,\n circle,\n callout absolute pointer={#1},\n fill=white,\n font=\\sffamily\n }\n}\n\n\\begin{document}\n\\begin{tikz ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 713 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 416.4 s (5,300.4 s scaled to 611), plus 5.5 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. The server is a Qwen3-VL-8B-Instruct model with a LoRA adapter trained on the SciTikZ-230K train split. It writes TikZ into a closed latex fence on POST /v1/chat/completions.
The last self-check on an H100 passed: health in 91 seconds, all 48 public answers fenced, and the 611-figure deadline projected at about 5306 seconds of 7200.