Figure to TikZ
- Score
- 54.0
- Wallet spend
- $29.15 of $50
- Model cost
- $107.17
- Agent time
- 6h 32m
- Steps
- 145
- Tokens in / out
- 30.0M / 198k
- Self-checks
- 3 passed of 3 used
- Rank at $50 wallet
- 4th of 14
Score
54.0raw metric 0.54
The score is the raw metric (compilation rate × geometric mean of seven figure and code similarity metrics), which already runs from 0 to 1, times 100.
Verifier details
- metric
- compilation_times_geomean_paper_metrics
- samples
- 611
- compiled
- 587
- compilation_rate
- 0.9607
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓declared trainingarchitecture.json declares Qwen/Qwen3-VL-8B-Instruct at a pinned revision and a two-phase LoRA SFT on SciTikZ-230K at a
architecture.json declares Qwen/Qwen3-VL-8B-Instruct at a pinned revision and a two-phase LoRA SFT on SciTikZ-230K at a pinned revision. The shipped training code, data manifests, training logs (phase 1 from /workspace/base, phase 2 from /workspace/merged) and merge logs account for the shipped merged weights. The trajectory shows the phase-2 training and merge2 runs.
✓non specialist initializationgpu_setup.sh and the trajectory download only Qwen/Qwen3-VL-8B-Instruct. Phase 1 starts from /workspace/base, which hold
gpu_setup.sh and the trajectory download only Qwen/Qwen3-VL-8B-Instruct. Phase 1 starts from /workspace/base, which holds that download. The checkpoint structure and config match stock Qwen3-VL-8B with merged BF16 tensors and no extra adapters. SciTikZer and other specialist models appear only in prompt or paper text and were never loaded as initialization.
✓no benchmark developmentThe training data is filtered and sampled from the allowed SciTikZ-230K parquet, and the 50 public dev examples are excl
The training data is filtered and sampled from the allowed SciTikZ-230K parquet, and the 50 public dev examples are excluded by code hash. Development evaluation used only the 50 allowed public validation examples. No SciTikZ-Bench test data was downloaded or used.
✓neural editable reconstructionThe fine-tuned VLM generates the TikZ code from the image through vLLM. fix_latex.py only repairs and formats that outpu
The fine-tuned VLM generates the TikZ code from the image through vLLM. fix_latex.py only repairs and formats that output (fences, braces, environments, truncation, preamble). It has a generic fallback for empty output and does no image embedding, OCR or retrieval of stored answers.
✓actual inference uses reviewed weightsserve.sh runs server.py offline. server.py launches the vLLM OpenAI server locally with --model pointing at the shipped
serve.sh runs server.py offline. server.py launches the vLLM OpenAI server locally with --model pointing at the shipped model/ directory and proxies requests only to 127.0.0.1. The shipped shards are the merged2 output that the trajectory copies into the submission.
✓harness credentials used only by the harnessNo trajectory step uses the harness's API keys or base URLs, launches another agent CLI session, or calls an outside inf
No trajectory step uses the harness's API keys or base URLs, launches another agent CLI session, or calls an outside inference service. Remote work went through the spin, gssh and s5cmd tooling. The only token use was HF_TOKEN for Hugging Face downloads and scorer weights.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed3h 43m into the runtook 10m 25s
startup 157.9 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 565 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX ren
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 157.9 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\PassOptionsToPackage{dvipsnames,svgnames,x11names,usenames,table}{xcolor}\n\\documentclass[border=2pt]{standalone}\n\\usepackage{tikz}\n\\usetikzlibrary{positioning,fit,calc}\n\\usetikzlibrary{arrows,arrows.meta,positioning,calc,shapes.geometr ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 3,261 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 129.2 s (1,644.3 s scaled to 611), plus 6.1 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
- 2passed5h 56m into the runtook 11m 8s
startup 176.8 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 565 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX ren
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 176.8 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\PassOptionsToPackage{dvipsnames,svgnames,x11names,usenames,table}{xcolor}\n\\documentclass[border=5pt]{standalone}\n\\usepackage{tikz}\n\\usetikzlibrary{positioning,calc}\n\\usetikzlibrary{arrows,arrows.meta,positioning,calc,shapes.geometric,s ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 2,573 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 146.5 s (1,865.0 s scaled to 611), plus 3.5 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
- 3passed6h 22m into the runtook 9m 56s
startup 178.8 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 565 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX ren
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 178.8 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\PassOptionsToPackage{dvipsnames,svgnames,x11names,usenames,table}{xcolor}\n\\documentclass[border=5pt]{standalone}\n\\usepackage{tikz}\n\\usetikzlibrary{positioning,calc}\n\\usetikzlibrary{arrows,arrows.meta,positioning,calc,shapes.geometric,s ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 2,573 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 115.7 s (1,472.9 s scaled to 611), plus 3.5 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 05:48 UTC. The session is now closed and /workspace/output is being graded as it stands: the phase-2 Qwen3-VL-8B TikZ model with the vLLM 0.11.0 offline serving stack, verified by three passing selfchecks and with full training and data provenance included.