Figure to TikZ
- Score
- 53.2
- Wallet spend
- $90.75 of $200
- Model cost
- $26.53
- Agent time
- 7h 58m
- Steps
- 842
- Tokens in / out
- 84.5M / 503k
- Self-checks
- 3 passed of 3 used
- Rank at $200 wallet
- 7th of 19
Score
53.2raw metric 0.5318
The score is the raw metric (compilation rate × geometric mean of seven figure and code similarity metrics), which already runs from 0 to 1, times 100.
Verifier details
- metric
- compilation_times_geomean_paper_metrics
- samples
- 611
- compiled
- 583
- compilation_rate
- 0.9542
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓declared trainingarchitecture.json names Qwen3-VL-8B-Instruct at a pinned revision and a LoRA SFT run: r64, alpha128, 2500 steps, 40000 e
architecture.json names Qwen3-VL-8B-Instruct at a pinned revision and a LoRA SFT run: r64, alpha128, 2500 steps, 40000 examples. The retained files match that declaration: evidence/train_sft.py, evidence/train_command.sh, evidence/data_manifest.json and evidence/train_main40k.log. The trajectory shows train_sft.py being launched with the declared arguments (--max-steps 2500, manifest data/selected/train.jsonl) on Spin compute.
✓non specialist initializationThe starting model is a general-purpose vision-language model. evidence/download_data.sh fetches it with snapshot_downlo
The starting model is a general-purpose vision-language model. evidence/download_data.sh fetches it with snapshot_download of Qwen/Qwen3-VL-8B-Instruct at the pinned revision, and train_sft.py loads it with from_pretrained. The trajectory confirms that download. The merged safetensors inventory has the shapes of Qwen3-VL-8B (for example lm_head [151936,4096], 36 language layers). No specialist image-to-TikZ checkpoint such as DeTikZify or SciTikZer was loaded; those names appear only in task text.
✓no benchmark developmentTraining uses SciTikZ-230K at a pinned revision plus the 50 supplied public validation examples, both of which are allow
Training uses SciTikZ-230K at a pinned revision plus the 50 supplied public validation examples, both of which are allowed. Checkpoint selection used a 550-row internal dev set carved out of the SciTikZ-230K train split. data_manifest.json declares these sources and lists no forbidden ones. Every SciTikZ-Bench mention I found is task text or a disclaimer, not data use.
✓neural editable reconstructionserver.py forwards each image request to a local vLLM server running the fine-tuned model, which generates TikZ from the
server.py forwards each image request to a local vLLM server running the fine-tuned model, which generates TikZ from the image. latex_repair.py only fixes syntax (xcolor, axes, truncation). It never reads the image and holds no stored answers. A blank-canvas FALLBACK_CODE is returned only on errors or before the engine is ready.
✓actual inference uses reviewed weightsserve.sh installs offline wheels, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and execs server.py. server.py starts vl
serve.sh installs offline wheels, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and execs server.py. server.py starts vllm.entrypoints.openai.api_server with --model ROOT/model. The shipped model/ shards are the merged checkpoint-2500 BF16 weights; the trajectory shows the final model directory pointed at merged2500. Their tensor inventory matches the declared base architecture, and there are no external inference calls.
✓harness credentials used only by the harnessHF_TOKEN was used only for Hugging Face metadata reads and downloads; I found no HF upload_folder call. Checkpoint uploa
HF_TOKEN was used only for Hugging Face metadata reads and downloads; I found no HF upload_folder call. Checkpoint uploads went through Spin transfer sessions (evidence/merge_upload_checkpoint.sh). ANTHROPIC and OPENROUTER strings appear only in process listings and in a secret-scan script, with no value reused. I found no claude -p call, harness CLI launch or direct LLM API call by the participant.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed2h 6m into the runtook 8m 25s
startup 113.7 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 564 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX ren
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 113.7 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\documentclass[border=5pt]{standalone}\n\\usepackage{tikz}\n\\usetikzlibrary{positioning}\n\n\\tikzset{\n mynode/.style={\n draw,\n minimum width=1cm,\n minimum height=0.5cm,\n font=\\sffamily\n },\n mynode2/.style={\n draw,\n ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 1,788 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 114.0 s (1,451.1 s scaled to 611), plus 16.7 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
- 2passed7h 38m into the runtook 8m 28s
startup 128.7 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 565 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX ren
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 128.7 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\documentclass{standalone}\n\\usepackage{tikz}\n\\usetikzlibrary{positioning}\n\n\\begin{document}\n\\begin{tikzpicture}[\n node distance=0pt,\n mynode/.style={draw, fill=#1!30},\n mynode/.default=white,\n every edge/.append style={ ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 1,197 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 124.8 s (1,588.9 s scaled to 611), plus 13.3 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
- 3passed7h 49m into the runtook 7m 25s
startup 114.9 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 565 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX ren
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 114.9 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\documentclass{standalone}\n\\usepackage{tikz}\n\\usetikzlibrary{positioning}\n\n\\begin{document}\n\\begin{tikzpicture}[\n node distance=0pt,\n mynode/.style={draw, fill=#1!30},\n mynode/.default=white,\n every edge/.append style={ ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 1,197 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 94.4 s (1,201.1 s scaled to 611), plus 11.0 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
The agent's last message before its session ended.
The exact final folder passed its selfcheck: 114.9 s startup, 1212.1 s projected for 611 figures, and all response gates passed. Its 218-file byte count matches the final audit exactly. I’m proceeding with the two-step final submission.