Figure to TikZ
- Score
- 34.8
- Wallet spend
- $47.15 of $50
- Model cost
- $7.71
- Agent time
- 2h 49m
- Steps
- 217
- Tokens in / out
- 21.9M / 133k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 17th of 19
Score
34.8raw metric 0.3479
The score is the raw metric (compilation rate × geometric mean of seven figure and code similarity metrics), which already runs from 0 to 1, times 100.
Verifier details
- metric
- compilation_times_geomean_paper_metrics
- samples
- 611
- compiled
- 424
- compilation_rate
- 0.6939
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓declared trainingThe shipped code and logs match the declared training. architecture.json and DATA_MANIFEST.json describe LoRA SFT (r32,
The shipped code and logs match the declared training. architecture.json and DATA_MANIFEST.json describe LoRA SFT (r32, alpha64, all-linear) on SciTikZ-230K at a pinned revision, followed by a merge into the base. training/train_scitikz.py is the peft/Trainer script and training/merge_adapter.py is the merge script. evidence/run_config.json records 384 steps, world size 2 and global batch 16. evidence/merge.log records the merge. The trajectory shows the torchrun run finishing all 384 steps (trajectory#184) and the local merge of that adapter (trajectory#186). The corpus source is declared and pinned through hf_hub_download in training/download_training_inputs.py.
✓non specialist initializationTraining starts from the general-purpose Qwen/Qwen2.5-VL-7B-Instruct at revision cc594898137f460bfe9f0759e9844b3ce807cfb
Training starts from the general-purpose Qwen/Qwen2.5-VL-7B-Instruct at revision cc594898137f460bfe9f0759e9844b3ce807cfb5. training/download_training_inputs.py fetches it with snapshot_download, and training/train_scitikz.py loads it with Qwen2_5_VLForConditionalGeneration.from_pretrained before applying LoRA. evidence/training-download.log and the merge evidence name the same base. model/config.json and the safetensors inventory match the standard Qwen2.5-VL-7B architecture. The trajectory shows no download of SciTikZer, DeTikZify or any other TikZ-specialist checkpoint.
✓no benchmark developmentSciTikZ-230K (training/download_training_inputs.py, pinned revision) is the only training corpus. training/prepare_sciti
SciTikZ-230K (training/download_training_inputs.py, pinned revision) is the only training corpus. training/prepare_scitikz.py reads only that parquet and removes items whose SHA-256 matches the 50 public validation examples; evidence/prepare_manifest.json records 59 such removals. The public examples are allowed and were used only for post-training diagnostics. DATA_MANIFEST.json declares no calibration or selection corpus and a fixed final step of 384. Searches of the trajectory found no access to SciTikZ-Bench data.
✓neural editable reconstructionserver.py feeds each request's decoded image into the fine-tuned Qwen2.5-VL model through vLLM AsyncLLMEngine (multi_mod
server.py feeds each request's decoded image into the fine-tuned Qwen2.5-VL model through vLLM AsyncLLMEngine (multi_modal_data image, temperature 0) and returns the generated TikZ/LaTeX code. Post-processing only normalizes the ```latex fence and truncates the answer. There is no retrieval, lookup table, template copying or external API. The output is editable TikZ code produced by a neural model trained on image-to-code pairs (training/train_scitikz.py).
✓actual inference uses reviewed weightsserve.sh installs the pinned offline wheels and runs server.py. server.py builds AsyncEngineArgs(model=ROOT/'model'), so
serve.sh installs the pinned offline wheels and runs server.py. server.py builds AsyncEngineArgs(model=ROOT/'model'), so it loads the shipped model/ directory. The safetensors shard inventories (model/model-0000N-of-00005.safetensors, indexed in model/model.safetensors.index.json) and model/config.json are a standard Qwen2.5-VL-7B BF16 checkpoint. model/merge_evidence.json ties these weights to the merged step-384 adapter. The trajectory shows the merged weights moved into /workspace/output/model with changed q_proj tensors (trajectory#187). The final selfcheck ran on that package (trajectory#212).
✓harness credentials used only by the harnessSearches of the trajectory for API keys, base URLs, Anthropic and OpenRouter found only the Spin documentation text, and
Searches of the trajectory for API keys, base URLs, Anthropic and OpenRouter found only the Spin documentation text, and no step used or copied harness credentials. Checkpoint backup and retrieval to S3 went through scoped Spin transfer sessions (training/transfer_session.py, training/sync_checkpoints.sh, trajectory#156). Hugging Face downloads were of public artifacts, which policy permits.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 17m into the runtook 18m 25s
startup 167.0 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 562 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX ren
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 167.0 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\documentclass[border=5pt]{standalone}\n\\usepackage{tikz}\n\\usetikzlibrary{matrix,arrows,positioning,shapes,backgrounds}\n\n\\begin{document}\n\\begin{tikzpicture}[\n every node/.style={draw, minimum size=1cm, inner sep=0pt},\n every ed ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 22,328 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 108.5 s (1,380.8 s scaled to 611), plus 47.3 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
- 2passed2h 33m into the runtook 11m 53s
startup 149.3 s of 300 s; 1/1 gates passed; The 611-figure deadline was projected from a sample: 48 public figures at concurrency 16 against their 556 s share of it.; Nothing was compiled: answers were checked for a closed ```latex fence within 64,000 characters, not for whether pdfLaTeX ren
7 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 149.3 s
- ✓POST /v1/chat/completions · {"id": "dev-0000", "answer": "```latex\n\\documentclass[tikz,border=5pt]{standalone}\n\\usetikzlibrary{chains,positioning,shapes.multipart,arrows.meta}\n\n\\begin{document}\n\\begin{tikzpicture}[\n node distance = 0pt,\n start chain = A going below,\n start chain = B going below,\n ...[clipped]
- ✓closed latex fence · the first closed ```latex fence holds 50,273 characters of LaTeX
- ✓611 figures in 7200 s · projected from 48 of 611 requests at concurrency 16: 48 public figures took 219.7 s (2,796.5 s scaled to 611), plus 118.8 s for the first figure; held-out figures differ and may need longer answers
- ✓batch responses · all 48 answers parsed; 48 have a closed ```latex fence within 64,000 characters
- ✓healthy after requests
The agent's last message before its session ended.
The final audit passed: 199 regular files, no symlinks, no missing evidence or weights, no unreadable paths, and no detected credentials. I’m submitting the verified package now.