All tasks
Task · scitikz

Figure to TikZ

Reconstruct scientific figures as editable TikZ code

Vision & AudioInspired byReducto
Score
compilation rate times image and code similarity
Best
65.1
Median over trials
48.4

Instruction

Train a vision-language model that takes an image of a scientific figure and writes TikZ/LaTeX code to recreate it. I have provided 50 example figures with reference code from SciTikZ-230K.

You can train on these and the rest of SciTikZ-230K, but don't use SciTikZ-Bench at all. The model itself has to write the TikZ: no embedding the input image, pixel tracing, classical OCR or looking up stored answers. You can start from any pretrained base model, as long as it isn't a specialist TikZ model (e.g., you may not start from JackieLin0123/SciTikZer-8B).

Write a script serve.sh which serves the model through an OpenAI-style POST /v1/chat/completions endpoint. It takes a figure image and returns a complete LaTeX document in a ```latex block. Run offline on one H100, ready within 300 seconds, and finish all 611 test figures within 2 hours.

Optimize how often the output compiles and how closely the rendered figure and code match the reference.

Do not train, fine-tune, calibrate or select on the validation or test split of any public dataset or benchmark; training splits and other public corpora are fine.

Submit in /workspace/output/. Interface and runtime details: /task/CONTRACT.md. Supporting files: /task/public/. Evaluate with python /task/validate.py --server-url URL. Test the finished submission the way it will be run with frontier selfcheck. Hand it in with frontier submit when it is final. Use spin --help for compute and model APIs.

Trials

42 graded · click a score for the trajectory
Model$10 wallet$50 wallet$200 wallet
Claude Opus 5.5xhigh56.457.065.1
Claude Sonnet 5.5xhigh54.757.758.7
Claude Fable 5.1xhigh54.554.056.5
Claude Haiku 5.5max42.453.352.2
GPT-6 Astraxhigh52.957.059.1
GPT-6.1 Solxhigh50.551.060.4
GPT-6 Lunaxhigh41.445.732.9
Gemini 3.8 Flashhigh30.627.834.9
Grok 4.7xhigh36.035.57.8
Muse Spark 1.3max49.343.647.8
Muse Wondermax41.444.443.0
Kimi K3max48.448.341.4
Qwen3.8 Maxxhigh44.446.856.3
GLM 5.3max39.540.149.0

Scoring

The score is the raw metric (compilation rate × geometric mean of seven figure and code similarity metrics), which already runs from 0 to 1, times 100.