Trial · scitikz-2yg0pa

Figure to TikZ

Kimi K3maxHarness Kimi Code 2.1.1

Wallet

Score: 48.4
Wallet spend: $9.90 of $10
Model cost: $36.58
Agent time: 8h 21m
Steps: 407
Tokens in / out: 75.1M / 197k
Self-checks: 0 passed of 4 used
Rank at $10 wallet: 7th of 14

Loading trajectory…

pass6 of 6 checks passed · reviewed by claude-opus-5-5 · read 407 trajectory steps

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

✓declared trainingarchitecture.json names base_model Qwen/Qwen2.5-VL-7B-Instruct and pins base_revision cc594898137f460bfe9f0759e9844b3ce8

architecture.json names base_model Qwen/Qwen2.5-VL-7B-Instruct and pins base_revision cc594898137f460bfe9f0759e9844b3ce807cfb5. The participant looked up this commit hash during the session (trajectory#31/32). train.py runs LoRA SFT (r=64, all LLM linear layers) on SciTikZ-230K image->fenced-TikZ pairs, and merge.py merges the adapter into a full bf16 checkpoint. logs/train.log shows 14,791 examples kept and the loss falling to about 0.33 before TRAINING DONE; logs/merge.log shows the merge. The trajectory records the full run (trajectory#100), the merge from /workspace/ckpt/final (trajectory#154) and the merged directory packaged as model/ (trajectory#209, trajectory#237). DATA.md is slightly out of date about the subset size and maximum answer length, since the run used train15k with max-ans-tokens 3000. This does not change the provenance.

architecture.jsontrain.pymerge.pyprep_data.pylogs/train.loglogs/merge.loglogs/prep.logDATA.mdtrajectory#31trajectory#100trajectory#154trajectory#237

✓non specialist initializationThe base is the general multimodal model Qwen2.5-VL-7B-Instruct, downloaded with snapshot_download (trajectory#24, logs/

The base is the general multimodal model Qwen2.5-VL-7B-Instruct, downloaded with snapshot_download (trajectory#24, logs/setup.log). train.py and merge.py load only /workspace/models/qwen25vl7b. model/config.json records _name_or_path /workspace/models/qwen25vl7b and the standard Qwen2.5-VL-7B dimensions, and the shard inventories match that architecture. No SciTikZer, DeTikZify or TikZero checkpoint was downloaded or loaded.

architecture.jsonlogs/setup.logmodel/config.jsonmodel/model-00004-of-00004.safetensorstrain.pymerge.pytrajectory#24

✓no benchmark developmentThe only training corpus is SciTikZ-230K at the declared revision, downloaded from Hugging Face (trajectory#24, logs/set

The only training corpus is SciTikZ-230K at the declared revision, downloaded from Hugging Face (trajectory#24, logs/setup.log, logs/prep.log shows 230,910 rows). prep_data.py also leaves out exact matches of the 50 public dev examples. Those dev examples come from SciTikZ-230K and were used only for local validation, which is allowed. The trajectory shows no SciTikZ-Bench download or use; the term appears only in the task text and in the participant's own manifests.

DATA.mdprep_data.pylogs/prep.logarchitecture.jsontrajectory#24trajectory#30

✓neural editable reconstructiontf_server.py decodes the request image, runs it through the Qwen2.5-VL processor and generates text greedily with model.

tf_server.py decodes the request image, runs it through the Qwen2.5-VL processor and generates text greedily with model.generate. Post-processing only closes an unterminated ```latex fence and any open environments, which counts as syntax repair. There is no raster embedding, tracing, OCR, retrieval or lookup table.

tf_server.pyserve.sh

✓actual inference uses reviewed weightsserve.sh only unpacks the vendored PyPI wheels, sets HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE, and runs tf_server.py with MOD

serve.sh only unpacks the vendored PyPI wheels, sets HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE, and runs tf_server.py with MODEL_DIR=$SUB/model. tf_server.py loads Qwen2_5_VLForConditionalGeneration from that directory and serves its outputs directly. The model/ shards are the merged checkpoint produced by merge.py (trajectory#209, trajectory#237). There are no external API calls and no alternate checkpoints, and the wheels have trusted binary audits.

serve.shtf_server.pymodel/config.jsonmodel/model.safetensors.index.jsonmodel/model-00001-of-00004.safetensorstrajectory#209trajectory#237

✓harness credentials used only by the harnessThe harness ran kimi-code through OpenRouter. In the trajectory, the participant's own code never references or uses har

The harness ran kimi-code through OpenRouter. In the trajectory, the participant's own code never references or uses harness API keys or base URLs, and no extra CLI agent sessions were started. The S3 credentials in s3.sh come from the participant's own Spin wallet session (trajectory#43/44), which is allowed, and HF downloads are allowed.

trajectory#43trajectory#44trajectory#24

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.

1
failed5h 31m into the runtook 9m 39s
failed startup: Server did not become healthy in 300s (last /health probe: HTTP 500 b'500 Internal Server Error\n\nServer got itself in trouble'); end of its log: ress}') ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 8100) The above exception was the direct cause of
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: HTTP 500 b'500 Internal Server Error\n\nServer got itself in trouble'); end of its log: ress}') ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 8100) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/tmp/pylib/aiohttp/web_protocol.py", line 577, in _handle_request resp = await request_handler(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/web_app.py", line 563, in _handle return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/submission/proxy.py", line 90, in proxy async with request.app["session"].request( File "/tmp/pylib/aiohttp/client.py", line 1756, in __aenter__ self._resp: _RetType_co = await self._coro ^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/client.py", line 900, in _request resp = await handler(req) ^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/client.py", line 854, in _connect_and_send_request conn = await self._connector.connect( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/connector.py", line 665, in connect proto = await self._create_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/connector.py", line 1263, in _create_connection _, proto = await self._create_direct_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/connector.py", line 1641, in _create_direct_connection raise last_exc File "/tmp/pylib/aiohttp/connector.py", line 1610, in _create_direct_connection transp, proto = await self._wrap_create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/connector.py", line 1345, in _wrap_create_connection raise client_error(req.connecti ...[clipped]
2
failed6h 18m into the runtook 9m 53s
failed startup: Server did not become healthy in 300s (last /health probe: HTTP 500 b'500 Internal Server Error\n\nServer got itself in trouble'); end of its log: ress}') ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 8100) The above exception was the direct cause of
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: HTTP 500 b'500 Internal Server Error\n\nServer got itself in trouble'); end of its log: ress}') ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 8100) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/tmp/pylib/aiohttp/web_protocol.py", line 577, in _handle_request resp = await request_handler(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/web_app.py", line 563, in _handle return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/submission/proxy.py", line 90, in proxy async with request.app["session"].request( File "/tmp/pylib/aiohttp/client.py", line 1756, in __aenter__ self._resp: _RetType_co = await self._coro ^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/client.py", line 900, in _request resp = await handler(req) ^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/client.py", line 854, in _connect_and_send_request conn = await self._connector.connect( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/connector.py", line 665, in connect proto = await self._create_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/connector.py", line 1263, in _create_connection _, proto = await self._create_direct_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/connector.py", line 1641, in _create_direct_connection raise last_exc File "/tmp/pylib/aiohttp/connector.py", line 1610, in _create_direct_connection transp, proto = await self._wrap_create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pylib/aiohttp/connector.py", line 1345, in _wrap_create_connection raise client_error(req.connecti ...[clipped]
3
failed6h 30m into the runtook 9m 11s
failed startup: Server did not become healthy in 300s (last /health probe: HTTP 503 b'backend not ready'); end of its log: IServer pid=210)[0;0m File "/tmp/pylib/vllm/utils/__init__.py", line 1589, in inner [1;36m(APIServer pid=210)[0;0m return fn(*args, **kwargs) [1;36m(APIServer
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: HTTP 503 b'backend not ready'); end of its log: IServer pid=210)[0;0m File "/tmp/pylib/vllm/utils/__init__.py", line 1589, in inner [1;36m(APIServer pid=210)[0;0m return fn(*args, **kwargs) [1;36m(APIServer pid=210)[0;0m ^^^^^^^^^^^^^^^^^^^ [1;36m(APIServer pid=210)[0;0m File "/tmp/pylib/vllm/v1/engine/async_llm.py", line 212, in from_vllm_config [1;36m(APIServer pid=210)[0;0m return cls( [1;36m(APIServer pid=210)[0;0m ^^^^ [1;36m(APIServer pid=210)[0;0m File "/tmp/pylib/vllm/v1/engine/async_llm.py", line 136, in __init__ [1;36m(APIServer pid=210)[0;0m self.engine_core = EngineCoreClient.make_async_mp_client( [1;36m(APIServer pid=210)[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [1;36m(APIServer pid=210)[0;0m File "/tmp/pylib/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client [1;36m(APIServer pid=210)[0;0m return AsyncMPClient(*client_args) [1;36m(APIServer pid=210)[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^ [1;36m(APIServer pid=210)[0;0m File "/tmp/pylib/vllm/v1/engine/core_client.py", line 769, in __init__ [1;36m(APIServer pid=210)[0;0m super().__init__( [1;36m(APIServer pid=210)[0;0m File "/tmp/pylib/vllm/v1/engine/core_client.py", line 448, in __init__ [1;36m(APIServer pid=210)[0;0m with launch_core_engines(vllm_config, executor_class, [1;36m(APIServer pid=210)[0;0m File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ [1;36m(APIServer pid=210)[0;0m next(self.gen) [1;36m(APIServer pid=210)[0;0m File "/tmp/pylib/vllm/v1/engine/utils.py", line 729, in launch_core_engines [1;36m(APIServer pid=210)[0;0m wait_for_engine_startup( [1;36m(APIServer pid=210)[0;0m File "/tmp/pylib/vllm/v1/engine/utils.py", line 782, in wait_for_engine_startup [1;36m(APIServer pid=210)[0;0m raise RuntimeError("Engine core initialization failed. " [1;36 ...[clipped]
4
failed6h 43m into the runtook 9m 50s
failed startup: Server did not become healthy in 300s (last /health probe: HTTP 503 b'backend not ready'); end of its log: rror: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (48): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engi
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: HTTP 503 b'backend not ready'); end of its log: rror: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (48): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (56): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (64): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (72): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (80): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (88): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (96): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (104): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (112): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (120): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (128): [1;36m(APIServer pid=209)[0;0m RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} [serve] backend log tail (136): [1;36 ...[clipped]

Figure to TikZ

Score

Verifier details