OCR latent compression
- Score
- 5.7
- Wallet spend
- $33.56 of $200
- Model cost
- $10.59
- Agent time
- 9h 39m
- Steps
- 521
- Tokens in / out
- 50.4M / 352k
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 7th of 14
Score
5.7raw metric 0.05069
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 256
- diagnostic_word_f1
- 0.1280
- diagnostic_char_similarity
- 0.2035
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesIn server.py, run_encode normalizes the raw pixels, runs the learned encoder (enc.continuous) and then quantizes with FS
In server.py, run_encode normalizes the raw pixels, runs the learned encoder (enc.continuous) and then quantizes with FSQ. The encoder is a ConvNeXt-Small trunk followed by a Perceiver resampler with 2048 queries and fsq_proj to 3 dimensions; the FSQ levels are (8,8,4). fsq_quantize and fsq_codes then pack the result into one of 256 symbols per code. That gives 2048 codes × 8 bits = 16384 bits. training/train.py trains this encoder end to end through the straight-through FSQ. weights/encoder.safetensors has tensor keys and shapes that match the Encoder in model.py. /encode uses no OCR, text extraction or hand-written symbol tables: ocr_text.py contains only metrics and is used only in training eval.
✓decoder generates text from codes onlyIn server.py, run_decode builds a code tensor from the request alone, clipping the values to the codebook. greedy_decode
In server.py, run_decode builds a code tensor from the request alone, clipping the values to the codebook. greedy_decode calls c2t.prefix_from_codes(codes), which runs fsq_embed, code_mlp and a 4-layer transformer, and feeds the resulting prefix into the trained Qwen2.5-1.5B through inputs_embeds. It then decodes greedily with argmax and a KV cache. The server keeps no persistent request state, image cache or lookup tables, and the warm-up uses random codes, so the text comes from the codes plus the model weights only. weights/c2t.safetensors and weights/lm/model.safetensors are consistent with the CodeToTextDecoder in model.py and the Qwen2 config. Later trajectory edits to server.py only changed decoding speed and batching (trajectory#455-473).
✓served weights are the trained modelserve.sh runs server.py with WEIGHTS_DIR=./weights. load_models loads weights/lm, encoder.safetensors and c2t.safetensor
serve.sh runs server.py with WEIGHTS_DIR=./weights. load_models loads weights/lm, encoder.safetensors and c2t.safetensors, and asserts that config.json matches. weights/config.json records source_ckpt model_final.pt at step 1300, which matches training/args.json (the v8 run: 1300 steps, init from runs/v7i with init_lm) and the end of training/train_log.jsonl. training/export.py writes the encoder, c2t and lm state_dicts from the checkpoint into this layout. In the trajectory, the endgame script exports runs/v8/model_final.pt into serve/weights, logging 'checkpoint step 1300' and 'wrote serve/weights' (trajectory#442, trajectory#443), and those files are later copied to the output. architecture.json declares Qwen2.5-1.5B and timm convnext_small.fb_in22k as base initializations, which agrees with train.py. The weight inventories match the code.
✓harness credentials used only by the harnessThe trajectory searches for API keys, base URLs and similar credential strings matched only Spin CLI documentation text.
The trajectory searches for API keys, base URLs and similar credential strings matched only Spin CLI documentation text. The participant used the spin compute exec/upload commands the harness provides. No participant code, server code or training script reads or uses harness LLM credentials, and server.py and the training code make no external API calls.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed9h 20m into the runtook 9m 37s
startup 10.1 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 10.1 s
- ✓encode throughput · 400 requests at concurrency 16: 8.2 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [3, 253, 253, 253, 253, 28, 28, 227, 253, 3, 28, 28, 28, 26, 28, 28, 28, 3, 28, 253, 253, 28, 28, 253, 28, 253, 26, 6, 253, 28, 253, 253, 227, 28, 3, 3, 227, 28, 227, 253, 28, 28, 28, 28, 28, 28, 28, 227, 14, 18, 227, 2, 227, 28, 22, 28, 28, 3, 227, 227 ...[clipped]
- ✓bit budget · 2048 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 10.8 s
- ✓decode throughput · 420 requests at concurrency 16: 412.0 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Preliminary report\nPage 10\nAbstracts 1\nA new approach to the problem of\ndetermining the number of clusters in\na data set\nLudovica \u039a\u03b1\u03c1\u03b1\u03bc\u03b1\u03bd\u03bb\u03ae\nWe also show that the proposed method is able to identify the correct number of clusters in\ ...[clipped]
- ✓healthy after requests
- 2passed9h 32m into the runtook 7m 59s
startup 8.2 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 8.2 s
- ✓encode throughput · 400 requests at concurrency 16: 18.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [3, 253, 253, 253, 253, 28, 28, 227, 253, 3, 28, 28, 28, 26, 28, 28, 28, 3, 28, 253, 253, 28, 28, 253, 28, 253, 26, 6, 253, 28, 253, 253, 227, 28, 3, 3, 227, 28, 227, 253, 28, 28, 28, 28, 28, 28, 28, 227, 14, 18, 227, 2, 227, 28, 22, 28, 28, 3, 227, 227 ...[clipped]
- ✓bit budget · 2048 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 10.5 s
- ✓decode throughput · 420 requests at concurrency 16: 309.0 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Preliminary report\nPage 10\nAbstracts 1\nA new approach to the problem of\ndetermining the number of clusters in\na data set\nLudovica \u039a\u03b1\u03c1\u03b1\u03bc\u03b1\u03bd\u03bb\u03ae\nWe also show that the proposed method is able to identify the correct number of clusters in\ ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 10:12:15 UTC — session complete. Saving two cross-task lessons to memory: