OCR latent compression
- Score
- 67.5
- Wallet spend
- $42.52 of $50
- Model cost
- $61.94
- Agent time
- 11h 5m
- Steps
- 649
- Tokens in / out
- 62.9M / 703k
- Self-checks
- 3 passed of 3 used
- Rank at $50 wallet
- 2nd of 14
Score
67.5raw metric 0.6026
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 65536
- diagnostic_word_f1
- 0.6856
- diagnostic_char_similarity
- 0.7082
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesIn server.py, /encode loads the canvas (ptok/imageio.py does a deterministic resize and pad) and calls engine.encode. Th
In server.py, /encode loads the canvas (ptok/imageio.py does a deterministic resize and pad) and calls engine.encode. That function, in ptok/engine.py, runs PageModel.features: the SAM-ViT-B and CLIP-L encoders, a projector and the learned LayerNorm/MLP to_code head. PageModel.quantize then applies FSQ (ptok/fsq.py: tanh bound and rounding with a straight-through estimator, 11 channels at 7 levels). The FSQ digits become a mixed-radix index and are packed into 16-bit words (ptok/common.py: codebook 65536, at most 16384 bits; 520×log2(7^11)≈16058 bits fits in 1004 words). The encoder and the to_code head were trained end to end through the quantizer in train/train.py (stage 2 and the stage 3 warm restart). The encode path has no OCR, text extraction or side channel. The codes are learned, discrete and visual.
✓decoder generates text from codes only/decode in server.py checks the code count, then calls engine.codes_to_digits (unpack the words, then index to FSQ digit
/decode in server.py checks the code count, then calls engine.codes_to_digits (unpack the words, then index to FSQ digits). PageModel.decoder_embeds turns the digits into FSQ values, passes them through the from_code MLP and adds row/col embeddings and image_newline. The DeepSeek-3B-MoE LM in ptok/lm.py then decodes greedily (argmax prefill and decode_step, a static KV cache and CUDA graphs) under the fixed prompt, with trim_loop repeat truncation, and the tokens are detokenized. The decoder has no image access, retrieval, cache keyed on images or other state. Text comes only from the submitted codes and the model weights.
✓served weights are the trained modelserve.sh runs server.py. server.py calls build_model(None) and then load_state_dict(strict=True) from model/model.safete
serve.sh runs server.py. server.py calls build_model(None) and then load_state_dict(strict=True) from model/model.safetensors, and the weight inventory (clip.*, sam.*, to_code, from_code, row/col embeddings, lm.*) matches PageModel. The trajectory shows how this file was produced. At step 623, train/export_weights.py exported the stage 3 ckpt (/workspace/runs/stage3/ckpt_model.pt, step 2737, the same file as ckpt_final) by merging its trained parameters with the base DeepSeek-OCR weights into /dev/shm/s3/model.safetensors. At steps 633-634 that file was transferred and its md5 checked (21a64cb6…). At step 635 it was installed as /workspace/output/model/model.safetensors, at the 6,677,857,942-byte size that matches the inventory. Stage 3 was selected on the participant's own synthetic held-out eval (logs/eval/summary.json). The served weights are the trained model.
✓harness credentials used only by the harnessThe trajectory shows no use of harness or Anthropic credentials by participant code. The subagent instructions say expli
The trajectory shows no use of harness or Anthropic credentials by participant code. The subagent instructions say explicitly never to use the harness API key (trajectory#28). Model and compute work went through the Spin CLI and spin compute boxes. The S3 credentials in /root/s3env.sh (trajectory#627) are Spin's own transfer-storage credentials for spin-data/spin-transfers, not harness credentials. The final secret grep of /workspace/output turned up nothing (trajectory#647). No violation was established.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed55m 8s into the runtook 11m 34s
startup 30.3 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 30.3 s
- ✓encode throughput · 400 requests at concurrency 16: 14.6 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [56174, 46980, 3397, 44462, 19044, 40356, 19716, 23317, 48726, 40377, 6852, 19662, 49663, 39006, 42124, 437, 12120, 14242, 23306, 34671, 17901, 27488, 8715, 34346, 29134, 63194, 3830, 56826, 61287, 21350, 28954, 53538, 30025, 41552, 31848, 64433, 4721 ...[clipped]
- ✓bit budget · 1004 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 30.1 s
- ✓decode throughput · 420 requests at concurrency 16: 327.4 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Search\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSearch\nSear ...[clipped]
- ✓healthy after requests
- 2passed4h 4m into the runtook 5m 21s
startup 7.6 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 7.6 s
- ✓encode throughput · 400 requests at concurrency 16: 19.9 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [22293, 1165, 50701, 20499, 52729, 42834, 53854, 18186, 62136, 24704, 34760, 11229, 38091, 57399, 32876, 60966, 45813, 3591, 13723, 55140, 16897, 14419, 28467, 37579, 24519, 39179, 39154, 63642, 18946, 52655, 12272, 28913, 3297, 56175, 39283, 7012, 26 ...[clipped]
- ✓bit budget · 1004 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 7.0 s
- ✓decode throughput · 420 requests at concurrency 16: 73.4 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Generation Using Geometric Constraints for Natural and Realistic Talking Face-Making\nNiu Ma, Lusi Millang\nJun 8, 2026\nAbstract\nGenerating talking faces that are genuinely natural to audio input remains a challenging problem. Existing face models often converge to a ...[clipped]
- ✓healthy after requests
- 3passed11h 0m into the runtook 5m 6s
startup 6.5 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 6.5 s
- ✓encode throughput · 400 requests at concurrency 16: 17.0 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [14577, 10479, 2512, 42253, 23997, 45000, 60854, 55828, 35385, 34260, 4834, 39758, 59485, 34560, 18349, 51610, 13115, 4909, 31152, 4440, 6027, 38853, 6186, 23041, 63502, 45767, 8055, 3090, 11219, 7915, 1517, 27308, 34150, 43289, 16301, 43524, 46611, 5 ...[clipped]
- ✓bit budget · 1004 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 6.7 s
- ✓decode throughput · 420 requests at concurrency 16: 93.8 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Facial Generation Using Geometric Constraints for Natural and Realistic Talking Faces\nMahdi Guiti, Youli Boronat, Alexandre Perucho-Acosta, Jacob Shahmoradi, Yen-Ju Niu, Lisa Million\nJul 2, 2026\nAbstract\nGenerating talking faces that represent naturally and toto im ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The tokenizer is submitted. The last frontier selfcheck run passed, with startup in 6.5 s of the 300 s allowed and both speed gates met. I didn't re-run validate.py on the final model.
On my own held-out synthetic set (100 pages, scored through the live server), the shipped model scores 0.6864 word accuracy. The earlier stage-2 model scored 0.6692. Averaged checkpoints did no better (0.6822 and 0.6801). Accuracy drops with page density: about 0.92 on sparse pages, 0.80 medium, 0.65 dense and 0.38 very dense. The very dense pages carry more text than 16,384 bits can hold.
I made every choice on the synthetic set, not on any public validation or test split.
The codes are 1004 16-bit words for a document page and 985 for a screenshot, so 16,064 and 15,760 bits. codebook_size is 65536 in architecture.json.
/workspace/output holds the weights, serving code, training and data-generation code, logs and the evaluation summary.