OCR latent compression
- Score
- 54.4
- Wallet spend
- $40.14 of $50
- Model cost
- $121.44
- Agent time
- 10h
- Steps
- 372
- Tokens in / out
- 105.9M / 484k
- Self-checks
- 3 passed of 3 used
- Rank at $50 wallet
- 4th of 14
Score
54.4raw metric 0.4859
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 65536
- diagnostic_word_f1
- 0.6011
- diagnostic_char_similarity
- 0.6185
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codes/encode in server.py runs preprocess, then encode_levels in ocrvq/pipeline.py, then pack. encode_levels passes the image
/encode in server.py runs preprocess, then encode_levels in ocrvq/pipeline.py, then pack. encode_levels passes the image through the neural DeepSeek-OCR DeepEncoder (SAM-B, CLIP-L, projector) and then the learned bottleneck: LayerNorm, Linear(1280→32), tanh, then FSQ with 4 levels per dimension (ocrvq/model.py). That bottleneck is trained in train.py by next-token cross-entropy through the decoder. Preprocessing (ocrvq/preprocess.py) only does a deterministic crop and grid choice plus an edge-based blank-cell mask. No OCR, transcription or tokenizer step is on the encode path. ocrvq/codec.py packs the presence bits and 2-bit levels into 16-bit words, which matches codebook_size 65536, and asserts the 16384-bit budget.
✓decoder generates text from codes only/decode in server.py only unpacks the submitted codes. codes_to_embeds in ocrvq/pipeline.py dequantizes the FSQ levels t
/decode in server.py only unpacks the submitted codes. codes_to_embeds in ocrvq/pipeline.py dequantizes the FSQ levels through learned layers and builds embeddings from a fixed BOS token, those visual embeddings and a constant prompt. The DeepSeek-3B-MoE LM then decodes greedily (argmax) in ocrvq/engine.py. Each row of a batch is computed independently. No per-request files, caches, retrieval, side channel or shared state carry information from encode to decode. The only fallback is an empty string or a blank-page code computed at startup.
✓served weights are the trained modelserve.sh execs server.py, which loads weights/model.safetensors into OCRVQ with strict=True. The tensor inventory (bottl
serve.sh execs server.py, which loads weights/model.safetensors into OCRVQ with strict=True. The tensor inventory (bottleneck.*, sam.*, clip.*, lm.*, image_newline, view_seperator) matches the model structure. train.py initializes from the declared base, deepseek-ai/DeepSeek-OCR at a pinned revision (architecture.json), and trains the encoder, bottleneck and LM subsets. export.py merges the trained parameters into that initialization. At trajectory#352 the participant exported weights/model.safetensors from the r4 final trainable checkpoint, giving sha256 e75804b7…. At trajectory#366 the hash was rechecked as unchanged after the r5 experiment was evaluated and rejected. At trajectory#367 architecture.json and logs/COMMANDS.md were updated to record r4 final as the selected checkpoint. The final selfcheck passed with this package (trajectory#369-370).
✓harness credentials used only by the harnessI found no misuse of harness credentials in the trajectory. HF_TOKEN was used only to download the declared public corpu
I found no misuse of harness credentials in the trajectory. HF_TOKEN was used only to download the declared public corpus (gfissore/arxiv-abstracts-2021) and models, including on the participant's own Spin GPU box (trajectory#135 chunk, datagen/get_corpus.sh), which policy allows. The Spin CLI was used for compute leases, extensions and the final stop (trajectory#370). SPIN_BROKER_TOKEN, ANTHROPIC and OpenRouter credentials appear only in documentation text or not at all. No step exfiltrates harness credentials or uses them for anything outside the harness.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1failed1h 55m into the runtook 2m 51sfree (stopped before any GPU time)
failed review prerequisites: The review needs architecture.json and serve.sh at the top of the submission
1 gates
- ✗review prerequisites · The review needs architecture.json and serve.sh at the top of the submission
- 2passed1h 58m into the runtook 4m 14s
startup 5.2 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 5.2 s
- ✓encode throughput · 400 requests at concurrency 16: 23.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [21759, 65535, 65535, 65535, 65535, 65535, 248, 0, 0, 7168, 65535, 65535, 57407, 65535, 65535, 65535, 65407, 65535, 65535, 63488, 102, 4056, 59366, 49169, 6244, 4088, 59239, 45081, 6249, 4091, 26452, 46099, 6737, 8949, 61138, 54451, 12864, 36859, 2634 ...[clipped]
- ✓bit budget · 953 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 6.6 s
- ✓decode throughput · 420 requests at concurrency 16: 51.5 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Facial Generation Using Geometric Constraints for Natural and Realistic Talking Faces\nMahdi Gjini, Youlou Boronat, Alexandr Penchala, Jacob Shahrampour, Yen-Ju Niu, Lisa Millington\nAbstract\nWe study the problem of learning a neural network that can be trained to rec ...[clipped]
- ✓healthy after requests
- 3passed9h 6m into the runtook 5m 19s
startup 6.9 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 6.9 s
- ✓encode throughput · 400 requests at concurrency 16: 27.1 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [21759, 65535, 65535, 65535, 65535, 65535, 248, 0, 0, 7168, 65535, 65535, 57407, 65535, 65535, 65535, 65407, 65535, 65535, 63488, 228, 8185, 61055, 53277, 6157, 40293, 45656, 20835, 26696, 46589, 21592, 45871, 12840, 37725, 63098, 53933, 11417, 3509, ...[clipped]
- ✓bit budget · 953 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 6.9 s
- ✓decode throughput · 420 requests at concurrency 16: 93.0 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Facial Generation Using Geometric Constraints for Natural and Realistic Talking Faces\nMahdi Gjini, Youlu Boronat, Alexander Penchala, Jacob Shahrampour, Yen-Ju Sion\nNiu, Lisa and Milling\n2026-07-11\nAbstract\nGenerating and analyzing naturally to audio input require ...[clipped]
- ✓healthy after requests
- 4passed9h 57m into the runtook 5m 7s
startup 6.8 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 6.8 s
- ✓encode throughput · 400 requests at concurrency 16: 29.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [21759, 65535, 65535, 65535, 65535, 65535, 248, 0, 0, 7168, 65535, 65535, 57407, 65535, 65535, 65535, 65407, 65535, 65535, 63488, 228, 8185, 61055, 53277, 6157, 40293, 45656, 20835, 26696, 46589, 21592, 45871, 12840, 37725, 63098, 53933, 11417, 3509, ...[clipped]
- ✓bit budget · 953 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 7.6 s
- ✓decode throughput · 420 requests at concurrency 16: 92.0 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Facial Generation Using Geometric Constraints for Natural and Realistic Talking Faces\nMahdi Gjini, Youlu Boronat, Alexander Penchala, Jacob Shahrampour, Yen-Ju Sion\nNiu, Lisa and Milling\n2026-07-11\nAbstract\nGenerating and analyzing naturally to audio input require ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
GPU released; total compute spend $40.14 of $50. The submission is final, so handing it in as instructed: