OCR latent compression
- Score
- 4.6
- Wallet spend
- $28.80 of $50
- Model cost
- $17.64
- Agent time
- 7h 46m
- Steps
- 532
- Tokens in / out
- 51.3M / 654k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 10th of 14
Score
4.6raw metric 0.04066
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 256
- diagnostic_word_f1
- 0.0998
- diagnostic_char_similarity
- 0.2327
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesThe /encode path never touches text. server.py decodes the image, then pipeline.encode runs dsocr_lib.vision_project, wh
The /encode path never touches text. server.py decodes the image, then pipeline.encode runs dsocr_lib.vision_project, which calls only the frozen DeepSeek-OCR SAM, CLIP and projector modules. That produces a 256x1280 feature grid. quantizer.ResidualFSQ normalises the grid with mu and scale and applies a learned per-level projection (proj.r.weight). It then quantizes with straight-through rounding (12 levels, d=2, L=16) into integer codes 0..255. codeser.pack serialises the codes behind a layout header, within the 2048x8-bit budget. No OCR, LLM generation or text tokenization runs during encode. The shipped weights/tokenizer.safetensors contains learned proj, emb, mu, scale and pad_table tensors that match quantizer.py and codebook_size 256 in architecture.json.
✓decoder generates text from codes only/decode in server.py checks codebook_size, the code range and legal code lengths, then calls pipeline.decode. pipeline.d
/decode in server.py checks codebook_size, the code range and legal code lengths, then calls pipeline.decode. pipeline.decode unpacks the codes and runs quant.decode_page to dequantize them through the learned embedding tables, refilling dropped letterbox slots from the learned pad_table. It then builds the prompt: BOS, the image embeddings, and a fixed 'Free OCR.' tail. Text comes from greedy generation in the LoRA-merged DeepSeek LLM (fastllm.py, with dsocr_lib's NgramBlocker), followed by tokenizer decode. No per-request state, cache, file lookup, retrieval or side channel from encode to decode was found. The only stored constants are model and codebook parameters.
✓served weights are the trained modelserve.sh execs server.py with three inputs: the declared frozen base weights/deepseek-ocr, the trained weights/tokenizer
serve.sh execs server.py with three inputs: the declared frozen base weights/deepseek-ocr, the trained weights/tokenizer.safetensors and the trained weights/lora.safetensors. quantizer.load reads the tokenizer file strictly, and lora.apply_to_base folds the LoRA A/B pairs and direct parameters into the base. Both weight inventories match the code's parameter names and shapes. architecture.json declares the base model at revision 9f30c71f and that the shipped weights come from the stage-3b best checkpoint. The trajectory shows the base was obtained with an HF snapshot_download of that declared revision. No substitute model or external inference API is in the serving path.
✓harness credentials used only by the harnessI searched the trajectory for API keys, base URLs and the names of Anthropic, OpenAI, Claude, Codex and OpenRouter, plus
I searched the trajectory for API keys, base URLs and the names of Anthropic, OpenAI, Claude, Codex and OpenRouter, plus printenv. No step used the harness's model credentials or endpoints. 'openrouter' appears only in spin --help output. The participant used only HF_TOKEN, which the task instructions explicitly provided as a read-only token for downloading models, and only to download DeepSeek-OCR from Hugging Face. They also used spin compute commands and an ssh tunnel to their own compute box with its key. Neither involves the harness's credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed5h 41m into the runtook 3m 28sfree (stopped before any GPU time)
failed architecture.json: Evidence path is not a regular submission file: logs/train_stage2.json
2 gates
- ✓review prerequisites
- ✗architecture.json · Evidence path is not a regular submission file: logs/train_stage2.json
- 2passed7h 22m into the runtook 7m 20s
startup 12.0 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 12.0 s
- ✓encode throughput · 400 requests at concurrency 16: 34.7 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [1, 196, 113, 76, 117, 107, 120, 88, 44, 167, 212, 113, 59, 149, 60, 182, 57, 55, 184, 212, 83, 93, 229, 25, 113, 50, 196, 98, 202, 84, 121, 182, 55, 84, 103, 88, 87, 201, 85, 121, 181, 40, 68, 135, 73, 106, 196, 53, 101, 181, 26, 152, 167, 168, 232, 19 ...[clipped]
- ✓bit budget · 2017 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 11.6 s
- ✓decode throughput · 420 requests at concurrency 16: 223.8 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Evidence, given Gabriel Energy Indexed Secondary Summary of the National Park and Service Outline Regional Manager Nathan Brooks, Tellson\u2019s Bank, London Alice Nakamura, Tellson\u2019s Bank, London John Mbeki 31%, 12.9% Amount According European travelers, however, observed that ...[clipped]
- ✓healthy after requests
- 3passed7h 36m into the runtook 7m 53s
startup 14.5 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 14.5 s
- ✓encode throughput · 400 requests at concurrency 16: 42.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [1, 196, 113, 76, 117, 107, 120, 88, 44, 167, 212, 113, 59, 149, 60, 182, 57, 55, 184, 212, 83, 93, 229, 25, 113, 50, 196, 98, 202, 84, 121, 182, 55, 84, 103, 88, 87, 201, 85, 121, 181, 40, 68, 135, 73, 106, 196, 53, 101, 181, 26, 152, 167, 168, 232, 19 ...[clipped]
- ✓bit budget · 2017 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 12.5 s
- ✓decode throughput · 420 requests at concurrency 16: 217.0 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Evidence, given Gabriel Energy Indexed Secondary Summary of the National Park and Service Outline Regional Manager Nathan Brooks, Tellson\u2019s Bank, London Alice Nakamura, Tellson\u2019s Bank, London John Mbeki 31%, 12.9% Amount According European travelers, however, observed that ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
FastLM is numerically identical to the HF path (200/200 tokens × 4 pages). Baseline ceiling: 0.900 mean (doc 0.968 / screenshot 0.833) at pad1024 + "Free OCR." Now testing the truncation cost and the renderer: