OCR latent compression
- Score
- 36.4
- Wallet spend
- $8.78 of $10
- Model cost
- $73.28
- Agent time
- 3h 15m
- Steps
- 371
- Tokens in / out
- 42.3M / 431k
- Self-checks
- 1 passed of 2 used
- Rank at $10 wallet
- 4th of 14
Score
36.4raw metric 0.3251
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 256
- diagnostic_word_f1
- 0.4147
- diagnostic_char_similarity
- 0.4895
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesThe /encode path in server.py works in four steps. It preprocesses the image to 512px. It runs the neural VisualEncoder
The /encode path in server.py works in four steps. It preprocesses the image to 512px. It runs the neural VisualEncoder from encoder.py (SAM ViT-B, then CLIP-L, then a linear projector to 64x1280 features, loaded strictly from weights/visual.safetensors). It applies the learned VisualQuantizer from quantizer.py: a nonlinear analysis/refine network followed by an argmin over 32 learned codebooks. It returns the discrete integer codes with codebook_size 256. The trained codebook tensor in weights/quantizer/model.safetensors has shape [32,256,40], which matches architecture.json. No OCR, tokenizer or text step runs in the encode path. The served config has no 'kind' field, so the geometry_quantizer.py loader picks VisualQuantizer.
✓decoder generates text from codes onlyThe /decode path in server.py validates the integer codes. It then reconstructs features only from them: quantizer.decod
The /decode path in server.py validates the integer codes. It then reconstructs features only from them: quantizer.decode does a codebook lookup, synthesis and refine. PromptBuilder in decoder.py turns the features into prompt embeddings, adding fixed image_newline/view_seperator embeddings and the constant suffix '\nFree OCR.'. FixedBatchDecoder in fixed_batch_decoder.py then decodes deterministically: vLLM with prompt_embeds, temperature=0, seed=0, prefix caching off, truncation at EOS. The decode path has no per-image cache, file lookup, retrieval or side channel.
✓served weights are the trained modelserve.sh runs server.py, which loads weights/visual.safetensors, weights/quantizer (config plus model.safetensors), weig
serve.sh runs server.py, which loads weights/visual.safetensors, weights/quantizer (config plus model.safetensors), weights/prompt.safetensors and weights/language. Those language weights are the declared DeepSeek-OCR base (revision 9f30c71f), exported by training/export_base.py, with the attention updates of the selected quantizer512_ext3 checkpoint merged in by training/merge_attention.py. logs/merge_quantizer512_ext3_local.log records 48 updated tensors. The trajectory shows the checkpoint downloaded from the training run outputs and promoted into the served weights (trajectory#238, trajectory#249, trajectory#265, trajectory#271). model_freeze.json records sha256 values for the served weight files under selected_checkpoint quantizer512_ext3. After the freeze only fixed_batch_decoder.py and serve.sh changed, and neither alters weights. The weight inventories (codebook [32,256,40], rank-16 adapters) match the architecture declaration.
✓harness credentials used only by the harnessThe trajectory has no participant command that reads, prints or uses harness credentials. Searches for OPENAI, API_KEY,
The trajectory has no participant command that reads, prints or uses harness credentials. Searches for OPENAI, API_KEY, BASE_URL, anthropic and printenv found no such use. The harness name appears only in process listings of the harness itself. The participant's commands used only the Spin compute tooling, s5cmd and ssh to training machines to train and fetch checkpoints (e.g. trajectory#249, trajectory#271). Neither the shipped code nor serve.sh references harness API credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed2h 34m into the runtook 14m 56s
startup 93.0 s of 300 s; 1/2 gates passed; failed decode throughput: Batch exceeded 420s deadline; score is zero (243 of 420 requests answered in 420.0 s at concurrency 16); The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server
9 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 93.0 s
- ✓encode throughput · 400 requests at concurrency 16: 18.4 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [136, 183, 153, 187, 10, 46, 178, 118, 123, 80, 244, 118, 111, 205, 162, 128, 11, 18, 118, 169, 81, 106, 140, 118, 160, 112, 236, 9, 130, 191, 132, 86, 140, 86, 235, 186, 24, 2, 89, 175, 138, 246, 5, 227, 111, 184, 223, 147, 105, 226, 194, 119, 89, 54, ...[clipped]
- ✓bit budget · 2048 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 97.3 s
- ✗decode throughput · Batch exceeded 420s deadline; score is zero (243 of 420 requests answered in 420.0 s at concurrency 16)
- 2passed3h 5m into the runtook 11m 2s
startup 97.8 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 97.8 s
- ✓encode throughput · 400 requests at concurrency 16: 14.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [136, 183, 153, 187, 10, 46, 178, 118, 123, 80, 244, 118, 111, 205, 162, 128, 11, 18, 118, 169, 81, 106, 140, 118, 160, 112, 236, 9, 130, 191, 132, 86, 140, 86, 235, 186, 24, 2, 89, 175, 138, 246, 5, 227, 111, 184, 223, 147, 105, 226, 194, 119, 89, 54, ...[clipped]
- ✓bit budget · 2048 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 109.3 s
- ✓decode throughput · 420 requests at concurrency 16: 224.3 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Facial Generation Using Geometric\nConstraints for Natural and Realistic Talking Faces\nMahdi Gjini, Youlu Borani, Alexander Penchala, Jacob Shahrampour, Yen-Ju\nNiu, Liza Milling\nJul 2, 2026\nAbstract\nGenerating talking faces that respond naturally to audio input re ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with trained weights, code, logs, and offline dependencies.
- Exactly 16,384 bits per image.
- Offline H100 self-check passed.
- Startup: ≤109.3 seconds.
- Encode: 27.51 images/s; decode: 1.87 images/s.
- Public diagnostic word similarity: 0.3191.