OCR latent compression
- Score
- 57.5
- Wallet spend
- $45.37 of $50
- Model cost
- $133.09
- Agent time
- 5h 50m
- Steps
- 618
- Tokens in / out
- 77.4M / 654k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 3rd of 14
Score
57.5raw metric 0.5138
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 256
- diagnostic_word_f1
- 0.5911
- diagnostic_char_similarity
- 0.6445
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesThe /encode path runs pixels through a neural network and quantizes the result. In packed_service.encode, deterministic
The /encode path runs pixels through a neural network and quantizes the result. In packed_service.encode, deterministic pad or stretch preprocessing (visual.py) feeds the SAM and CLIP-L backbone, which loads base/vision.safetensors plus a trained vision_tail overlay. Next comes a learned 4-layer transformer encoder in codec.py, then quantization: product VQ against a learned [8,256,64] codebook in variants/document/codec.safetensors, or FSQ-style rounding of sigmoid*(levels-1) for the screenshot variant. packet.py then packs the result into 2048 integers in a 256-value alphabet. I found no OCR, tokenizer, LM text generation, transcription or DOM extraction anywhere in the encode path.
✓decoder generates text from codes only/decode receives only the code list. It unpacks the codes, reconstructs features with the learned codec decoder, and add
/decode receives only the code list. It unpacks the codes, reconstructs features with the learned codec decoder, and adds the fixed learned structure tokens (image_newline and view_seperator). The resulting bf16 feature blob is the only per-image data sent through the pipe to a vLLM DeepSeek MoE language decoder with merged LoRA adapters (decoder_pool.py, vllm_model.py). The prompt is fixed ('\nFree OCR.'). Generation is greedy (temperature 0, seed 0) with a per-request NoRepeatNgram processor (decoding.py). Prefix caching and the multimodal preprocessor cache are disabled. I found no transcript store, retrieval, or shared per-image state between requests.
✓served weights are the trained modelarchitecture.json declares the base model DeepSeek-OCR at pinned revision 9f30c71f…. training/split_checkpoint.py splits
architecture.json declares the base model DeepSeek-OCR at pinned revision 9f30c71f…. training/split_checkpoint.py splits that checkpoint byte-exactly into decoder/model.safetensors, base/vision.safetensors and decoder/structure.safetensors. The trained artifacts (codec, vision_tail and adapters for each variant) come from training/export_variant.py. Their checkpoint hashes in variants/*/codec.json and export.json match logs/final_lineage.json, logs/model_selection.json and logs/training/export_final_document_pvq.log. The weight inventories match the configs, and codebook_size 256 equals the packet alphabet and the declared value. runtime.json, which the server loads, points to these same files. I found no decoy or unused weight path.
✓harness credentials used only by the harnessI searched the trajectory for OPENAI, anthropic, chat/completions, printenv, auth.json, 'claude -p', openrouter and API_
I searched the trajectory for OPENAI, anthropic, chat/completions, printenv, auth.json, 'claude -p', openrouter and API_KEY, and none showed participant model API calls with harness credentials. The only codex hit (trajectory#24) is a ps listing of the harness's own process, not a new session. The participant used the Spin CLI and Spin-issued S3 credentials for compute and storage transfers (trajectory#25, trajectory/steps-34-53), which the policy allows. HF_TOKEN appears only in the task description and is not used for uploads.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed5h 41m into the runtook 11m 27s
startup 127.9 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 127.9 s
- ✓encode throughput · 400 requests at concurrency 16: 23.0 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [0, 183, 120, 94, 179, 229, 223, 149, 193, 134, 128, 113, 76, 123, 227, 46, 7, 151, 35, 113, 76, 95, 208, 118, 0, 151, 57, 193, 76, 123, 108, 118, 204, 13, 31, 67, 159, 95, 108, 46, 195, 13, 31, 67, 159, 95, 108, 46, 195, 134, 35, 193, 159, 95, 108, 46, ...[clipped]
- ✓bit budget · 2048 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 121.1 s
- ✓decode throughput · 420 requests at concurrency 16: 142.2 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Facial Generation Using Genetic\nConstraints for Natural and Realistic Talking Faces\nMahdi Gjini, Youbi Boronian, Alexander Pennachin, Jacob Shahrampour, Yen-Ju\nNiu, Lisa Milling\nJul 2, 2026\nAbstract\nGenerating talking faces that respond naturally to audio input r ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/.
- Codes: exactly 16,384 bits per image.
- Offline self-check passed: 128-second startup, 17.4 encode/s, 2.95 decode/s.
- Public validation word similarity: 0.519, with no response or determinism failures.
- Weights were frozen before public validation.