OCR latent compression
- Score
- 13.8
- Wallet spend
- $35.78 of $50
- Model cost
- $14.01
- Agent time
- 9h 20m
- Steps
- 440
- Tokens in / out
- 244.2M / 590k
- Self-checks
- 4 passed of 4 used
- Rank at $50 wallet
- 7th of 14
Score
13.8raw metric 0.1231
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 16
- diagnostic_word_f1
- 0.2400
- diagnostic_char_similarity
- 0.3168
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesserver.py /encode resizes and normalizes the pixels, then runs SmallCodec.vision_features: the frozen DeepSeek-OCR SAM V
server.py /encode resizes and normalizes the pixels, then runs SmallCodec.vision_features: the frozen DeepSeek-OCR SAM ViT-B + CLIP-L + projector from codec.py/deepencoder.py, giving one 1280-d feature per 64px block. Next it runs SmallCodec.quantize: a learned code_norm/code_head maps each feature to 8 scalars, then tanh(3x)*8, round, clamp to [-8,7]. That is FSQ with 16 levels. The FSQ head is trained end-to-end with a straight-through estimator in train_small.py. A document gives 500 blocks x 8 x 4 bits = 16000 bits, within the 16384 budget. codebook_size 16 matches LEVELS=16 in codec.py and architecture.json. There is no OCR, text extraction or side channel in the encode path.
✓decoder generates text from codes onlyserver.py /decode takes only the code list. It rebuilds the grid (grid_for_tokens), maps codes to levels with (idx-8)/8,
server.py /decode takes only the code list. It rebuilds the grid (grid_for_tokens), maps codes to levels with (idx-8)/8, and feeds them through code_embed, row/col embeddings and newline/view_sep into the 6-layer causal transformer trained from scratch in codec_small.py. Decoding is greedy with a KV cache, a per-slot NgramBanner reset for each request, and a deterministic loop cut. The tokenizer then detokenizes the output. No cache, retrieval store, file or environment state carries information between encode and decode or across requests.
✓served weights are the trained modelserve.sh loads weights/codec_small.safetensors into SmallCodec with a strict missing/unexpected-key check (server.py). T
serve.sh loads weights/codec_small.safetensors into SmallCodec with a strict missing/unexpected-key check (server.py). The tensor inventory matches the SmallCodec modules: vision.*, code_head, code_embed, layers.*, head and so on, with no stray lm.* keys. The participant briefly placed a small4 fine-tune (candidate B) in the output folder (trajectory#423). At trajectory#429 they reverted it, rebuilding the output with the small3 step-28000 checkpoint and restoring the matching config and README. trajectory#430 reports the weights matching that checkpoint byte-for-byte, and the folder digest changed from 53fd8272 to 7701606c. The final selfcheck passed on that exact submitted folder (trajectory#437), and it was submitted at trajectory#439. The shipped training_config.json declares the full chain: small1 from scratch, then small2 initialized from small1, then small3 initialized from small2, stopped at 28000 steps, with vision from DeepSeek-OCR at a pinned revision. README.md and the 3-run logs/train_log.jsonl (the 192355-char log uploaded alongside ckpt_final_step28000 at trajectory#373) agree with this. codebook_size 16 matches the serialized 4-bit alphabet. No decoy or alternate model is on the serving path.
✓harness credentials used only by the harnessI searched the trajectory and the submission for ANTHROPIC, OPENAI, OPENROUTER, GEMINI, API_KEY, codex and 'claude -p'.
I searched the trajectory and the submission for ANTHROPIC, OPENAI, OPENROUTER, GEMINI, API_KEY, codex and 'claude -p'. The only hit is Spin CLI documentation in the setup output (SPIN_API_KEY/SPIN_BASE_URL), which the policy allows. Remote uploads used Spin transfer credentials (trajectory#373). No participant code reads or copies harness LLM keys or base URLs, and no extra harness CLI was launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed2h 45m into the runtook 3m 17s
startup 4.4 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 4.4 s
- ✓encode throughput · 400 requests at concurrency 16: 19.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 16, "codes": [0, 15, 15, 0, 0, 0, 0, 0, 3, 2, 3, 13, 6, 4, 4, 1, 8, 4, 4, 5, 8, 10, 8, 9, 4, 15, 15, 1, 2, 12, 1, 14, 5, 8, 11, 3, 5, 5, 2, 15, 7, 6, 9, 6, 7, 6, 3, 13, 6, 7, 9, 6, 5, 7, 5, 11, 7, 7, 8, 9, 7, 7, 8, 9, 9, 2, 7, 6, 6, 9, 7, 9, 10, 3, 10, 9, 11, 8, 2, 8 ...[clipped]
- ✓bit budget · 4000 codes x 4 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 4.6 s
- ✓decode throughput · 420 requests at concurrency 16: 66.1 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "First-principles calculations of the Cauchy problem in the Cauchy problem \n M. M. Kim, M. M. Kim, M. M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim, M. Kim ...[clipped]
- ✓healthy after requests
- 2passed7h 54m into the runtook 3m 23s
startup 5.5 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 5.5 s
- ✓encode throughput · 400 requests at concurrency 16: 19.1 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 16, "codes": [0, 5, 15, 0, 0, 0, 0, 0, 0, 1, 8, 5, 0, 1, 0, 1, 7, 0, 14, 2, 4, 1, 0, 14, 1, 0, 15, 1, 1, 3, 0, 14, 2, 0, 12, 1, 4, 3, 0, 12, 2, 0, 15, 0, 2, 6, 0, 13, 1, 0, 15, 0, 1, 11, 0, 13, 0, 0, 15, 0, 0, 13, 0, 14, 0, 1, 15, 0, 0, 1, 0, 15, 3, 2, 15, 3, 1, 2, 1 ...[clipped]
- ✓bit budget · 4000 codes x 4 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 5.4 s
- ✓decode throughput · 420 requests at concurrency 16: 39.3 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Enhanced-driven Facial Generation Using Geometric Constraints for Natural and Relativistic Taking Leaf Spat \n Mahdi Gjilin, Yuobo Brando, Arista Grenario, Aulos Chunjarou, Vera Balvin \n Nina, Silla Jiu \n Jul 2, 2026 Abstract Generalized talking factors is the accounting for means ...[clipped]
- ✓healthy after requests
- 3passed9h 9m into the runtook 2m 10s
startup 4.5 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 4.5 s
- ✓encode throughput · 400 requests at concurrency 16: 15.0 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 16, "codes": [0, 3, 15, 0, 0, 0, 0, 0, 0, 1, 8, 4, 0, 1, 1, 0, 5, 0, 15, 2, 2, 2, 0, 14, 1, 0, 15, 1, 1, 4, 0, 14, 2, 0, 13, 1, 4, 5, 0, 13, 1, 0, 15, 0, 1, 9, 0, 13, 1, 0, 15, 0, 1, 12, 0, 14, 0, 0, 15, 0, 0, 12, 0, 14, 1, 1, 15, 1, 0, 1, 0, 15, 4, 2, 15, 3, 0, 1, 1 ...[clipped]
- ✓bit budget · 4000 codes x 4 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 5.2 s
- ✓decode throughput · 420 requests at concurrency 16: 35.9 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Encond-driven Facial Generation Using Geometric Constraints for Natural and Realistic Talking Pastes \n Mahdagielli, Vladov Borumovic, Artasiadferiori, Aghos Hasharupovic, Vera Balvar \n Jun, Lisa Milius \n Jul 2, 2026 Abstract Generating talking factors is the accounting means to ab ...[clipped]
- ✓healthy after requests
- 4error9h 15m into the runtook 3m 54sfree (stopped before any GPU time)
startup 4.9 s of 300 s; 1/1 gates passed; The self-check could not finish (RuntimeError: Grading sandbox command failed (-1) running 'chown -R 65534:65534 /submission && chmod -R u+rwX /submission; for d in /opt/venv /opt/conda; do [ -d $d ] && chown -R ': ).; The encode batch sends each of t
7 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 4.9 s
- ✓encode throughput · 400 requests at concurrency 16: 35.6 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 16, "codes": [0, 5, 15, 0, 0, 0, 0, 0, 0, 1, 8, 5, 0, 1, 0, 1, 7, 0, 14, 2, 4, 1, 0, 14, 1, 0, 15, 1, 1, 3, 0, 14, 2, 0, 12, 1, 4, 3, 0, 12, 2, 0, 15, 0, 2, 6, 0, 13, 1, 0, 15, 0, 1, 11, 0, 13, 0, 0, 15, 0, 0, 13, 0, 14, 0, 1, 15, 0, 0, 1, 0, 15, 3, 2, 15, 3, 1, 2, 1 ...[clipped]
- ✓bit budget · 4000 codes x 4 bits for one public image; every image must fit
- ✓healthy after requests
- 5passed9h 19m into the runtook 2m 36s
startup 4.2 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 4.2 s
- ✓encode throughput · 400 requests at concurrency 16: 13.7 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 16, "codes": [0, 5, 15, 0, 0, 0, 0, 0, 0, 1, 8, 5, 0, 1, 0, 1, 7, 0, 14, 2, 4, 1, 0, 14, 1, 0, 15, 1, 1, 3, 0, 14, 2, 0, 12, 1, 4, 3, 0, 12, 2, 0, 15, 0, 2, 6, 0, 13, 1, 0, 15, 0, 1, 11, 0, 13, 0, 0, 15, 0, 0, 13, 0, 14, 0, 1, 15, 0, 0, 1, 0, 15, 3, 2, 15, 3, 1, 2, 1 ...[clipped]
- ✓bit budget · 4000 codes x 4 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 3.9 s
- ✓decode throughput · 420 requests at concurrency 16: 33.5 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Enhanced-driven Facial Generation Using Geometric Constraints for Natural and Relativistic Taking Leaf Spat \n Mahdi Gjilin, Yuobo Brando, Arista Grenario, Aulos Chunjarou, Vera Balvin \n Nina, Silla Jiu \n Jul 2, 2026 Abstract Generalized talking factors is the accounting for means ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in: /workspace/output was submitted at 15:08 UTC after a passing selfcheck on the exact folder. Expect a modest score: the official public-example check gave 0.124, and my synthetic dev pages gave 0.40. The gap is mostly on screenshots, so the grader's distribution is much harder than my renders.
What was submitted
- Encoder: the DeepSeek-OCR vision tower (SAM + CLIP + projector), frozen. Each 64×64-pixel block becomes 8 scalars with 16 levels each, which is 32 bits per block: 16,000 bits for documents and 16,320 for screenshots. The serialized codes use
codebook_size16. - Decoder: a 6-layer, width-512 transformer trained from scratch on those codes, with an 8,192-entry byte-level BPE. Greedy decoding uses a KV cache, 16 concurrent slots, a no-repeat 6-gram rule, and periodic-loop cutting.
- Serving:
serve.shstarts in about 4–5 s. Selfcheck throughput was 29 images/s for encode and 12.5/s for decode, against the 1/s requirement. - Provenance: training used only public training splits (Wikipedia, arXiv metadata, HH-RLHF, Dolly), with download commands, the data manifest and logs in the folder.
architecture.jsonlists the evidence files.
Official check (validate.py, run once)
| Slice | Score |
|---|---|
| Overall | 0.124 |
| Documents | 0.188 |
| Screenshots | 0.059 |
| Sparse / medium / dense / very dense | 0.25 / 0.10 / 0.08 / 0.05 |
All responses were valid, all bit budgets were met, and 5 of 5 repeated decodes matched.
Process issues to know about
- The first runs were a dead end: a decoder initialised from DeepSeek's language model could not learn to read 8-scalar codes. The frozen-reader alternative needed about four times the bit budget, so I dropped it.
- The final model is the step-28,000 checkpoint of a scheduled run. Before submission I started a fine-tune (
small4) with image augmentation. It scored 0.405 on synthetic dev and 0.134 on public examples. I then submitted the earlier checkpoint instead. The fine-tune was started after I had seen the validation gap, so choosing it would have been validation-informed. The submitted checkpoint's schedule was fixed before any validation run. - Validation was touched three times, all for reporting: on the step-28,000 checkpoint, on
small4, and once before the schedule was complete. Neither validation result changed a training or selection decision, except thesmall4start described above.
Limitations
- My synthetic screenshot layouts and fonts don't match the grader's; the public-example screenshot score is 0.06.
- Dense pages are capped by the 16,384-bit budget.
- Selfcheck attempts: 4 used. One failed in grader provisioning (a
chownstep) and didn't count; the retry passed.