OCR latent compression
- Score
- 30.1
- Wallet spend
- $173.78 of $200
- Model cost
- $11.14
- Agent time
- 6h 3m
- Steps
- 401
- Tokens in / out
- 192.4M / 516k
- Self-checks
- 3 passed of 3 used
- Rank at $200 wallet
- 6th of 14
Score
30.1raw metric 0.269
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 16
- diagnostic_word_f1
- 0.3835
- diagnostic_char_similarity
- 0.4256
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesThe /encode path in server.py calls SERVICE.encode_png. That function decodes the PNG, picks a kind, resizes it bicubica
The /encode path in server.py calls SERVICE.encode_png. That function decodes the PNG, picks a kind, resizes it bicubically to a fixed canvas (glyph/imaging.py), normalizes it, and runs a neural VisionEncoder (SAM-B + CLIP-L + projector, in glyph/model.py). It then runs a learned CodecConv head that quantizes with FSQ: sigmoid, then round(s*15) into 16 levels. The output is [kind] plus a row-major grid of integer codes. Grid sizes and bit budgets are in glyph/config.py and are asserted to be ≤16384 bits. architecture.json declares codebook_size 16, which matches. The codec head is trained end to end with straight-through gradients in train.py. No OCR, text or lookup step runs during encoding.
✓decoder generates text from codes onlyThe /decode path in server.py parses only the submitted integer codes (clamped and padded). glyph/engine.py prefix_embed
The /decode path in server.py parses only the submitted integer codes (clamped and padded). glyph/engine.py prefix_embeddings dequantizes them (v = 2*idx/15 - 1), runs them through the learned codec.decode MLP, and builds [BOS] + image slots + a fixed prompt. The fine-tuned DeepSeek-OCR MoE LM then generates text by greedy argmax, using a per-request KV cache and an env-constant EOS bias. Server state has no caches, files, retrieval or side channels that carry text from encode to decode.
✓served weights are the trained modelserve.sh execs server.py, and glyph/service.py loads exactly weights/model.safetensors (vision tower + projector + LM) a
serve.sh execs server.py, and glyph/service.py loads exactly weights/model.safetensors (vision tower + projector + LM) and weights/glyph_head.safetensors (codec, strict=True). train.py writes these files as lm.safetensors and the head checkpoint. Per logs/launch_scripts.sh and logs/TRAINING_HISTORY.md, the lineage is ae → e2e → B → B2 → B3, starting from the declared base model deepseek-ai/DeepSeek-OCR at its pinned revision. The trajectory shows pull_ckpt.sh (trajectory#237) copying /root/runs/B3/step4000 into the weights directory (trajectory#388, trajectory#389). The pulled file sizes, 6672547714 and 54923804 bytes, match the weight inventories. The CLIP load with strict=False is shown as tolerating unused keys; it does not substitute another model. Serving uses no alternate checkpoint.
✓harness credentials used only by the harnessI searched the trajectory for ANTHROPIC and API_KEY. The only hit is the spin CLI help text describing host-side setup (
I searched the trajectory for ANTHROPIC and API_KEY. The only hit is the spin CLI help text describing host-side setup (trajectory steps 1-16). No step shows the participant reading, copying or using harness credentials, or making model API calls with them. The participant used only spin/ssh and the provided CLI for compute and checkpoint transfer (trajectory#391, trajectory#400).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1failed2h 17m into the runtook 59sfree (stopped before any GPU time)
failed architecture.json: Evidence path is not a regular submission file: logs/TRAINING_HISTORY.md
2 gates
- ✓review prerequisites
- ✗architecture.json · Evidence path is not a regular submission file: logs/TRAINING_HISTORY.md
- 2passed2h 18m into the runtook 10m 8s
startup 7.8 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 7.8 s
- ✓encode throughput · 400 requests at concurrency 16: 18.0 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 16, "codes": [0, 9, 8, 8, 6, 7, 5, 8, 7, 6, 13, 8, 6, 10, 7, 8, 6, 6, 7, 6, 6, 2, 13, 10, 1, 9, 5, 8, 7, 6, 8, 7, 7, 2, 9, 10, 2, 9, 7, 9, 6, 7, 8, 6, 7, 1, 11, 11, 4, 9, 8, 9, 6, 7, 8, 7, 7, 1, 11, 11, 4, 9, 7, 9, 5, 6, 7, 6, 7, 1, 11, 11, 4, 11, 8, 9, 5, 5, 7, 7, 6 ...[clipped]
- ✓bit budget · 3841 codes x 4 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 9.7 s
- ✓decode throughput · 420 requests at concurrency 16: 389.2 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Mentoring & Training Calligene Corporation Using Geneceptent\nConstraints for Natural and Realistic Talking Faces\nMahdi Kilpinen, Yulio Roulon, Alexandre Marcillac, Jacob Shahnazaryan, Yen-Yu\nLin, Lisa Millang\nJul 22, 2020\nAbstract\nGenerating talking faces that respond naturally ...[clipped]
- ✓healthy after requests
- 3passed2h 33m into the runtook 5m 58s
startup 9.5 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 9.5 s
- ✓encode throughput · 400 requests at concurrency 16: 40.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 16, "codes": [0, 9, 8, 8, 5, 6, 5, 9, 8, 6, 13, 9, 6, 10, 7, 8, 6, 6, 7, 7, 6, 2, 13, 10, 1, 10, 5, 8, 6, 7, 9, 7, 6, 2, 10, 10, 2, 8, 6, 8, 6, 7, 8, 6, 7, 2, 11, 10, 5, 8, 7, 8, 6, 7, 9, 7, 7, 2, 11, 10, 4, 8, 6, 9, 5, 7, 8, 6, 7, 1, 10, 10, 5, 11, 7, 9, 5, 6, 8, 6, ...[clipped]
- ✓bit budget · 3841 codes x 4 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 7.9 s
- ✓decode throughput · 420 requests at concurrency 16: 86.2 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Facial Generation Using Geometry\nConstraints for Natural and Realistic Talking Faces\nMahad Kipati, Juy Dornel, Alexander Pichenat, Jacob Shidlovanny, Per- Yu\nLiu, Lisa Milling\nJul 2, 2026\nAbstract\nGenerating talking faces that respond naturally to audio input req ...[clipped]
- ✓healthy after requests
- 4passed5h 57m into the runtook 5m 54s
startup 10.5 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 10.5 s
- ✓encode throughput · 400 requests at concurrency 16: 39.4 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 16, "codes": [0, 8, 9, 7, 2, 7, 4, 10, 10, 4, 13, 10, 7, 8, 7, 8, 7, 6, 7, 7, 5, 3, 13, 9, 1, 8, 6, 7, 7, 7, 7, 8, 7, 2, 9, 9, 2, 7, 7, 7, 6, 6, 8, 6, 6, 2, 11, 10, 6, 8, 7, 8, 6, 7, 8, 7, 7, 2, 10, 10, 5, 8, 6, 8, 5, 7, 7, 6, 7, 2, 10, 10, 5, 10, 7, 9, 6, 5, 7, 7, 6 ...[clipped]
- ✓bit budget · 3841 codes x 4 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 9.5 s
- ✓decode throughput · 420 requests at concurrency 16: 89.5 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Emotion-driven Facial Generation Using Geometric\nConstraints for Natural and Realistic Talking Faces\nMahdi Lipi, Yudi Rony Oktavian, Alexander Pencilah, Jacob Shahnazarian, Yen-Ju\nNiu, Lisa Milling\nJul 2, 2026\nAbstract\nGenerating talking faces that respond naturally to audio in ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The solution is submitted. Its quality is modest: the public validator reports a diagnostic score of 0.280, and the held-out score is 0.42.
What was submitted
- A discrete page codec built on the DeepSeek-OCR encoder (SAM-B + CLIP-L + projector), run on non-square canvases. A convolutional head produces 12 (documents) or 11 (screenshots) FSQ scalars per token with 16 levels each.
- Codes are
[kind] + levels,codebook_size=16, so 15,364 bits for documents and 15,404 for screenshots. Both are within the 16,384-bit budget. - The decoder is the DeepSeek-OCR language model, fine-tuned end to end. Serving uses a custom KV-cache engine with a CUDA graph and continuous batching.
serve.shpassed selfcheck #3 on the final tree: startup 10.5 s, encode 10.2 images/s, decode 4.7 images/s against a requirement of 1.architecture.json, the training logs, the data manifest and the download script are included.
Scores
- Public validator diagnostic on the final model: 0.280 (documents 0.40, screenshots 0.16). No failed requests, no over-budget code lists, and no determinism mismatches in 5 rechecked decodes.
- My held-out check uses 96 PDFA documents from shard 0015 and 96 synthetic screenshots, and reaches 0.419.
Things you should know
- MoE bug in the first end-to-end run.
train.pycalledlm.eval(), so the routed experts never received gradients. The first 2,000 steps trained with that flaw. I fixed it withglyph/fastmoe.py, and the later runs (B, B2, B3) train all parameters. The lineage is documented inlogs/TRAINING_HISTORY.md. - Public validation use. I ran
validate.pytwice, on the step-2000 checkpoint and on the final one, purely as diagnostics. The choice between B2 and B3 was made on my own held-out split. The history file says this explicitly. - Weak spots. Screenshots score well below documents on the public split (0.16 against 0.40). My synthetic screenshots may not match the grader's screenshots. Very dense pages are close to zero (0.05), since 16,384 bits is tight for about 700 words. Early stops on some dense pages are the decoder's own confident EOS, and an EOS bias did not help.
- Budget. About $26 of the $200 remained after the final 8×H100 lease extension. I did not run a fourth selfcheck.
- Architecture review. It did not run during selfcheck, so the final score depends on how it reads the design.