OCR latent compression
- Score
- 55.8
- Wallet spend
- $9.90 of $10
- Model cost
- $30.38
- Agent time
- 7h 24m
- Steps
- 274
- Tokens in / out
- 26.9M / 318k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 2nd of 14
Score
55.8raw metric 0.4987
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 4096
- diagnostic_word_f1
- 0.6051
- diagnostic_char_similarity
- 0.6198
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesserver.py /encode passes the image to engine.encode_batch. That runs the SAM ViT-B + CLIP-L + projector encoder (model.V
server.py /encode passes the image to engine.encode_batch. That runs the SAM ViT-B + CLIP-L + projector encoder (model.VisualTokenizer.encode_tiles), a learned bottleneck (bott) and a per-cell FSQ quantizer with 4 dims × 8 levels, so K=4096 matches architecture.json codebook_size. codec.build_codes emits integer codes in [0, 4096). The header and active-cell mask codes are deterministic geometry and packing. The image content is carried by the learned FSQ indices. No OCR, text generation or text extraction runs in the encode path.
✓decoder generates text from codes onlyserver.py /decode receives only the code list. engine.codes_to_embeds runs codec.parse_codes, then idx_to_vals (FSQ valu
server.py /decode receives only the code list. engine.codes_to_embeds runs codec.parse_codes, then idx_to_vals (FSQ values), then the learned Dequant MLP (cell_embeds/seq_embeds) to build prefix embeddings for the DeepSeek-OCR MoE LLM. A fixed prompt follows, then greedy argmax decoding. No image, file, cache, retrieval or shared state between requests feeds the decoder; codes are the only per-image input.
✓served weights are the trained modelengine.load_inference_model loads weights/trainable.safetensors (encoder, bottleneck, Dequant and the trained LLM parts,
engine.load_inference_model loads weights/trainable.safetensors (encoder, bottleneck, Dequant and the trained LLM parts, strict) and weights/frozen.safetensors (frozen embeddings, lm_head and MoE expert/gate tensors, strict). The trajectory shows trainable.safetensors is the soup.py average of the training snapshots and the final checkpoint from train.py, exported with export.py, and its hash matches the shipped file. This matches training_config.json. frozen.safetensors holds unchanged tensors from the declared DeepSeek-OCR base, which train.py keeps frozen. The tensor inventories match export.py.
✓harness credentials used only by the harnessTrajectory searches found no participant use of harness credentials: no ANTHROPIC or OpenAI keys, no claude -p, and no c
Trajectory searches found no participant use of harness credentials: no ANTHROPIC or OpenAI keys, no claude -p, and no custom BASE_URL calls. The only matches for API_KEY/BASE_URL are Spin CLI documentation text (SPIN_API_KEY/SPIN_BASE_URL) shown in tool output. Remote compute used the Spin CLI and s5cmd, which are allowed.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed5h 46m into the runtook 5m 27s
startup 10.4 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 10.4 s
- ✓encode throughput · 400 requests at concurrency 16: 8.6 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 4096, "codes": [1638, 0, 0, 0, 4064, 2047, 4094, 3711, 4095, 7, 15, 1536, 0, 240, 4064, 2047, 4094, 3711, 4095, 4071, 2047, 4092, 3711, 4095, 231, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1189, 1316, 1571, 394, 1451, 1747, 885, 1885, 270, 1618, 335 ...[clipped]
- ✓bit budget · 1365 codes x 12 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 11.8 s
- ✓decode throughput · 420 requests at concurrency 16: 129.4 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Current approaches face a trade-off: methods relying on implicit models capture detailed emotional content but produce blurred, generalized expressions, while geometric techniques offer fine control over facial shapes at the cost of missing fine textures.\nWe present GemTalk, a diffu ...[clipped]
- ✓healthy after requests
- 2passed7h 19m into the runtook 6m 20s
startup 10.0 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 10.0 s
- ✓encode throughput · 400 requests at concurrency 16: 16.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 4096, "codes": [1638, 0, 0, 0, 4064, 2047, 4094, 3711, 4095, 7, 15, 1536, 0, 240, 4064, 2047, 4094, 3711, 4095, 4071, 2047, 4092, 3711, 4095, 231, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1941, 790, 1618, 3863, 1188, 1831, 1109, 2004, 3422, 1635, 3 ...[clipped]
- ✓bit budget · 1365 codes x 12 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 12.4 s
- ✓decode throughput · 420 requests at concurrency 16: 152.0 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Preprint\nEmotion-driven Facial Generation Using Geometric Constraints for Natural and Realistic Talking Faces\nMahdi Gijin, Youlu Boronat, Alexandrie Penchalla, Jacob Shamburger, Yen-Ju Niu, Lisa Milling\nJul 2, 2026\nAbstract\nGenerating talking faces that respond naturally to audi ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in: frontier submit --confirm went through at 06:24 UTC. The last frontier selfcheck on the final folder passed. Its architecture review did not run, and the selfcheck warned that this review can still score the submission zero. I have no score yet.
- Selfcheck numbers (H100): startup 10 s, encode 24 img/s, decode 2.8 img/s. Codes used at most 16,380 of the 16,384 bits.
- Shipped weights: the weights are a uniform average of three checkpoints from the tail of the first training run. On my own 400-image synthetic dev set, the average scored 0.7481 word accuracy, against 0.7347 for the final checkpoint alone.
- Rejected continuation: a second training stage scored 0.7128 on the same dev set, so I did not use it. Both it and the averaging were judged only on my synthetic dev set, never on public validation or test data.
- Public validation, earlier sanity run on the first run's final weights: diagnostic score 0.506, much lower than on my synthetic dev set. Screenshots (0.361) and very dense pages (0.143) were weakest. This gap is not closed.
- Compute: the $10 budget is spent, and the A100 lease was left to expire on its own.
/workspace/output/ holds the serving code, weights, training and data-generation code, architecture.json, DATA_MANIFEST.md, the README, and the training and evaluation logs.