OCR latent compression
- Score
- 2.2
- Wallet spend
- $20.94 of $50
- Model cost
- $1.18
- Agent time
- 3h 14m
- Steps
- 439
- Tokens in / out
- 43.5M / 509k
- Self-checks
- 2 passed of 3 used
- Rank at $50 wallet
- 15th of 19
Score
2.2raw metric 0.01956
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 256
- diagnostic_word_f1
- 0.0317
- diagnostic_char_similarity
- 0.1417
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesThe codes come from a learned encoder. In server.py, Engine.encode_batch calls the trained VisualEncoder from model.py,
The codes come from a learned encoder. In server.py, Engine.encode_batch calls the trained VisualEncoder from model.py, which runs a conv stem, 12 windowed-attention ViT blocks, adaptive pooling to a 32x64 grid, lat_proj and FSQ quantization. That gives 2048 codes from an alphabet of 256 (4^4), so 8 bits per code and 16384 bits in total, matching cfg.json and architecture.json. train.py starts the encoder from random weights and trains it end to end with the decoder. The /encode path has no OCR or text-extraction step. The DeepSeek-OCR files under /artifacts were only looked at (trajectory#11) and are not used by any shipped code.
✓decoder generates text from codes onlyIn server.py, decode_batch dequantizes the submitted codes with fsq_indices_to_z, runs mem_proj plus mem_pos to build th
In server.py, decode_batch dequantizes the submitted codes with fsq_indices_to_z, runs mem_proj plus mem_pos to build the cross-attention memory, and then runs the Qwen-style neural Decoder from model.py. It uses either greedy generate (argmax) or the non-autoregressive parallel_decode (argmax). The choice between the two comes from a linear classifier stored in kind.json, applied to code-derived features. The in-memory cache is keyed on tuple(codes). Apart from the codes, the only inputs are fixed shipped files: cfg.json, kind.json, tokenizer.json and model.safetensors. The decoder never sees the image, any OCR text or any retrieved reference text.
✓served weights are the trained modelserve.sh runs server.py, which loads model.safetensors with strict=True into the enc.* and dec.* modules. The weight inv
serve.sh runs server.py, which loads model.safetensors with strict=True into the enc.* and dec.* modules. The weight inventory lists only enc.* and dec.* tensors, including cross_attn, mem_proj and mem_pos, and no vdec.* keys. That matches the model train.py builds, with the training-only visual decoder removed. In the trajectory, run4/latest.safetensors, written by train.py, was copied to keep_ar.safetensors (trajectory#356-372). pack2.py removed the vdec.* tensors, cast to bf16 and wrote final2/model.safetensors (trajectory#393-395). That file was verified (trajectory#409-422) and copied into the output, where its sha256 was checked (trajectory#423-439). The decoder was initialized from Qwen/Qwen2.5-0.5B at the pinned revision declared in architecture.json and data_manifest.json, via init_decoder_from_qwen in train.py.
✓harness credentials used only by the harnessI found no misuse of credentials. The read-only HF_TOKEN was mentioned only in the task prompt (trajectory#1), and the l
I found no misuse of credentials. The read-only HF_TOKEN was mentioned only in the task prompt (trajectory#1), and the logged downloads ran unauthenticated (init_log.txt). The trajectory was searched for $HF_TOKEN, API_KEY, MLE_JUDGE_API_KEY, BROKER_TOKEN, openrouter keys and claude CLI calls. The matches were only documentation and help text (trajectory#12, trajectory#300-317). No openrouter key was created, no model API was called, and no judge key was used. The spin compute and SSH leases were the participant's own budgeted compute resources, not harness credentials. No credentials appear in the shipped files.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1failed2h 14m into the runtook 9m 29s
startup 20.0 s of 300 s; 1/2 gates passed; failed decode throughput: Batch exceeded 420s deadline; score is zero (358 of 420 requests answered in 420.0 s at concurrency 16); The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server
9 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 20.0 s
- ✓encode throughput · 400 requests at concurrency 16: 22.1 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [197, 197, 197, 197, 197, 197, 133, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 6 ...[clipped]
- ✓bit budget · 2048 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 20.6 s
- ✗decode throughput · Batch exceeded 420s deadline; score is zero (358 of 420 requests answered in 420.0 s at concurrency 16)
- 2passed2h 27m into the runtook 7m 49s
startup 14.5 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 14.5 s
- ✓encode throughput · 400 requests at concurrency 16: 24.1 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [197, 197, 197, 197, 197, 197, 133, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 6 ...[clipped]
- ✓bit budget · 2048 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 13.7 s
- ✓decode throughput · 420 requests at concurrency 16: 327.5 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Fieldwork\nSearch\nLatest\nTech\nScience\nCulture\nOpinion\nPRODUCT\nSubjective value theory\nKaren Gonzalez\nAug 1, 2025 \u00b7 10 min read\nFrom September onwards Australia struggled financially owing to declining exports comprising nearly half exports totaling \u20b9 70 crore \u ...[clipped]
- ✓healthy after requests
- 3passed3h 10m into the runtook 3m 28s
startup 9.0 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 9.0 s
- ✓encode throughput · 400 requests at concurrency 16: 20.4 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [85, 85, 85, 85, 149, 149, 101, 106, 102, 102, 102, 102, 106, 106, 106, 106, 106, 106, 106, 106, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 101, 101, 101, 101, 101, 102, 1 ...[clipped]
- ✓bit budget · 2048 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 9.7 s
- ✓decode throughput · 420 requests at concurrency 16: 48.3 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Overview the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 2026-10-20 22:50:11 UTC.
What was shipped (/workspace/output, 27 files, 1.51 GB): a discrete visual tokenizer — conv-stem shifted-window ViT encoder → 32×64 grid → FSQ 4×4 (256 entries) → exactly 2,048 codes × 8 bits = 16,384 bits, decoded by a plain-PyTorch Qwen2.5-0.5B with cross-attention in all 24 layers, initialized from Qwen/Qwen2.5-0.5B@060db649.
Final architecture at decode time: the decode instance sees only codes, so a 133-feature linear score over the code grid (fit on rendered training pages, 100% accurate on held-out renders and dev) picks one of two branches — one blank-prefix forward for document pages, 128 greedy steps for screenshots. Same network and weights; no retrieval, no stored transcripts.
Verified before submitting:
frontier selfcheck: all gates pass — startup 9 s, encode 19.6 img/s, decode 8.7 img/s (required 1), bit budget exactly 16,384, healthy after request storms./task/validate.pyagainst two fresh instances on the pod: 0 failed requests, 0/5 determinism mismatches, 6.7 decode img/s.
The honest result: the decoder only weakly uses the visual codes. Swapping a page's codes for another page's leaves teacher-forced CE at 5.008 vs 5.012 and 94.8% of predictions identical, while the codes themselves vary strongly (std 0.69). I attacked this directly — a blank-prefix objective that makes the codes the only input, and a pure blank-prefix run — but 1.4k steps moved the dev score only 0.034 → 0.036 while destroying the autoregressive path, so I stopped it and shipped the best measured configuration. Dev similarity was 0.075 routed, but that set shares the training renderer; the public-validation diagnostic, run once after all choices were frozen and never used for selection, was 0.021.
Training/calibration provenance (wikitext-103 train only, validation/test splits explicitly excluded, with download commands), three phase logs, and the selection scripts are all included as evidence files.