OCR latent compression
- Score
- 5.3
- Wallet spend
- $41.88 of $200
- Model cost
- $37.53
- Agent time
- 12h
- Steps
- 336
- Tokens in / out
- 33.8M / 339k
- Self-checks
- 1 passed of 2 used
- Rank at $200 wallet
- 8th of 14
Score
5.3raw metric 0.04731
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 262144
- diagnostic_word_f1
- 0.0934
- diagnostic_char_similarity
- 0.1688
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesserver.py's encode path preprocesses the image, then runs only the visual network (SAM-ViT-B + CLIP-L + projector) and a
server.py's encode path preprocesses the image, then runs only the visual network (SAM-ViT-B + CLIP-L + projector) and a learned Quantizer: a 1280->9 down projection, then FSQ with levels [4]*9 (262144 codes), via model.visual_features(quantize=True). It emits [layout] + row codes, 857 codes x 18 bits = 15426 bits, which is within budget. No OCR, language-model or text step runs at encode time. The quantizer parameters (quant.down/up, fsq strides) were trained in src/model/train.py stages A and B and appear in the weights/delta.safetensors inventory. The codebook size in architecture.json (262144) matches the FSQ levels.
✓decoder generates text from codes onlyThe decode path in server.py rebuilds the sequence with vqocr.sequence_from_codes from the code list alone. It embeds co
The decode path in server.py rebuilds the sequence with vqocr.sequence_from_codes from the code list alone. It embeds codes through the FSQ grid and the up projection (9->1280), adds BOS and the fixed prompt, and decodes greedily with the LoRA-merged DeepSeek-V2 LM (lora.py merge_lora). The only cache is dec_cache, keyed by the codes tuple, so it is a deterministic memo of codes. Decode reads no image, side file, text store or external service.
✓served weights are the trained modelserve.sh points DELTA_PATH at ./weights/delta.safetensors. server.py builds the base from weights/model-00001-of-000001.
serve.sh points DELTA_PATH at ./weights/delta.safetensors. server.py builds the base from weights/model-00001-of-000001.safetensors, applies rank-16 LoRA, loads the delta and merges it. The delta contains frontend, LoRA A/B, quant and newline/sep tensors, matching what save_delta in src/model/train.py writes. The trajectory shows the provenance chain: the base was copied from /artifacts/deepseek-ocr/weights (trajectory#198); the stage-B step-1000 snapshot was preserved as delta_stageB_s1000.safetensors at 958394420 bytes (trajectory#232); it was later staged as delta.safetensors with matching md5 and size (trajectory#288). This size matches the inventory, and logs/train_stageB.log records the step-1000 snapshot. architecture.json declares this snapshot as the shipped delta.
✓harness credentials used only by the harnessSearches of the trajectory for harness API keys, tokens and base URLs found no participant use of harness credentials. T
Searches of the trajectory for harness API keys, tokens and base URLs found no participant use of harness credentials. The participant used only the Spin compute CLI and ssh/scp to its own training box (e.g. trajectory#232, trajectory#288). Neither server.py nor serve.sh reads harness credential environment variables.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed11h 12m into the runtook 12m 2s
startup 46.0 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 46.0 s
- ✓encode throughput · 400 requests at concurrency 16: 46.9 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 262144, "codes": [0, 136482, 149862, 149862, 149862, 149858, 166246, 166182, 166246, 166246, 170342, 170342, 170342, 166230, 170342, 170342, 170326, 170342, 169318, 166230, 182610, 170342, 166226, 182610, 166246, 166230, 166230, 166230, 166230, 166230, 182610, 166230 ...[clipped]
- ✓bit budget · 857 codes x 18 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 49.5 s
- ✓decode throughput · 420 requests at concurrency 16: 405.1 s of a 420 s deadline, timed from a fresh instance's readiness as in grading; 2 code lists decoded to different texts when sent more than once (decoding must be deterministic; grading keeps a repeated code list's lower score)
- ✓POST /decode · "A Survey on Deep Learning for Image Classification\nYara Duarte, Obi Marchetti, Obi Marchetti, Obi Marchetti, Obi Marchetti, Obi Marchetti\n2026-12-14\nAbstract\nThe 2006\u201307 season was the 100th season of the club's existence. The club was relegated to the 2nd tier of the Scotti ...[clipped]
- ✓healthy after requests
- 2failed11h 45m into the runtook 12m 20s
startup 53.6 s of 300 s; 1/2 gates passed; failed decode throughput: Batch exceeded 420s deadline; score is zero (417 of 420 requests answered in 420.0 s at concurrency 16); The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server
9 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 53.6 s
- ✓encode throughput · 400 requests at concurrency 16: 56.0 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 262144, "codes": [0, 136482, 149862, 149862, 149862, 149858, 166246, 166182, 166246, 166246, 170342, 170342, 170342, 166230, 170342, 170342, 170326, 170342, 169318, 166230, 182610, 170342, 166226, 182610, 166246, 166230, 166230, 166230, 166230, 166230, 182610, 166230 ...[clipped]
- ✓bit budget · 857 codes x 18 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 51.5 s
- ✗decode throughput · Batch exceeded 420s deadline; score is zero (417 of 420 requests answered in 420.0 s at concurrency 16)
The agent's last message before its session ended.
Selfcheck #2 is in flight. Polling in the foreground so the turn always ends on a tool call: