OCR latent compression
- Score
- 2.1
- Wallet spend
- $8.63 of $10
- Model cost
- $1.37
- Agent time
- 2h 38m
- Steps
- 708
- Tokens in / out
- 100.3M / 454k
- Self-checks
- 3 passed of 4 used
- Rank at $10 wallet
- 13th of 14
Score
2.1raw metric 0.01896
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 32768
- diagnostic_word_f1
- 0.0462
- diagnostic_char_similarity
- 0.1413
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesOn /encode, server.py runs visual_encoder.extract_visual_features_batch. This passes the image's pixel views (a 1024 glo
On /encode, server.py runs visual_encoder.extract_visual_features_batch. This passes the image's pixel views (a 1024 global view plus 640 tiles) through the frozen DeepSeek-OCR SAM, CLIP and projector stack to get 1280-dim features. The trained FeatureCodec in codec.py then compresses them: Linear-GELU-Linear-tanh down to 8 latents, FSQ with levels (4,4,4,4,4,4,4,2) and a straight-through estimator, packed mixed-radix into codes 0..32767 (15 bits). There is no OCR or text-token path anywhere on the encode side. The codec.pt inventory matches this module: encoder.0 [512,1280], encoder.2 [8,512], decoder tensors, and mean/scale [1280].
✓decoder generates text from codes onlyOn /decode, server.py checks codebook_size, length and code range. It then reconstructs visual embeddings through CODEC.
On /decode, server.py checks codebook_size, length and code range. It then reconstructs visual embeddings through CODEC.decode, which unpacks the codes and runs the decoder MLP and denormalization, and appends the base model's fixed view_seperator. model_loader.py's precomputed-visual override places these vectors at the image-token positions of a fixed prompt, and the frozen DeepSeek LM generates greedily (do_sample=False). The override is cleared in a finally block after every batch. Batch padding only repeats codes from the same request batch. No per-image cache, side channel or text store is consulted, so the output text depends only on the submitted codes plus fixed model weights.
✓served weights are the trained modelserver.py loads model/ (the declared base deepseek-ai/DeepSeek-OCR at the pinned revision) and codec.pt into the Feature
server.py loads model/ (the declared base deepseek-ai/DeepSeek-OCR at the pinned revision) and codec.pt into the FeatureCodec architecture with weights_only. The trajectory shows the base safetensors copied unmodified from the host artifacts into output/model in the early setup steps. It also shows the codec.pt from the final stage-3 train.py run (resumed from stage 2, matching training_config.json and training.log, which records the 5298789-byte save) being uploaded and copied into the output (trajectory#629, trajectory#630), with logs appended at trajectory#632. The codec.pt inventory is complete, is 5298789 bytes, and its shapes match codec.py. model_loader.py's SDPA swap loads the same state_dict with strict=True, and its batched MoE dispatch reuses the same expert weights.
✓harness credentials used only by the harnessThe trajectory contains no use of the harness API key or base URL by the participant. The 'codex exec' matches are only
The trajectory contains no use of the harness API key or base URL by the participant. The 'codex exec' matches are only ps listings of the harness process (trajectory#523). Artifact transfers used the spin s5cmd/R2 transfer tooling rather than harness model credentials (trajectory#629, trajectory#630). Neither the shipped server nor training code references harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed1h 37m into the runtook 7m 12s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: You are using a model of type deepseek_vl_v2 to instantiate a model of type DeepseekOCR. This is n
3 gates
- ✓review prerequisites
- ✓architecture.json
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: You are using a model of type deepseek_vl_v2 to instantiate a model of type DeepseekOCR. This is not supported for all configurations of models and can yield errors. Traceback (most recent call last): File "/submission/server.py", line 40, in <module> MODEL, TOKENIZER, CODEC = load_runtime() ^^^^^^^^^^^^^^ File "/submission/server.py", line 31, in load_runtime model = load_ocr_model(MODEL_DIR) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/submission/model_loader.py", line 77, in load_ocr_model model = AutoModel.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained return model_class.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 3577, in from_pretrained raise ImportError( ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install 'accelerate>=0.26.0'`
- 2passed1h 44m into the runtook 8m 31s
startup 10.8 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 10.8 s
- ✓encode throughput · 400 requests at concurrency 16: 59.2 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 32768, "codes": [13248, 9993, 5966, 5965, 1870, 1870, 2894, 1870, 18255, 974, 13248, 15240, 1870, 1869, 1870, 1871, 2894, 1870, 1862, 1865, 29644, 8990, 9038, 18254, 846, 1870, 1870, 846, 1870, 1935, 17294, 798, 1806, 1869, 1806, 1806, 1806, 1870, 1806, 1802, 797, 29 ...[clipped]
- ✓bit budget · 902 codes x 15 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 11.2 s
- ✓decode throughput · 420 requests at concurrency 16: 216.6 s of a 420 s deadline, timed from a fresh instance's readiness as in grading; 3 code lists decoded to different texts when sent more than once (decoding must be deterministic; grading keeps a repeated code list's lower score)
- ✓POST /decode · "Discussion\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nWorkspace\nworkspace\nworkspace\nworkspace\nworkspace\nworkspace\nworkspace\nworkspace\nworkspace\nworkspace\nworkspace\nworkspace\nworkspace"
- ✓healthy after requests
- 3passed2h 9m into the runtook 3m 54s
startup 10.9 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 10.9 s
- ✓encode throughput · 400 requests at concurrency 16: 61.0 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 32768, "codes": [13248, 9993, 5966, 5965, 1870, 1870, 2894, 1870, 18255, 974, 13248, 15240, 1870, 1869, 1870, 1871, 2894, 1870, 1862, 1865, 29644, 8990, 9038, 18254, 846, 1870, 1870, 846, 1870, 1935, 17294, 798, 1806, 1869, 1806, 1806, 1806, 1870, 1806, 1802, 797, 29 ...[clipped]
- ✓bit budget · 902 codes x 15 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 9.6 s
- ✓decode throughput · 420 requests at concurrency 16: 4.5 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · ""
- ✓healthy after requests
- 4passed2h 31m into the runtook 9m 52s
startup 10.7 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 10.7 s
- ✓encode throughput · 400 requests at concurrency 16: 60.9 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 32768, "codes": [13248, 14094, 1871, 4942, 1870, 1870, 1870, 1870, 1871, 911, 13248, 15237, 1870, 1870, 1870, 1871, 1870, 1871, 1798, 1866, 29644, 9070, 8975, 19279, 846, 1870, 1933, 846, 1870, 1871, 911, 4879, 1806, 2894, 1870, 1806, 1870, 1871, 1870, 1802, 778, 296 ...[clipped]
- ✓bit budget · 902 codes x 15 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 12.2 s
- ✓decode throughput · 420 requests at concurrency 16: 342.7 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Summary :\nPapers :\nSenes of Justice : Dr. Justice S. Nalwa , a local\nRamu Joy , a local\nSekhri Zangoora : a local\nSanghar : a local\nSanghar : a local\nSanghar : a local\nSanghar : a local\nSanghar : a local\nSangbar : a local\nSangbar : a local\nSangbar : a local\nSangbar : a l ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/. The final selfcheck passed startup, code-budget, and throughput gates: startup 10.7–12.2 s, encoding 6.57 images/s, decoding 1.23 images/s, and 13,530 bits per page. It did not run the separate architecture review.
Training used only synthetic pages rendered from the WikiText-2 raw v1 train split. A pre-V5 diagnostic showed low transcription similarity; the final selfcheck does not measure accuracy, so final transcription quality remains unverified.