OCR latent compression
- Score
- 0.0
- Wallet spend
- $9.86 of $10
- Model cost
- $2.69
- Agent time
- 5h 31m
- Steps
- 262
- Tokens in / out
- 25.5M / 91k
- Self-checks
- 0 passed of 4 used
- Rank at $10 wallet
- 17th of 19
22 of 420 requests failed, timed out or returned malformed responses, more than 5%; the server is not serving reliably. First errors: ["image 37: failed 3 attempts: RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500:
Score
0.0
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesThe /encode route in server.py calls model.encode_image in model.py. That function runs the DeepSeek-OCR SAM and CLIP vi
The /encode route in server.py calls model.encode_image in model.py. That function runs the DeepSeek-OCR SAM and CLIP vision encoders, then the 2048-to-1280 projector, then quantizer.encode. The quantizer is the FSQ module in fsq.py: 8 dimensions with 4 levels each, giving 65536 codes. It has learned pre and post Linear layers and a learned gain. Those tensors are present in weights/model.safetensors as model.projector.q.pre.weight [8,1280], post.weight [1280,8], gain and half_levels. The encode path does no OCR text generation, transcription or markup extraction. The codes are discrete integers taken from neural visual features.
✓decoder generates text from codes onlyThe /decode route in server.py takes only req.codes and codebook_size and calls decode_codes in model.py. That function
The /decode route in server.py takes only req.codes and codebook_size and calls decode_codes in model.py. That function maps the codes back to features with quantizer.decode (the FSQ post Linear) and places them at the IMAGE_TOKEN positions in the prompt embeddings. A fixed 'Free OCR' prompt is used and no per-image input other than the codes. The trained DeepseekV2 language model then generates the text with model.generate(do_sample=False), which is deterministic greedy decoding. The server keeps no state between requests and uses no cache, file lookup or retrieval of per-image outputs. The only cache is the standard transformer KV cache within a single generation.
✓served weights are the trained modelload_all in model.py builds DeepseekOCRForCausalLM with the FSQ-wrapped projector and loads weights/model.safetensors, t
load_all in model.py builds DeepseekOCRForCausalLM with the FSQ-wrapped projector and loads weights/model.safetensors, the file serve.sh/server.py serves. The trajectory traces that file to training: - train.py ran on synthetic data from data_gen.py (trajectory#66, #72), starting from the declared base deepseek-ai/DeepSeek-OCR pinned to a revision in architecture.json. - The trained file was saved (trajectory#73) and copied into output/weights via spin storage (trajectory#87-#89). - train_cpu2 then continued training from that file and wrote back to it (trajectory#221, #223, #224). Either way the served file holds trained weights, and its projector.q.* tensors are FSQ parameters that do not exist in the base model. The codebook_size of 65536 in architecture.json matches FSQ([4]*8). ckpt_cpu.pt and fsq_backup.py are leftovers that serving never loads, not alternate models.
✓harness credentials used only by the harnessIn the trajectory, HF_TOKEN was used only to download the declared base model (trajectory#29-#31, #201), which is allowe
In the trajectory, HF_TOKEN was used only to download the declared base model (trajectory#29-#31, #201), which is allowed. Spin R2 transfer credentials moved the trained weights (trajectory#74-#76, #87, #88), which is also allowed. The participant used only the provided frontier selfcheck and submit tools (trajectory#226-#261). No harness API key was read, copied or used by participant code, no harness CLI was launched, and nothing was uploaded with HF_TOKEN.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed3h 51m into the runtook 8m 15s
startup 35.3 s of 300 s; 1/2 gates passed; failed decode phase: 22 of 420 requests failed, timed out or returned malformed responses, more than 5%; the server is not serving reliably. First errors: ["image 84: failed 3 attempts: RuntimeError: Submission returned HTTP 500: b'Internal Server E
9 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 35.3 s
- ✓encode throughput · 400 requests at concurrency 16: 34.9 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [10762, 8744, 43680, 10912, 10912, 10912, 10784, 10400, 8864, 10912, 10890, 10794, 10912, 10914, 10274, 10784, 10786, 10272, 43680, 10784, 35498, 41634, 10920, 43680, 10912, 10912, 10912, 10912, 10912, 43680, 10912, 10912, 8864, 10912, 10912, 10272, 1 ...[clipped]
- ✓bit budget · 903 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 68.3 s
- ✗decode phase · 22 of 420 requests failed, timed out or returned malformed responses, more than 5%; the server is not serving reliably. First errors: ["image 84: failed 3 attempts: RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error'", "image 340: failed 3 attempts: RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error'", "image 117: failed 3 attempts: RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error'", "image 302: failed 3 attempts: RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error'", "image 80: failed 3 attempts: RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error' | RuntimeError: Submission returned HTTP 500: b'Internal Server Error'"]
- 2failed4h 0m into the runtook 48m 32s
startup 45.4 s of 300 s; 1/2 gates passed; failed decode throughput: Batch exceeded 420s deadline; score is zero (0 of 420 requests answered in 420.0 s at concurrency 16); The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server th
9 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 45.4 s
- ✓encode throughput · 400 requests at concurrency 16: 52.0 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [10762, 8744, 43680, 10912, 10912, 10912, 10784, 10400, 8864, 10912, 10890, 10794, 10912, 10914, 10274, 10784, 10786, 10272, 43680, 10784, 35498, 41634, 10920, 43680, 10912, 10912, 10912, 10912, 10912, 43680, 10912, 10912, 8864, 10912, 10912, 10272, 1 ...[clipped]
- ✓bit budget · 903 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 35.2 s
- ✗decode throughput · Batch exceeded 420s deadline; score is zero (0 of 420 requests answered in 420.0 s at concurrency 16)
- 3failed4h 52m into the runtook 15m 15s
startup 41.6 s of 300 s; 1/2 gates passed; failed decode throughput: Batch exceeded 420s deadline; score is zero (0 of 420 requests answered in 420.0 s at concurrency 16); The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server th
9 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 41.6 s
- ✓encode throughput · 400 requests at concurrency 16: 42.5 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [10762, 8744, 43680, 10912, 10912, 10912, 10784, 10400, 8864, 10912, 10890, 10794, 10912, 10914, 10274, 10784, 10786, 10272, 43680, 10784, 35498, 41634, 10920, 43680, 10912, 10912, 10912, 10912, 10912, 43680, 10912, 10912, 8864, 10912, 10912, 10272, 1 ...[clipped]
- ✓bit budget · 903 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 66.2 s
- ✗decode throughput · Batch exceeded 420s deadline; score is zero (0 of 420 requests answered in 420.0 s at concurrency 16)
- 4failed5h 12m into the runtook 12m 48s
startup 44.0 s of 300 s; 1/2 gates passed; failed decode throughput: Batch exceeded 420s deadline; score is zero (80 of 420 requests answered in 420.0 s at concurrency 16); The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server t
9 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 44.0 s
- ✓encode throughput · 400 requests at concurrency 16: 45.3 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 65536, "codes": [10762, 8744, 43680, 10912, 10912, 10912, 10784, 10400, 8864, 10912, 10890, 10794, 10912, 10914, 10274, 10784, 10786, 10272, 43680, 10784, 35498, 41634, 10920, 43680, 10912, 10912, 10912, 10912, 10912, 43680, 10912, 10912, 8864, 10912, 10912, 10272, 1 ...[clipped]
- ✓bit budget · 903 codes x 16 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 43.4 s
- ✗decode throughput · Batch exceeded 420s deadline; score is zero (80 of 420 requests answered in 420.0 s at concurrency 16)
The agent's last message before its session ended.
Submitted.