OCR latent compression
- Score
- 35.2
- Wallet spend
- $125.64 of $200
- Model cost
- $10.28
- Agent time
- 8h 2m
- Steps
- 352
- Tokens in / out
- 39.5M / 330k
- Self-checks
- 2 passed of 3 used
- Rank at $200 wallet
- 5th of 14
Score
35.2raw metric 0.3145
The raw metric (word-level edit similarity of the decoded text) is rescaled so that empty text scores 0 and unmodified DeepSeek-OCR reading the page image (0.893) scores 100, clamped to that range.
Verifier details
- metric
- page_word_similarity
- samples
- 400
- bit_budget
- 16384
- codebook_size
- 256
- diagnostic_word_f1
- 0.4030
- diagnostic_char_similarity
- 0.4763
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned discrete visual codesIn server.py, /encode only runs the image through the visual network and the codec, and returns integer codes. The visua
In server.py, /encode only runs the image through the visual network and the codec, and returns integer codes. The visual network is the DeepSeek-OCR SAM+CLIP encoder plus a trained projector (visual_encoder.py, visual_adapter.safetensors). The codec is a learned product vector quantizer (codec.py): it projects the features, picks the nearest entry in each EMA-trained 256x16 codebook, and outputs those indices plus one geometry byte. The output is codebook_size 256 with 2017 or 1921 codes, under the 16384-bit budget. No OCR, transcription or language-model call happens in encode. architecture.json declares transcription_in_encoder false.
✓decoder generates text from codes onlyIn server.py, /decode checks that the code count and values are valid, rebuilds features using only the codebook embeddi
In server.py, /decode checks that the code count and values are valid, rebuilds features using only the codebook embeddings (codec.reconstruct plus fixed newline and separator tensors from the base model), and passes them to the DiscreteOCRForCausalLM vLLM model (discrete_decoder.py). Decoding is greedy (temperature 0, seed 0) with deterministic kernels (deterministic_kernels.py). max_tokens comes from a length estimator that also reads only the codes (length_estimator.py). The RepetitionBlocker only bans tokens based on the text generated so far. I found no per-image cache, retrieval or side channel; the request counter only labels requests. architecture.json declares per_image_state_or_retrieval false.
✓served weights are the trained modelserve.sh runs server.py, which loads codec.safetensors, visual_adapter.safetensors, length_estimator.safetensors and the
serve.sh runs server.py, which loads codec.safetensors, visual_adapter.safetensors, length_estimator.safetensors and the base model. The base model's config names DiscreteOCRForCausalLM, whose load_weights swaps in the language_adapter.safetensors tensors. The tensor names and shapes in the weight inventories match the modules that train_joint.py trains and saves (codec codebooks [9,256,16], projector, CLIP layers 16-23, merged LoRA attention and expert weights, and the length-estimator patch/readout weights). language_adapter.py writes the merged weights in this format. The starting checkpoint is declared with a pinned revision and file hashes in logs/starting_artifact.json. The trajectory shows trained snapshots copied to the served file names (for example step 294, which copies the stage-4 language adapter), and logs/selection_report.json records hashes of the selected stage-5 artifacts. I found no substituted or untrained weights.
✓harness credentials used only by the harnessI searched the trajectory for harness credentials and found no use of them by the participant. The session only used the
I searched the trajectory for harness credentials and found no use of them by the participant. The session only used the provided spin CLI (compute exec and budget status), localhost health and self-check probes, and public downloads. I saw no token or secret being read out, exported or used outside the harness.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1failed30m 41s into the runtook 9m 37s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: ning: on_event is deprecated, use lifespan event handlers instead. Read more abo
3 gates
- ✓review prerequisites
- ✓architecture.json
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: ning: on_event is deprecated, use lifespan event handlers instead. Read more about it in the [FastAPI docs for Lifespan Events](https://fastapi.tiangolo.com/advanced/events/). @app.on_event("startup") INFO: Started server process [186] INFO: Waiting for application startup. ERROR: Traceback (most recent call last): File "/tmp/visual_codec_packages/starlette/routing.py", line 692, in lifespan async with self.lifespan_context(app) as maybe_state: File "/tmp/visual_codec_packages/starlette/routing.py", line 569, in __aenter__ await self._router.startup() File "/tmp/visual_codec_packages/starlette/routing.py", line 669, in startup await handler() File "/submission/server.py", line 114, in initialize runtime = Runtime() ^^^^^^^^^ File "/submission/server.py", line 44, in __init__ self.llm = LLM(model=str(ROOT / "base"), trust_remote_code=True, dtype="bfloat16", tensor_parallel_size=1, gpu_memory_utilization=0.62, max_model_len=4608, max_num_seqs=16, max_num_batched_tokens=8192, enforce_eager=False, enable_prefix_caching=False, disable_custom_all_reduce=True, limit_mm_per_prompt={"image": 1}, mm_processor_cache_gb=0, seed=0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/visual_codec_packages/vllm/utils.py", line 1161, in inner return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/tmp/visual_codec_packages/vllm/entrypoints/llm.py", line 217, in __init__ eng ...[clipped]
- 2passed3h 31m into the runtook 9m 57s
startup 109.4 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 109.4 s
- ✓encode throughput · 400 requests at concurrency 16: 13.1 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [0, 126, 126, 31, 126, 158, 126, 101, 137, 126, 214, 126, 101, 158, 176, 62, 101, 214, 101, 214, 125, 104, 233, 176, 62, 31, 66, 178, 212, 158, 104, 158, 158, 104, 164, 164, 178, 212, 158, 104, 158, 186, 104, 164, 164, 158, 212, 31, 248, 233, 158, 248, ...[clipped]
- ✓bit budget · 2017 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 135.1 s
- ✓decode throughput · 420 requests at concurrency 16: 115.2 s of a 420 s deadline, timed from a fresh instance's readiness as in grading; 20 code lists decoded to different texts when sent more than once (decoding must be deterministic; grading keeps a repeated code list's lower score)
- ✓POST /decode · "Operations\n\nImplementation-draw Financial Generation Using Geometric\n\n6. Comments on National Trail and Realistic Talking Faces\n\nMuhadi Gijni, Youlu Boranto, Alexei Karpov, Jaaco Bashhampoun, Yen-Ju\nNiu, Lisa Milling\n\n2. Introduction\n\n3. Abstract\n\nGenerating a new produc ...[clipped]
- ✓healthy after requests
- 3passed7h 52m into the runtook 10m 45s
startup 124.1 s of 300 s; 2/2 gates passed; The encode batch sends each of the 100 public images about 4 times and the decode batch their code lists, so a server that caches repeated images or code lists measures faster here than on grading's distinct held-out images.
11 gates
- ✓review prerequisites
- ✓architecture.json
- ✓startup · GET /health answered 200 after 124.1 s
- ✓encode throughput · 400 requests at concurrency 16: 13.8 s of a 400 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /encode · {"codebook_size": 256, "codes": [0, 219, 126, 104, 126, 158, 126, 101, 137, 126, 219, 214, 101, 214, 176, 62, 101, 214, 178, 212, 31, 104, 145, 176, 255, 139, 164, 178, 212, 158, 104, 158, 158, 104, 139, 164, 178, 212, 158, 104, 158, 158, 104, 139, 164, 178, 212, 31, 248, 145, 158, 10 ...[clipped]
- ✓bit budget · 2017 codes x 8 bits for one public image; every image must fit
- ✓healthy after requests
- ✓startup · GET /health answered 200 after 111.2 s
- ✓decode throughput · 420 requests at concurrency 16: 120.7 s of a 420 s deadline, timed from a fresh instance's readiness as in grading
- ✓POST /decode · "Operations\n\nImplementation-driven Federal Government Using Geometry\n\nConstructions of the Natural and Realistic Talking Faces\nMahdi Gijni, Yolmo Borom, Alexander Kharabian, Jacob Shahrampour, Yeni-Ju\nNiu, Lisa Milling\n\nJuli 2020\n\n2. Abstract\n\nGenerating a random data stre ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with trained weights, serving code, offline dependencies, and training provenance.
- Maximum code budget: 16,136 bits, below 16,384.
- Final offline selfcheck: passed.
- Startup: 124.1 seconds or faster.
- Encode throughput: 28.97 images/s.
- Decode throughput: 3.48 images/s.
- Independent-instance testing: zero mismatches across 208 repeated decodes.
validate.pycompleted successfully after checkpoint selection was frozen.