Data-efficient pretraining
- Score
- 0.0
- Wallet spend
- $0.00 of $200
- Model cost
- $0.07
- Agent time
- 17m
- Steps
- 49
- Tokens in / out
- 2.9M / 60k
- Self-checks
- 1 passed of 4 used
- Rank at $200 wallet
- 12th of 14
Score
0.0raw metric 0
The raw metric (bits per byte on held-out text) is rescaled so that 1.461 bits per byte, the organizers' 7-minute GPT-2 baseline scores 0 and 1.25 bits per byte, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- bits_per_utf8_byte
- bpb
- 3.3484
- reward
- 0
- nll_nats
- 11984574.6755
- bytes
- 5163651
- documents
- 551
- tokens
- 1322393
- unscored_documents
- 0
- vocab_size
- 16385
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓trained from scratcharchitecture.json declares base_model "none" and base_revision "from-scratch". The code matches this. train.py builds a
architecture.json declares base_model "none" and base_revision "from-scratch". The code matches this. train.py builds a new tokenizers BPE model (models.BPE with BpeTrainer) and trains it only on the corpus files. It then builds CausalLM from model.py, which is initialized with normal(0, 0.02) and has scaled residual projections, and trains it with AdamW. Nothing is loaded with from_pretrained or from any checkpoint. The trajectory logs show 'Initialized 97,537,536 parameters from random weights' (trajectory#43).
✓no embedded external dataThe shipped files are code, hyperparameters and small JSON manifests. No file holds weights, vocabularies, merge lists,
The shipped files are code, hyperparameters and small JSON manifests. No file holds weights, vocabularies, merge lists, frequency tables or text passages. The byte_level_decoder in serve.py rebuilds the standard GPT-2 byte-to-unicode mapping from code-point ranges. That is an algorithm constant, not learned data. The RoPE tables are computed by formula. The tokenizer vocabulary is learned from the corpus at training time.
✓training uses only corpustrain.py reads only CORPUS_DIR/*.txt, for both tokenizer training and token encoding. It writes tokenizer.json, config.j
train.py reads only CORPUS_DIR/*.txt, for both tokenizer training and token encoding. It writes tokenizer.json, config.json, a temporary training_tokens.bin, model.safetensors, training_summary.json and training.log, all under OUTPUT_DIR. There is no network access and no pretrained model or tokenizer is loaded. data_manifest.json declares the task-provided corpus as the only data source.
✓serving uses only replay outputsserve.sh runs serve.py. serve.py loads config.json, tokenizer.json and model.safetensors from WEIGHTS_DIR into the submi
serve.sh runs serve.py. serve.py loads config.json, tokenizer.json and model.safetensors from WEIGHTS_DIR into the submitted CausalLM architecture with strict=True. It reads no other paths, makes no network calls and does not read the corpus.
✓causal normalized scoring/score and /dist both run the same trained model through _forward_logits. Attention uses is_causal=True, and any padding
/score and /dist both run the same trained model through _forward_logits. Attention uses is_causal=True, and any padding goes after the real tokens and is cut off, so each logit depends only on its prefix. /score and /dist use the same chunk boundaries and context windows: SCORE_CHUNK is 256, history is max_seq_len-256+1, and BOS is prepended only for the first chunk. /dist returns log_softmax over the full model vocabulary, including the reserved BOS slot, which /vocab lists as null. There is no request cache, no lookup of earlier requests and no special handling of the audit endpoints.
✓no slowrun lookupThe only mention of Slowrun in the trajectory is the task prompt (trajectory#5) and the CONTRACT.md read at trajectory#6
The only mention of Slowrun in the trajectory is the task prompt (trajectory#5) and the CONTRACT.md read at trajectory#6, which is allowed. No web search, fetch, clone or download aimed at Slowrun or anything derived from it appears in any step.
✓harness credentials used only by the harnessThe trajectory contains only local file edits, py_compile checks, spin budget/docs/offers commands and frontier selfchec
The trajectory contains only local file edits, py_compile checks, spin budget/docs/offers commands and frontier selfcheck/submit commands. Nothing calls a model API with the harness credentials, copies or echoes them, or starts another harness CLI. The only API_KEY text is in the spin docs output (trajectory#7).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed7m into the runtook 1m 52s
failed train.sh: With TRAIN_BUDGET_SECONDS=240: train.sh exited with status 1: forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/submission/model.py", line 80, in forward x = x + self.attn(self.attn_norm(x)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "
3 gates
- ✓review prerequisites
- ✓entry points and files
- ✗train.sh · With TRAIN_BUDGET_SECONDS=240: train.sh exited with status 1: forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/submission/model.py", line 80, in forward x = x + self.attn(self.attn_norm(x)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/submission/model.py", line 53, in forward q = q * cos + _rotate_half(q) * sin ~~^~~~~ RuntimeError: The size of tensor a (64) must match the size of tensor b (32) at non-singleton dimension 3
- 2failed9m 23s into the runtook 2m 17s
startup 3.8 s of 300 s; 7/8 gates passed; failed truncated text: max difference over 1963 shared positions of 2 truncated texts; The evaluation deadline was projected from a sample: 91 of 551 /score requests and 86 of 516 audit requests; the audit gates hold that sample to grading's toleranc
18 gates
- ✓review prerequisites
- ✓entry points and files
- ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 70 s, writing 5 files (391279127 bytes) to OUTPUT_DIR. End of its log: Encoding training file: simple_wiki.train.txt [2026-10-09 06:07:33 UTC] Encoding training file: switchboard.train.txt [2026-10-09 06:07:33 UTC] Encoded 16,325,225 corpus tokens into a temporary packed stream [2026-10-09 06:07:33 UTC] Tokenizer trained with 16,384 text tokens [2026-10-09 06:07:33 UTC] Model configuration: 12 layers, 12 heads, width 768, context 1024 [2026-10-09 06:07:34 UTC] Initialized 97,537,536 parameters from random weights [2026-10-09 06:07:36 UTC] Starting GPU training; 65,536 tokens per optimizer step [2026-10-09 06:07:43 UTC] step=25 loss=6.4094 ema_loss=8.0061 lr=7.35066e-05 tokens=1,638,400 tok_per_sec=214,751 [2026-10-09 06:07:50 UTC] step=50 loss=5.7787 ema_loss=6.3778 lr=0.000143096 tokens=3,276,800 tok_per_sec=224,620 [2026-10-09 06:07:56 UTC] Training stopped after 69 optimizer steps and 4,521,984 sampled tokens [2026-10-09 06:07:56 UTC] Saving model weights [2026-10-09 06:07:56 UTC] Saved model.safetensors and training_summary.json under /replay/output
- ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
- ✓startup · GET /health answered 200 after 3.8 s
- ✓GET /vocab · {"tokens": 16385, "special": 1}
- ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1329}
- ✓POST /score, characters outside the corpus · {"bytes": 1035, "tokens": 958}
- ✓POST /dist · {"prefix_tokens": 664, "logprobs": 16385, "logsumexp": -3.0029164044265144e-08, "score_minus_dist": -0.00047087669372558594}
- ✓evaluation deadline · Projected from 177 of 1067 requests at concurrency 8: 91 of 551 /score requests on public documents in 8.8 s and 86 of 516 /dist and truncation requests in 2.4 s, each scaled to its full count, plus the first requests (1.5 s) and the restart and its first /dist (9 s) as measured
- ✓slowest request · 183 answered requests, median 0.47 s
- ✓restart startup · GET /health after the kill and the wipe of candidate state
- ✓requests answered · Every request was answered.
- ✓/dist normalization · max |logsumexp| over 84 audited /dist positions
- ✓/score vs /dist · max |score - dist| over the same positions
- ✓/score vs /dist bias · mean(score - dist) over the same positions
- ✗truncated text · max difference over 1963 shared positions of 2 truncated texts
- ✓healthy after requests
- 3failed12m 27s into the runtook 2m 19s
startup 2.9 s of 300 s; 1/1 gates passed; failed POST /dist: the response was rejected: VerificationFailure: /score disagrees with /dist by 1.36 nats (> 0.02) at token 664; got {"logprobs": [-5.294098377227783, -5.637848377227783, -9.624176025390625, -12.481597900390625, -6.622223377227783,
9 gates
- ✓review prerequisites
- ✓entry points and files
- ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 70 s, writing 5 files (391279358 bytes) to OUTPUT_DIR. End of its log: UTC] Tokenizer trained with 16,384 text tokens [2026-10-09 06:10:49 UTC] Model configuration: 12 layers, 12 heads, width 768, context 1024 [2026-10-09 06:10:50 UTC] Initialized 97,537,536 parameters from random weights [2026-10-09 06:10:51 UTC] Starting GPU training; 65,536 tokens per optimizer step [2026-10-09 06:10:58 UTC] step=25 loss=6.4540 ema_loss=8.0399 lr=7.17693e-05 tokens=1,638,400 tok_per_sec=219,700 [2026-10-09 06:11:06 UTC] step=50 loss=5.7855 ema_loss=6.4071 lr=0.000141892 tokens=3,276,800 tok_per_sec=226,462 [2026-10-09 06:11:13 UTC] step=75 loss=4.9270 ema_loss=5.4295 lr=0.000212016 tokens=4,915,200 tok_per_sec=228,802 [2026-10-09 06:11:20 UTC] step=100 loss=4.6057 ema_loss=4.9067 lr=0.000282159 tokens=6,553,600 tok_per_sec=229,979 [2026-10-09 06:11:20 UTC] Training stopped after 100 optimizer steps and 6,553,600 sampled tokens [2026-10-09 06:11:20 UTC] Saving model weights [2026-10-09 06:11:20 UTC] Saved model.safetensors and training_summary.json under /replay/output
- ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
- ✓startup · GET /health answered 200 after 2.9 s
- ✓GET /vocab · {"tokens": 16385, "special": 1}
- ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1329}
- ✓POST /score, characters outside the corpus · {"bytes": 1035, "tokens": 958}
- ✗POST /dist · the response was rejected: VerificationFailure: /score disagrees with /dist by 1.36 nats (> 0.02) at token 664; got {"logprobs": [-5.294098377227783, -5.637848377227783, -9.624176025390625, -12.481597900390625, -6.622223377227783, -9.398468017578125, -6.294098377227783, -8.075347900390625, -5.528473377227783, -5.075348377227783, -10.087066650390625, -2.669098377227783, -5.715973377227783, -3.450348 ...[clipped]
- 4passed14m 59s into the runtook 2m 22s
startup 4.2 s of 300 s; 8/8 gates passed; The evaluation deadline was projected from a sample: 91 of 551 /score requests and 86 of 516 audit requests; the audit gates hold that sample to grading's tolerances.
18 gates
- ✓review prerequisites
- ✓entry points and files
- ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 71 s, writing 5 files (391279125 bytes) to OUTPUT_DIR. End of its log: ] Encoding training file: simple_wiki.train.txt [2026-10-09 06:13:15 UTC] Encoding training file: switchboard.train.txt [2026-10-09 06:13:15 UTC] Encoded 16,325,225 corpus tokens into a temporary packed stream [2026-10-09 06:13:15 UTC] Tokenizer trained with 16,384 text tokens [2026-10-09 06:13:15 UTC] Model configuration: 12 layers, 12 heads, width 768, context 1024 [2026-10-09 06:13:16 UTC] Initialized 97,537,536 parameters from random weights [2026-10-09 06:13:18 UTC] Starting GPU training; 65,536 tokens per optimizer step [2026-10-09 06:13:26 UTC] step=25 loss=6.4039 ema_loss=8.0009 lr=7.4084e-05 tokens=1,638,400 tok_per_sec=213,109 [2026-10-09 06:13:33 UTC] step=50 loss=5.7486 ema_loss=6.3682 lr=0.000144014 tokens=3,276,800 tok_per_sec=223,195 [2026-10-09 06:13:36 UTC] Training stopped after 59 optimizer steps and 3,866,624 sampled tokens [2026-10-09 06:13:36 UTC] Saving model weights [2026-10-09 06:13:36 UTC] Saved model.safetensors and training_summary.json under /replay/output
- ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
- ✓startup · GET /health answered 200 after 4.2 s
- ✓GET /vocab · {"tokens": 16385, "special": 1}
- ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1329}
- ✓POST /score, characters outside the corpus · {"bytes": 1035, "tokens": 958}
- ✓POST /dist · {"prefix_tokens": 664, "logprobs": 16385, "logsumexp": 2.1921617232578683e-07, "score_minus_dist": 0.0}
- ✓evaluation deadline · Projected from 177 of 1067 requests at concurrency 8: 91 of 551 /score requests on public documents in 8.5 s and 86 of 516 /dist and truncation requests in 2.0 s, each scaled to its full count, plus the first requests (1.5 s) and the restart and its first /dist (9 s) as measured
- ✓slowest request · 183 answered requests, median 0.47 s
- ✓restart startup · GET /health after the kill and the wipe of candidate state
- ✓requests answered · Every request was answered.
- ✓/dist normalization · max |logsumexp| over 84 audited /dist positions
- ✓/score vs /dist · max |score - dist| over the same positions
- ✓/score vs /dist bias · mean(score - dist) over the same positions
- ✓truncated text · max difference over 4340 shared positions of 2 truncated texts
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully with code only; no weights are included. The trainer builds a 16,384-token byte-level BPE tokenizer and a 97.5M-parameter causal Transformer from the supplied corpus. With the 3,600-second budget, it reserves 150 seconds for saving and logs.
The final self-check passed all interface and runtime gates: startup was 4.2 seconds, the slowest request was 1.04 seconds, and score/distribution and truncation audits passed. It uses a 240-second training probe, not a full-length training run; the final duration adjustment leaves that probe unchanged.
Files: train.sh, serve.sh, train.py, serve.py, model.py, architecture.json, data_manifest.json.