Data-efficient pretraining
- Score
- 0.0
- Wallet spend
- $0.00 of $10
- Model cost
- $0.08
- Agent time
- 19m
- Steps
- 56
- Tokens in / out
- 4.3M / 59k
- Self-checks
- 3 passed of 4 used
- Rank at $10 wallet
- 10th of 14
Score
0.0raw metric 0
The raw metric (bits per byte on held-out text) is rescaled so that 1.461 bits per byte, the organizers' 7-minute GPT-2 baseline scores 0 and 1.25 bits per byte, a target set by the task author scores 100, clamped to that range.
Verifier details
- metric
- bits_per_utf8_byte
- bpb
- 1.9095
- reward
- 0
- nll_nats
- 6834253.8815
- bytes
- 5163651
- documents
- 551
- tokens
- 1427990
- unscored_documents
- 0
- vocab_size
- 8193
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓trained from scratcharchitecture.json sets base_model to "none" and base_revision to "from-scratch", and the code matches. model.py builds C
architecture.json sets base_model to "none" and base_revision to "from-scratch", and the code matches. model.py builds CausalLM from plain torch nn modules and initializes it randomly (normal std 0.02, with residual-scaled output projections). train.py creates a fresh tokenizers BPE model and trains it with BpeTrainer on the corpus files. There is no from_pretrained call, no checkpoint loading in training and no distillation. The trajectory shows only writing the code and running self-checks, with no model downloads (trajectory#12, trajectory#46).
✓no embedded external dataThe shipped files are source code, JSON manifests and a probe log. None of them contains weights, vocabularies, merge li
The shipped files are source code, JSON manifests and a probe log. None of them contains weights, vocabularies, merge lists, word lists or text passages. The tokenizer's initial alphabet is the standard 256-symbol byte-level alphabet from pre_tokenizers.ByteLevel.alphabet(), and serve.py rebuilds the same byte mapping algorithmically. Both are format constants, not learned data. The remaining constants are hyperparameters or RoPE algorithm constants. training.log and data_manifest.json hold only run statistics and file names.
✓training uses only corpustrain.sh runs train.py, which reads only glob(CORPUS_DIR/*.train.txt) to train the tokenizer and the model. It writes to
train.sh runs train.py, which reads only glob(CORPUS_DIR/*.train.txt) to train the tokenizer and the model. It writes tokenizer.json, data_manifest.json, config.json, model.pt, run_metadata.json and training.log under OUTPUT_DIR only, and makes no network calls or library-model loads. The corpus source is declared as the task-provided /task/public/corpus in architecture.json and data_manifest.json. In the session, the participant read validation.jsonl only to count bytes and genres (trajectory#9, trajectory#10). It was never used in training code.
✓serving uses only replay outputsserve.sh runs serve.py. serve.py loads config.json, the tokenizer file and model.pt only from WEIGHTS_DIR, with strict s
serve.sh runs serve.py. serve.py loads config.json, the tokenizer file and model.pt only from WEIGHTS_DIR, with strict state-dict loading into the submitted CausalLM. It reads no corpus, external model, cache or network resource.
✓causal normalized scoring/score and /dist both call the same predict() forward pass of the trained causal transformer, which uses SDPA with is_ca
/score and /dist both call the same predict() forward pass of the trained causal transformer, which uses SDPA with is_causal=True. /score feeds [BOS]+tokens[:-1] and takes gathered logits minus logsumexp over the full vocabulary. /dist feeds [BOS]+prefix and returns log_softmax over the full vocab_size output. There is no request-history caching, lookup of previously seen text or special handling of audit requests.
✓harness credentials used only by the harnessSearching the trajectory for API_KEY, BASE_URL, curl, openrouter and harness CLI launches turned up only the read-only s
Searching the trajectory for API_KEY, BASE_URL, curl, openrouter and harness CLI launches turned up only the read-only spin documentation output (trajectory#8). The participant made no model API calls with harness credentials, did not echo or copy credentials and started no extra harness sessions. Commands were limited to spin help/offers, file editing, frontier selfcheck and submit (trajectory#20, trajectory#48, trajectory#55).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed6m 36s into the runtook 3m 29s
startup 4.0 s of 300 s; 6/8 gates passed; failed /score vs /dist: max |score - dist| over the same positions; The evaluation deadline was projected from a sample: 91 of 551 /score requests and 86 of 516 audit requests; the audit gates hold that sample to grading's tolerances.
17 gates
- ✓review prerequisites
- ✓entry points and files
- ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 90 s, writing 6 files (83962024 bytes) to OUTPUT_DIR. End of its log: urces=["bnc_spoken.train.txt", "childes.train.txt", "gutenberg.train.txt", "open_subtitles.train.txt", "simple_wiki.train.txt", "switchboard.train.txt"] download command: none; task-provided read-only corpus only tokenizer trained: 8,192 byte-BPE tokens in 4.5s encoded bnc_spoken.train.txt: 1,098,118 tokens in 2.8s encoded childes.train.txt: 5,873,726 tokens in 16.0s encoded gutenberg.train.txt: 3,736,156 tokens in 8.6s encoded open_subtitles.train.txt: 3,892,242 tokens in 9.1s encoded simple_wiki.train.txt: 2,595,791 tokens in 6.0s encoded switchboard.train.txt: 42,410 tokens in 0.1s corpus token count including file boundaries: 17,238,449 tokenization and data preparation: 48.1s initialized model with 20,848,896 parameters training until 2026-10-02 10:22:05; batch=16 x 1024; time budget includes tokenizer and save step=574 elapsed=20.0s loss=5.6032 lr=0.000133209 tok_s=469877 checkpoint saved: step=1037, bytes=83,417,945 finished: steps=1037, sampled_tokens=16,990,208, elapsed=86.2s
- ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
- ✓startup · GET /health answered 200 after 4.0 s
- ✓GET /vocab · {"tokens": 8193, "special": 1}
- ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1436}
- ✓POST /dist · {"prefix_tokens": 718, "logprobs": 8193, "logsumexp": -1.1695195367700251e-07, "score_minus_dist": -0.00872802734375}
- ✓evaluation deadline · Projected from 177 of 1067 requests at concurrency 8: 91 of 551 /score requests on public documents in 2.3 s and 86 of 516 /dist and truncation requests in 2.2 s, each scaled to its full count, plus the first requests (2.0 s) and the restart and its first /dist (9 s) as measured
- ✓slowest request · 182 answered requests, median 0.17 s
- ✓restart startup · GET /health after the kill and the wipe of candidate state
- ✓requests answered · Every request was answered.
- ✓/dist normalization · max |logsumexp| over 84 audited /dist positions
- ✗/score vs /dist · max |score - dist| over the same positions
- ✓/score vs /dist bias · mean(score - dist) over the same positions
- ✗truncated text · max difference over 642 shared positions of 2 truncated texts
- ✓healthy after requests
- 2passed10m 38s into the runtook 2m 17s
startup 2.3 s of 300 s; 8/8 gates passed; The evaluation deadline was projected from a sample: 91 of 551 /score requests and 86 of 516 audit requests; the audit gates hold that sample to grading's tolerances.
17 gates
- ✓review prerequisites
- ✓entry points and files
- ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 91 s, writing 6 files (83962022 bytes) to OUTPUT_DIR. End of its log: sources=["bnc_spoken.train.txt", "childes.train.txt", "gutenberg.train.txt", "open_subtitles.train.txt", "simple_wiki.train.txt", "switchboard.train.txt"] download command: none; task-provided read-only corpus only tokenizer trained: 8,192 byte-BPE tokens in 4.0s encoded bnc_spoken.train.txt: 1,098,118 tokens in 2.6s encoded childes.train.txt: 5,873,726 tokens in 12.7s encoded gutenberg.train.txt: 3,736,156 tokens in 9.1s encoded open_subtitles.train.txt: 3,892,242 tokens in 8.4s encoded simple_wiki.train.txt: 2,595,791 tokens in 5.3s encoded switchboard.train.txt: 42,410 tokens in 0.1s corpus token count including file boundaries: 17,238,449 tokenization and data preparation: 43.0s initialized model with 20,848,896 parameters training until 2026-10-02 10:24:59; batch=16 x 1024; time budget includes tokenizer and save step=595 elapsed=20.0s loss=5.5992 lr=0.000133144 tok_s=487331 checkpoint saved: step=975, bytes=83,417,945 finished: steps=975, sampled_tokens=15,974,400, elapsed=77.6s
- ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
- ✓startup · GET /health answered 200 after 2.3 s
- ✓GET /vocab · {"tokens": 8193, "special": 1}
- ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1436}
- ✓POST /dist · {"prefix_tokens": 718, "logprobs": 8193, "logsumexp": 1.1061479199270252e-07, "score_minus_dist": 0.0}
- ✓evaluation deadline · Projected from 177 of 1067 requests at concurrency 8: 91 of 551 /score requests on public documents in 1.5 s and 86 of 516 /dist and truncation requests in 1.6 s, each scaled to its full count, plus the first requests (1.3 s) and the restart and its first /dist (7 s) as measured
- ✓slowest request · 182 answered requests, median 0.11 s
- ✓restart startup · GET /health after the kill and the wipe of candidate state
- ✓requests answered · Every request was answered.
- ✓/dist normalization · max |logsumexp| over 84 audited /dist positions
- ✓/score vs /dist · max |score - dist| over the same positions
- ✓/score vs /dist bias · mean(score - dist) over the same positions
- ✓truncated text · max difference over 3095 shared positions of 2 truncated texts
- ✓healthy after requests
- 3passed14m into the runtook 2m 47s
startup 2.3 s of 300 s; 8/8 gates passed; The evaluation deadline was projected from a sample: 91 of 551 /score requests and 86 of 516 audit requests; the audit gates hold that sample to grading's tolerances.
17 gates
- ✓review prerequisites
- ✓entry points and files
- ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 90 s, writing 6 files (83962088 bytes) to OUTPUT_DIR. End of its log: rain.txt", "open_subtitles.train.txt", "simple_wiki.train.txt", "switchboard.train.txt"] download command: none; task-provided read-only corpus only tokenizer trained: 8,192 byte-BPE tokens in 3.6s encoded bnc_spoken.train.txt: 1,098,118 tokens in 2.6s encoded childes.train.txt: 5,873,726 tokens in 12.1s encoded gutenberg.train.txt: 3,736,156 tokens in 7.7s encoded open_subtitles.train.txt: 3,892,242 tokens in 7.8s encoded simple_wiki.train.txt: 2,595,791 tokens in 5.4s encoded switchboard.train.txt: 42,410 tokens in 0.1s corpus token count including file boundaries: 17,238,449 tokenization and data preparation: 40.1s initialized model with 20,848,896 parameters training until 2026-10-02 10:28:43; batch=16 x 1024; time budget includes tokenizer and save step=588 elapsed=20.0s loss=5.6100 lr=0.000133115 tok_s=481689 step=1198 elapsed=40.0s loss=4.0795 lr=0.000266642 tok_s=490344 checkpoint saved: step=1344, bytes=83,417,945 finished: steps=1344, sampled_tokens=22,020,096, elapsed=87.0s
- ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
- ✓startup · GET /health answered 200 after 2.3 s
- ✓GET /vocab · {"tokens": 8193, "special": 1}
- ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1436}
- ✓POST /dist · {"prefix_tokens": 718, "logprobs": 8193, "logsumexp": 3.867457820661002e-08, "score_minus_dist": 4.76837158203125e-07}
- ✓evaluation deadline · Projected from 177 of 1067 requests at concurrency 8: 91 of 551 /score requests on public documents in 2.2 s and 86 of 516 /dist and truncation requests in 2.5 s, each scaled to its full count, plus the first requests (1.6 s) and the restart and its first /dist (8 s) as measured
- ✓slowest request · 182 answered requests, median 0.15 s
- ✓restart startup · GET /health after the kill and the wipe of candidate state
- ✓requests answered · Every request was answered.
- ✓/dist normalization · max |logsumexp| over 84 audited /dist positions
- ✓/score vs /dist · max |score - dist| over the same positions
- ✓/score vs /dist bias · mean(score - dist) over the same positions
- ✓truncated text · max difference over 3360 shared positions of 2 truncated texts
- ✓healthy after requests
- 4passed17m 34s into the runtook 2m 18s
startup 2.8 s of 300 s; 8/8 gates passed; The evaluation deadline was projected from a sample: 91 of 551 /score requests and 86 of 516 audit requests; the audit gates hold that sample to grading's tolerances.
17 gates
- ✓review prerequisites
- ✓entry points and files
- ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 90 s, writing 6 files (83962122 bytes) to OUTPUT_DIR. End of its log: berg.train.txt", "open_subtitles.train.txt", "simple_wiki.train.txt", "switchboard.train.txt"] download command: none; task-provided read-only corpus only tokenizer trained: 8,192 byte-BPE tokens in 3.8s encoded bnc_spoken.train.txt: 1,098,118 tokens in 2.8s encoded childes.train.txt: 5,873,726 tokens in 12.8s encoded gutenberg.train.txt: 3,736,156 tokens in 7.3s encoded open_subtitles.train.txt: 3,892,242 tokens in 7.2s encoded simple_wiki.train.txt: 2,595,791 tokens in 4.7s encoded switchboard.train.txt: 42,410 tokens in 0.1s corpus token count including file boundaries: 17,238,449 tokenization and data preparation: 39.5s initialized model with 20,848,896 parameters training until 2026-10-02 10:31:54; target=500,000,000 sampled tokens; batch=16 x 1024 step=550 elapsed=20.0s loss=5.2547 lr=0.000299784 tok_s=450385 step=1123 elapsed=40.0s loss=3.8367 lr=0.000299101 tok_s=459884 checkpoint saved: step=1240, bytes=83,417,945 finished: steps=1240, sampled_tokens=20,316,160, elapsed=85.9s
- ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
- ✓startup · GET /health answered 200 after 2.8 s
- ✓GET /vocab · {"tokens": 8193, "special": 1}
- ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1436}
- ✓POST /dist · {"prefix_tokens": 718, "logprobs": 8193, "logsumexp": 4.5754382504981095e-08, "score_minus_dist": -4.76837158203125e-07}
- ✓evaluation deadline · Projected from 177 of 1067 requests at concurrency 8: 91 of 551 /score requests on public documents in 1.7 s and 86 of 516 /dist and truncation requests in 1.9 s, each scaled to its full count, plus the first requests (1.8 s) and the restart and its first /dist (8 s) as measured
- ✓slowest request · 182 answered requests, median 0.12 s
- ✓restart startup · GET /health after the kill and the wipe of candidate state
- ✓requests answered · Every request was answered.
- ✓/dist normalization · max |logsumexp| over 84 audited /dist positions
- ✓/score vs /dist · max |score - dist| over the same positions
- ✓/score vs /dist bias · mean(score - dist) over the same positions
- ✓truncated text · max difference over 6471 shared positions of 2 truncated texts
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully to frontier.
The final frontier selfcheck passed every gate. Startup was 2.8s; the slowest request was 1.58s; normalization error was 0 nats; maximum /score versus /dist difference was 0.0002 nats; and shared truncated-prefix scores matched exactly.
The submission includes the from-scratch training and serving scripts, architecture and corpus provenance manifests, and a readable training probe log. The training script targets 500M sampled tokens and stops before the supplied training deadline. The self-check notes that the separate architecture review did not run.