Data-efficient pretraining

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.

1. 1
   failed3h 36m into the runtook 4m 45s

   startup 4.8 s of 300 s; 1/1 gates passed; failed POST /score, characters outside the corpus: the response was rejected: VerificationFailure: /score tokens do not concatenate to the exact UTF-8 text. No /vocab token holds the bytes cc ('̃'), cd ('ͱ'), cf ('ς'), d0 ('Р'), d1 ('у'), d2 ('қ'), d

   8 gates

   • ✓review prerequisites
   • ✓entry points and files
   • ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 51 s, writing 16 files (474331201 bytes) to OUTPUT_DIR. End of its log: IA H100 80GB HBM3, 81559 MiB torch 2.8.0+cu128 cuda True tok 0.22.1 {"event": "start", "env": {"torch": "2.8.0+cu128", "cuda_available": true, "device": "cuda", "gpu": "NVIDIA H100 80GB HBM3", "args": {"corpus_dir": "/corpus", "output_dir": "/replay/output", "budget_seconds": 240.0, "start_time": 1791013799.0, "save_margin_seconds": 240.0, "seed": 1234, "vocab_size": 16384, "min_frequency": 2, "d_model": 768, "n_layer": 12, "n_head": 12, "n_kv_head": 12, "ffn_dim": 2048, "seq_len": 8192, "dropout": 0.0, "batch_tokens": 262144, "micro_seqs": 8, "lr": 0.001, "min_lr_ratio": 0.1, "warmup_steps": 150, "weight_decay": 0.1, "grad_clip": 1.0, "checkpoint": 0, "val_fraction": 0.0, "val_every_seconds": 600.0, "log_every": 25}}} {"event": "tokenizer", "vocab_size": 16384, "bos_id": 0, "bytes_per_token": 5.6446926692303} {"event": "data", "train_chunks": 1994, "val_chunks": 0, "bpt": 3.33} {"event": "model", "n_params": 97536768} {"event": "saving", "steps": 0} {"event": "done", "elapsed": 45.3}
   • ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
   • ✓startup · GET /health answered 200 after 4.8 s
   • ✓GET /vocab · {"tokens": 16384, "special": 1}
   • ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1329}
   • ✗POST /score, characters outside the corpus · the response was rejected: VerificationFailure: /score tokens do not concatenate to the exact UTF-8 text. No /vocab token holds the bytes cc ('̃'), cd ('ͱ'), cf ('ς'), d0 ('Р'), d1 ('у'), d2 ('қ'), d3 ('ӣ'), d4 ('Ա'), d5 ('Հ'), d6 ('ր'), d8 ('ا'), da ('ک'), db ('ی'), dc ('ܣ'), dd ('ݢ'), de ('ދ') and 15 more, so no tokenization reproduces text containing them. The token bytes must concatenate to any UTF-8 input, and held-out documents are not limited to the characters of the public data.; got {"token_ids": [181, 152, 150, 150, 119, 117, 118, 26, 147, 95, 159, 114, 157, 161, 147, 115, 114, 147, 114, 158, 166, 147, 162, 151, 117, 154, 147, 121, 164, 112, 119, 117, 154, 147, 153, 149, 109, 122, 155, 152, 116, 150, 119, 117, 154, 147, 110, 152, 120, 123, 119, 12, 147, 113, 109 ...[clipped]
2. 2
   passed3h 47m into the runtook 1m 31s

   startup 4.0 s of 300 s; 8/8 gates passed; The evaluation deadline was projected from a sample: 91 of 551 /score requests and 86 of 516 audit requests; the audit gates hold that sample to grading's tolerances.

   18 gates

   • ✓review prerequisites
   • ✓entry points and files
   • ✓train.sh · Given TRAIN_BUDGET_SECONDS=240, exited 0 on NVIDIA H100 80GB HBM3 after 43 s, writing 16 files (474329461 bytes) to OUTPUT_DIR. End of its log: 100 80GB HBM3, 81559 MiB torch 2.8.0+cu128 cuda True tok 0.22.1 {"event": "start", "env": {"torch": "2.8.0+cu128", "cuda_available": true, "device": "cuda", "gpu": "NVIDIA H100 80GB HBM3", "args": {"corpus_dir": "/corpus", "output_dir": "/replay/output", "budget_seconds": 240.0, "start_time": 1791014313.0, "save_margin_seconds": 240.0, "seed": 1234, "vocab_size": 16384, "min_frequency": 2, "d_model": 768, "n_layer": 12, "n_head": 12, "n_kv_head": 12, "ffn_dim": 2048, "seq_len": 8192, "dropout": 0.0, "batch_tokens": 262144, "micro_seqs": 8, "lr": 0.001, "min_lr_ratio": 0.1, "warmup_steps": 150, "weight_decay": 0.1, "grad_clip": 1.0, "checkpoint": 0, "val_fraction": 0.0, "val_every_seconds": 600.0, "log_every": 25}}} {"event": "tokenizer", "vocab_size": 16384, "bos_id": 0, "bytes_per_token": 5.6160043947994875} {"event": "data", "train_chunks": 1994, "val_chunks": 0, "bpt": 3.329} {"event": "model", "n_params": 97536768} {"event": "saving", "steps": 0} {"event": "done", "elapsed": 38.1}
   • ✓train.sh wall clock · Grading's hard limit, with TRAIN_BUDGET_SECONDS=3600; this run had TRAIN_BUDGET_SECONDS=240.
   • ✓startup · GET /health answered 200 after 4.0 s
   • ✓GET /vocab · {"tokens": 16384, "special": 1}
   • ✓POST /score · {"id": "validation-0001", "bytes": 5203, "tokens": 1329}
   • ✓POST /score, characters outside the corpus · {"bytes": 1035, "tokens": 958}
   • ✓POST /dist · {"prefix_tokens": 664, "logprobs": 16384, "logsumexp": 7.345173501371871e-08, "score_minus_dist": 0.0}
   • ✓evaluation deadline · Projected from 177 of 1067 requests at concurrency 8: 91 of 551 /score requests on public documents in 2.5 s and 86 of 516 /dist and truncation requests in 2.2 s, each scaled to its full count, plus the first requests (0.4 s) and the restart and its first /dist (7 s) as measured
   • ✓slowest request · 183 answered requests, median 0.18 s
   • ✓restart startup · GET /health after the kill and the wipe of candidate state
   • ✓requests answered · Every request was answered.
   • ✓/dist normalization · max |logsumexp| over 84 audited /dist positions
   • ✓/score vs /dist · max |score - dist| over the same positions
   • ✓/score vs /dist bias · mean(score - dist) over the same positions
   • ✓truncated text · max difference over 536 shared positions of 2 truncated texts
   • ✓healthy after requests

The agent's last message before its session ended.