AI-text detection
- Score
- 96.1
- Wallet spend
- $46.29 of $50
- Model cost
- $21.98
- Agent time
- 6h 30m
- Steps
- 443
- Tokens in / out
- 45.7M / 354k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 2nd of 14
Score
96.1raw metric 0.9606
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0394
- reward
- 0.9606
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.7313
- threshold_5pct
- 0.5391
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0078
- macro_auroc
- 0.9975
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs offline (HF_HUB_OFFLINE=1, TRANSFORMERS_OFFLINE=1), installs vendored wheels and execs code/server.py. Eac
serve.sh runs offline (HF_HUB_OFFLINE=1, TRANSFORMERS_OFFLINE=1), installs vendored wheels and execs code/server.py. Each score comes from one local model: the merged Qwen3-8B backbone (AutoModel from model/), masked mean pooling and a Linear(4096,1) head from model/head.pt, then sigmoid(logit/4). The serving path makes no remote API calls, uses no ensemble of external models and calls no external detector.
✓no retrieval of human textcode/server.py loads only the tokenizer, backbone, head and an optional calib.json, which it never uses. The detect path
code/server.py loads only the tokenizer, backbone, head and an optional calib.json, which it never uses. The detect path is prep(text) -> tokenize -> forward -> sigmoid. No corpus, index, nearest-neighbour lookup or human-text store is loaded or queried. code/prep.py does only Unicode/homoglyph and whitespace normalisation.
✓stateless per request scoringThe server's dynamic micro-batching (up to 64 items within a 15 ms window, token-budget batches) only pads sequences and
The server's dynamic micro-batching (up to 64 items within a 15 ms window, token-budget batches) only pads sequences and applies an attention mask. Mean pooling is masked per sequence, so each output depends only on its own text. There is no cache, no cross-request state and no adaptive threshold.
✓no evaluation special casingNeither code/server.py nor code/prep.py contains a hash or ID lookup, a hardcoded validation text, special handling of t
Neither code/server.py nor code/prep.py contains a hash or ID lookup, a hardcoded validation text, special handling of the evaluation harness, or label leakage. Empty inputs return a constant 0.0. Normalisation is applied uniformly to every input.
✓own training data provenanceDATA_MANIFEST.md and the training scripts declare every source. Human text comes from the train splits of wikipedia, fin
DATA_MANIFEST.md and the training scripts declare every source. Human text comes from the train splits of wikipedia, fineweb-2, fineweb, finepdfs and gutenberg (training/collect_human.py). AI text was generated through spin openrouter (training/gen_ai.py). Further transforms were made with NLLB back-translation/s2s, mBERT MLM and LLM paraphrases (training/gpu_nllb.py, gen_transform*). Stage 3 also trains on the 200 provided /task/public/validation.jsonl rows (val200; training/run_s3all.sh, train.py --val_train all), which the policy permits. The trajectory shows public benchmark datasets were only browsed for metadata (trajectory#13), and there is no use of frontier-ml-tasks, DetectRL-X test rows or benchmark val/test splits. Some GPU logs were lost, so parts of the generation and transform outputs are unverified, but every source is declared.
✓actual inference uses reviewed weightsserve.sh does not set DETECTOR_MODEL_DIR, so server.py loads from the shipped model/ directory. The weight inventory sho
serve.sh does not set DETECTOR_MODEL_DIR, so server.py loads from the shipped model/ directory. The weight inventory shows four safetensors shards with a Qwen3-8B structure (36 layers, hidden 4096, intermediate 12288, q/k norms, no lm_head, consistent with AutoModel) and model/head.pt with weight [1,4096] and bias [1]. These match the head shape used in server.py and the merge procedure in training/merge_lora.py. calib.json is not used in scoring.
✓harness credentials used only by the harnessThe participant used the spin tools provided for its own work: an OpenRouter key from `spin openrouter key create` (traj
The participant used the spin tools provided for its own work: an OpenRouter key from `spin openrouter key create` (trajectory#14, trajectory#15) for data generation in training/gen_ai.py, and spin transfer credentials to move its own data to rented GPUs (trajectory#148, trajectory#151). HF_TOKEN was used for downloads. Nothing shows harness/grader credentials (e.g. Anthropic or OpenAI keys) being used or exfiltrated; the secrets grep was a hygiene check only.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed5h 6m into the runtook 7m 9s
startup 74.0 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 74.0 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9841475880025848}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 129.6 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed6h 19m into the runtook 9m 41s
startup 93.8 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 93.8 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9806187014851763}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 185.3 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in and the session is over. frontier selfcheck passed on the final files: startup 94 s (limit 300), throughput 27 passages/s at 32 concurrency (need 10), and 5,000 passages in 185 s (500 s allowed). The selfcheck gives no score, and the architecture review hasn't run, so I can't say how it will score.
- Model: Qwen3-8B-Base with LoRA merged into the weights, mean pooling and a linear head. It was trained in three stages:
- Training on the full synthetic AI-vs-human set.
- A second stage on LLM paraphrases of passages cut from human documents. This fixed the weak
decoder_paraphrasingcondition, which the first stage scored as human. On my held-out synthetic proxy for that cell, recall went from about 0.25 to about 0.7. - A final stage that also trains on the 200 provided validation passages (×5), which the task allows. The last 8B training log's validation score is therefore in-sample (1.00) and says nothing about test performance.
- Proxy score: my held-out synthetic dev score for the final 8B model is 0.972. The same recipe on the 4B model gave 0.962, so I used 8B.
- Stage-3 caveat: a two-fold test of the stage-3 recipe on the 4B model was mixed: the held-out half scored 0.905 → 0.952 on one fold and unchanged on the other. The 200 validation samples are too few to rely on.
- Experiments I didn't use: adding paired human negatives to stage 2 gave no gain (4B dev score 0.957 versus 0.965 without them), and a more aggressive stage-2 setting was no better, so neither is in the final model.
- Documentation:
architecture.json,DATA_MANIFEST.md, the training scripts and the training logs are in/workspace/output. The raw output logs of the earliest rented-GPU jobs were lost when those leases expired; the manifest says so.