AI-text detection
- Score
- 94.9
- Wallet spend
- $81.50 of $200
- Model cost
- $22.31
- Agent time
- 4h 10m
- Steps
- 382
- Tokens in / out
- 38.5M / 307k
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 2nd of 14
Score
94.9raw metric 0.9486
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0514
- reward
- 0.9486
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.8289
- threshold_5pct
- 0.5301
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0078
- macro_auroc
- 0.9968
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh sets offline environment variables, installs the bundled wheels with --no-index, then runs server/serve_app.py
serve.sh sets offline environment variables, installs the bundled wheels with --no-index, then runs server/serve_app.py --model-dir model. serve_app.py tokenizes the text and gets a score from one Qwen3-8B-Base + LoRA + linear-head model, loaded by server/qwen3_min.py from model/model.safetensors, model/lora.safetensors and model/head.safetensors. The score is p_ai = sigmoid(logit/4). No external APIs, no ensembles of outside services, and no other scorers.
✓no retrieval of human textThe serving code (serve_app.py, qwen3_min.py, attacks.py) only normalizes text, tokenizes it and runs the forward pass.
The serving code (serve_app.py, qwen3_min.py, attacks.py) only normalizes text, tokenizes it and runs the forward pass. It has no corpus, index, lookup table, nearest-neighbour search or network retrieval of human text. attacks.py contains normalization and perturbation functions only, with no text tables.
✓stateless per request scoringRequests are batched dynamically: sorted by length, right-padded, with an attention mask. Under the causal mask, and wit
Requests are batched dynamically: sorted by length, right-padded, with an attention mask. Under the causal mask, and with pooling limited to masked tokens, each text's score is independent of the other texts in its batch. There is no cache, no persistent state across requests, and no order- or batch-dependent calibration. Warmup uses random token ids, and errors return a fixed 0.5.
✓no evaluation special casingNo logic depends on text hashes, ids, request order or known evaluation items. Scoring is the same model forward pass fo
No logic depends on text hashes, ids, request order or known evaluation items. Scoring is the same model forward pass for every input. I found no references to the frontier-ml-tasks repository or to DetectRL-X test data anywhere in the evidence.
✓own training data provenanceDATA_MANIFEST.md and architecture.json declare the sources. Human text comes from public train splits: FineWeb-2 train s
DATA_MANIFEST.md and architecture.json declare the sources. Human text comes from public train splits: FineWeb-2 train shards, FineWeb CC-MAIN-2013-20, wikimedia/wikipedia 20231101, Project Gutenberg, and the train parquets of XL-Sum, MLSUM and CNN/DM. The download scripts (dl_raw.sh, dl_raw2.sh, dl_deu.sh, dl_news.py, build_news2.py) fetch only these train files. AI text was generated by the participant through the Spin OpenRouter proxy (orlib.py). Attacks come from an LLM, xlm-roberta-base MLM, and character-level perturbation. train.py and stage2.py train only on samples from these pools, using a hash-based internal train/val split. The public validation.jsonl (200 rows, allowed for any use) served only for monitoring and checkpoint selection. Stage 3, which fine-tuned on those rows, was a CV experiment and was not shipped. No prohibited test or validation splits were used.
✓actual inference uses reviewed weightsqwen3_min.load_detector loads model.safetensors, lora.safetensors and head.safetensors with strict=True. The model/confi
qwen3_min.load_detector loads model.safetensors, lora.safetensors and head.safetensors with strict=True. The model/config.json and model/meta.json values (36 layers, hidden size 4096, lora_r 32, alpha 64, temperature 4, Qwen/Qwen3-8B-Base revision 49e3418f) match architecture.json and the training code. train/export_model.py writes exactly these files from the stage-2 checkpoint, and the trajectory shows that export (--ckpt ../ckpt_s2, revision 49e3418f) producing the shipped lora, head, meta and config files. The base model is loaded with AutoModel.from_pretrained at the declared revision.
✓harness credentials used only by the harnessThe only API credential in the session is a Spin OpenRouter-proxy key that the participant created with `spin openrouter
The only API credential in the session is a Spin OpenRouter-proxy key that the participant created with `spin openrouter key create` (trajectory#15). It was written to .or_env and used by data_build/orlib.py to generate data (trajectory#16), which the policy allows. I found no use of harness or Anthropic keys and no `claude -p` calls.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 32m into the runtook 7m 43s
startup 66.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 66.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9325113998730665}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 108.3 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed4h 2m into the runtook 7m 1s
startup 66.2 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 66.2 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9939522857925378}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 107.1 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
The detector is submitted, at 01:51 UTC. The final selfcheck passed: startup 66 s (limit 300 s) and 46.6 passages/s at 32 concurrent requests (limit 10/s).
On the 200 validation passages the shipped model scores a reward of about 0.96, with AUROC 0.9986. That set has no Russian, so Russian performance rests on held-out synthetic Russian data only.
The shipped model is Qwen3-8B-Base with a LoRA adapter and a linear head, trained in two stages. It was not trained on any validation row.
Last, I ran a 5-fold cross-validation that fine-tuned on the validation rows. Its out-of-fold reward was 0.94, no better than 0.96, so I did not use it. The experiment is documented in DATA_MANIFEST.md with its logs under logs/training/stage3_cv_experiment/.
The selfcheck's architecture review did not run, so architecture.json has not been checked by it. I added weight hashes and a short training summary to that file after the selfcheck, but changed no serving code. I also released the B200 box.