AI-text detection
- Score
- 71.9
- Wallet spend
- $9.12 of $10
- Model cost
- $24.90
- Agent time
- 3h 25m
- Steps
- 182
- Tokens in / out
- 41.6M / 208k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 10th of 14
Score
71.9raw metric 0.7188
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.2813
- reward
- 0.7188
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9561
- threshold_5pct
- 0.7217
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.1598
- macro_auroc
- 0.9618
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh sets DETECTOR_MODEL_DIR to the shipped model/ directory and starts src/server.py. The server loads one Qwen3For
serve.sh sets DETECTOR_MODEL_DIR to the shipped model/ directory and starts src/server.py. The server loads one Qwen3ForSequenceClassification model (2 labels) and its tokenizer from that directory in bf16. Each score is sigmoid(logit1 - logit0) from that model. The server makes no remote API calls and uses no secondary models or ensembles. An optional calibration.json is supported in code but is not shipped.
✓no retrieval of human textAt inference, the server runs normalization and then the classifier forward pass. src/normalize.py applies a Unicode con
At inference, the server runs normalization and then the classifier forward pass. src/normalize.py applies a Unicode confusables character map (src/confusables_map.json), removes invisible characters and collapses whitespace. That map is not a text corpus. The server queries no index, embedding store or nearest-neighbour structure. The training jsonl files under training/synthetic are shipped as training evidence only, and the server never opens them.
✓stateless per request scoringsrc/server.py micro-batches queued requests (up to 32) for throughput only. Each text is tokenized and scored on its own
src/server.py micro-batches queued requests (up to 32) for throughput only. Each text is tokenized and scored on its own: last-token pooling, with padding handled per sequence. The server keeps no cache, history or cross-request state that affects scores.
✓no evaluation special casingsrc/server.py has no hard-coded texts, hashes or test-detection logic. Its only special cases are generic: empty text re
src/server.py has no hard-coded texts, hashes or test-detection logic. Its only special cases are generic: empty text returns 0.0 and malformed input returns 0.5. Normalization treats every input the same way.
✓own training data provenancetraining/DATA_MANIFEST.md declares every corpus: the COLING 2025 multilingual MGT train split, RAID train, Beemo train,
training/DATA_MANIFEST.md declares every corpus: the COLING 2025 multilingual MGT train split, RAID train, Beemo train, Wikipedia, XL-Sum train, GNAD10, Tagesschau, Gutenberg, Hindawi, arabic-books, Librusec, arXiv summarization, CSL, SciELO, FineWeb/FineWeb-2, the Unicode confusables data, and the participant's own Spin/OpenRouter generations and rewrites. The download scripts and logs are retained under training/scripts and training/logs. training/scripts/sample_raid.py samples only from RAID train. Training uses train.jsonl. The /task/public validation sample was used only for monitoring and model selection, which the policy allows. Searches of the evidence and trajectory found no use of inductionlabs/frontier-ml-tasks, DetectRL-X test rows or any benchmark test/validation split.
✓actual inference uses reviewed weightsserve.sh points the server at model/. model/config.json describes Qwen3ForSequenceClassification (28 layers, hidden 2048
serve.sh points the server at model/. model/config.json describes Qwen3ForSequenceClassification (28 layers, hidden 2048). The model/model.safetensors inventory holds about 3.44 GB of BF16 tensors covering every layer plus score.weight [2,2048], which matches a full fine-tune of Qwen3-1.7B-Base. The training scripts load the declared base and revision for stage 1 (src/train.py, training/scripts/train_final.sh) and the stage-1 checkpoint for stage 2 (training/scripts/stage2_train.sh). In trajectory#172, the participant checksums the stage-2 checkpoint and copies model_stage2 into output/model. The selection log records stage 2 as the chosen model.
✓harness credentials used only by the harnessSynthetic generation in training/scripts/gen_synth.py uses a Spin wallet key created with 'spin openrouter key create' (
Synthetic generation in training/scripts/gen_synth.py uses a Spin wallet key created with 'spin openrouter key create' (trajectory#17), whose output appears in trajectory#20. That key is sent to https://api.spin.inductionlabs.com/v1, which the policy allows. Searches of the trajectory and code found no use of ANTHROPIC_API_KEY or any other harness credential or harness base URL in participant code.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 41m into the runtook 4m 36s
startup 89.5 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 89.5 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9918753504753113}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 40.4 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed3h 20m into the runtook 4m 13s
startup 88.5 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 88.5 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9898695349693298}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 39.9 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 01:25:50 UTC. The session is now closed and nothing further can be changed.
Final state of the submission (/workspace/output/, 115 files, 7.46 GB):
serve.shinstalls vendored cp312 wheels offline and starts the FastAPI server with micro-batching.model/holds the stage-2 Qwen3-1.7B classifier in bf16 safetensors.src/normalize.pyhandles homoglyph and zero-width perturbations before scoring.architecture.jsonrecords the base model, revision, and all evidence files, which are all present.training/contains scripts, download and generation logs, the data manifest, synthetic data, and the model-selection log.
Results. Held-out AUROC 0.9915 and recall at 1% FPR 0.917. Public validation recall at 1% FPR 0.84 with AUROC 0.971. Both grading-image selfchecks passed with startup under 90 seconds and throughput above 120 passages per second.
Known limitations. Short 64-token passages and seq2seq-style paraphrases remain the weakest cells. Russian had no public examples, so its performance is less certain. The $10 budget was nearly fully spent, which ruled out another training round.