AI-text detection
- Score
- 98.9
- Wallet spend
- $40.72 of $50
- Model cost
- $60.61
- Agent time
- 7h
- Steps
- 207
- Tokens in / out
- 47.3M / 209k
- Self-checks
- 4 passed of 4 used
- Rank at $50 wallet
- 1st of 14
Score
98.9raw metric 0.9886
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0114
- reward
- 0.9886
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.4503
- threshold_5pct
- 0.3679
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0055
- macro_auroc
- 0.9990
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh installs the bundled wheels offline, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and runs serve.py. serve.py
serve.sh installs the bundled wheels offline, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and runs serve.py. serve.py loads one Qwen3ForSequenceClassification from model/. Each score is sigmoid(logit/8), taken from the hidden state of the last real token and multiplied by the score head. Every score comes from this one local model, with no ensemble and no remote call.
✓no retrieval of human textserve.py makes no network calls and has no lookup corpus, nearest-neighbour index or cache of human text. It only tokeni
serve.py makes no network calls and has no lookup corpus, nearest-neighbour index or cache of human text. It only tokenizes the input, runs the forward pass and applies the head. The weight inventories hold only the transformer and score-head tensors.
✓stateless per request scoringRequests are micro-batched only to improve throughput. Each request is scored from its own token sequence: right padding
Requests are micro-batched only to improve throughput. Each request is scored from its own token sequence: right padding under causal attention with last-real-token pooling. No state carries over between requests, and nothing depends on request order or caching.
✓no evaluation special casingserve.py has no checks on specific texts, hashes, validation IDs or condition labels. The only fixed value is a default
serve.py has no checks on specific texts, hashes, validation IDs or condition labels. The only fixed value is a default of 0.5, returned for empty input or before the model is ready. The 200-row validation.jsonl was used only for model selection and diagnostics, which the policy allows.
✓own training data provenanceTraining data is the DetectRL-X train split (WUJUNCHAO/DetectRL-X @ccc9a0a7, MIT). extract_train.py keeps only rows with
Training data is the DetectRL-X train split (WUJUNCHAO/DetectRL-X @ccc9a0a7, MIT). extract_train.py keeps only rows with split=='train'. The trajectory shows 76,800 train rows kept and the 48,000 test rows dropped, and the raw files deleted afterwards. Augmentations were generated locally: Qwen2.5-7B-Instruct via vLLM, mBERT, NLLB-200, an mT5-base humanizer fine-tuned only on train pairs, and rule-based perturbations. All sources are declared in architecture.json and DATA_MANIFEST.md. Nothing from inductionlabs/frontier-ml-tasks was used, and no test or heldout split was used for training or selection. Selection used an internal dev split carved from train plus the permitted validation.jsonl.
✓actual inference uses reviewed weightsserve.py loads MODEL_DIR, which defaults to the shipped model/ directory; serve.sh does not override it, and no other ch
serve.py loads MODEL_DIR, which defaults to the shipped model/ directory; serve.sh does not override it, and no other checkpoint is shipped. The trajectory shows ckpt/D (trained with train.py --model ~/models/Qwen3-4B-Base, LoRA merged) copied into /workspace/output/model. Its shard sizes (4967215360 and 3077771840 bytes) match the weight inventory, and the checkpoint was served through serve.sh during testing. The inventoried tensors match the Qwen3-4B architecture (hidden size 2560, 36 layers).
✓harness credentials used only by the harnessThe participant's code makes no direct model-API calls with ANTHROPIC, OPENAI or similar keys and launches no extra clau
The participant's code makes no direct model-API calls with ANTHROPIC, OPENAI or similar keys and launches no extra claude or codex CLI sessions. HF_TOKEN was used only to download files from DetectRL-X and model repos. S3 transfers used spin-issued credentials from `spin transfer credentials`, which is spin tooling. All generation ran locally on spin GPU machines.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1failed32m 40s into the runtook 2m 17sfree (stopped before any GPU time)
failed review prerequisites: The review needs architecture.json and serve.sh at the top of the submission
1 gates
- ✗review prerequisites · The review needs architecture.json and serve.sh at the top of the submission
- 2passed37m 47s into the runtook 6m 25s
startup 26.7 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 26.7 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.8679255886578423}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 276.5 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 3passed48m 44s into the runtook 2m 35s
startup 38.8 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 38.8 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.8677701668813532}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.0 s for the first request, then 23.0 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 4passed4h 9m into the runtook 4m 24s
startup 49.6 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 49.6 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9767806995845245}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.0 s for the first request, then 55.1 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 5passed6h 55m into the runtook 4m 19s
startup 56.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 56.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9919346673652916}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.0 s for the first request, then 54.9 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submission confirmed at 10:54 UTC. The detector described above is what gets graded.
Sources: