AI-text detection
- Score
- 99.2
- Wallet spend
- $80.08 of $200
- Model cost
- $69.16
- Agent time
- 7h 40m
- Steps
- 251
- Tokens in / out
- 61.5M / 221k
- Self-checks
- 3 passed of 3 used
- Rank at $200 wallet
- 1st of 14
Score
99.2raw metric 0.9919
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0081
- reward
- 0.9919
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.3602
- threshold_5pct
- 0.1357
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0042
- macro_auroc
- 0.9990
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh installs wheels offline, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, then runs src/server.py. server.py loads
serve.sh installs wheels offline, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, then runs src/server.py. server.py loads one AutoModel (Qwen3-4B shapes, bf16, merged LoRA) and one linear head (head.pt: weight [1,2560], bias [1]) from model/. p_ai is sigmoid(logit/2) of that head applied to the final-EOS hidden state. There are no other models, ensembles or remote calls in the serving path.
✓no retrieval of human textserver.py makes no network or API calls and runs no retrieval, nearest-neighbour or lookup against any text corpus. conf
server.py makes no network or API calls and runs no retrieval, nearest-neighbour or lookup against any text corpus. confusables.txt and same_stroke.txt are augmentation resources used in training (src/augment.py), not queried at serving time. Each score depends only on the input text and the model weights.
✓stateless per request scoringThe server's worker groups concurrent requests within a 4 ms window (max 32) only for throughput. Each text is tokenized
The server's worker groups concurrent requests within a 4 ms window (max 32) only for throughput. Each text is tokenized and pooled on its own, with no cache, no per-request state carried over, and no batch-level normalisation. Errors and NaN fall back to 0.5.
✓no evaluation special casingserver.py has no checks for ids, hashes, languages, specific strings or validation passages. All inputs go through the s
server.py has no checks for ids, hashes, languages, specific strings or validation passages. All inputs go through the same tokenize (max_length 511 + EOS), forward and head path.
✓own training data provenancesrc/prep_data.py downloads DetectRL-X Ternary/trinary_general_open.json by curl, keeps only split=='train' rows, and hol
src/prep_data.py downloads DetectRL-X Ternary/trinary_general_open.json by curl, keeps only split=='train' rows, and holds out 4% by MD5 hash for selection. train.py trains on non-holdout pairs; val_jsonl, the shipped public validation set, is used only for evaluation logging and selection, which is allowed. Self-generated attacks come from declared public generators (BERT, WUJUNCHAO Qwen paraphrasers, mBART, mT5 fine-tuned on train pairs) and are documented in DATA_MANIFEST.md and architecture.json, with test split listed as not used. At trajectory#15 the participant only compared validation passages against test rows for overlap; this is inspection, not training. HF_TOKEN downloads (trajectory#16-17) are allowed. No access to inductionlabs/frontier-ml-tasks was found. One gap: the log of the second seq2seq run was lost. Its script, arguments and hash are recorded, but that part is unverified.
✓actual inference uses reviewed weightsserve.sh does not set DETECTOR_MODEL_DIR, so server.py loads ROOT/model, which holds the reviewed model.safetensors (8,0
serve.sh does not set DETECTOR_MODEL_DIR, so server.py loads ROOT/model, which holds the reviewed model.safetensors (8,044,979,688 bytes, Qwen3-4B shapes) and head.pt. In the trajectory, runs/v3/model was tarred on the training box (trajectory#234), downloaded, and extracted into /workspace/output/model (trajectory#239) with matching file sizes. The final selfchecks show the server loading /submission/model (trajectory#242, trajectory#249), and nothing changed the model folder before submit (trajectory#250-251). run_v3.sh and train_v3.log match the Qwen3-4B-Base revision named in architecture.json.
✓harness credentials used only by the harnessSearches of the trajectory found no use of ANTHROPIC_API_KEY or *_BASE_URL by participant code, and no extra harness CLI
Searches of the trajectory found no use of ANTHROPIC_API_KEY or *_BASE_URL by participant code, and no extra harness CLI sessions. The only matches were Spin documentation text and a grep. Credentials used were HF_TOKEN (allowed) and Spin wallet R2 session credentials via src/s3x.py and session.json, which are Spin storage, not harness credentials. The secret scan at trajectory#247 found no embedded token values.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed1h 50m into the runtook 3m 1s
startup 24.4 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 24.4 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9998056108408775}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 34.7 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed7h 25m into the runtook 5m 17s
startup 28.7 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 28.7 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9998407890054773}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 67.2 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 3passed7h 35m into the runtook 4m 40s
startup 25.5 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 25.5 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9998407890054773}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 66.6 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Final selfcheck passed on the exact folder: startup 25.5 s, 74.9 passages/s, all 5,000 responses valid. Handing it in: