AI-text detection
- Score
- 98.3
- Wallet spend
- $7.01 of $10
- Model cost
- $76.68
- Agent time
- 8h 53m
- Steps
- 199
- Tokens in / out
- 48.8M / 189k
- Self-checks
- 4 passed of 4 used
- Rank at $10 wallet
- 1st of 14
Score
98.3raw metric 0.9834
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0166
- reward
- 0.9834
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.5107
- threshold_5pct
- 0.3433
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0049
- macro_auroc
- 0.9989
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh installs offline wheels, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, then runs server.py. server.py loads one
serve.sh installs offline wheels, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, then runs server.py. server.py loads one AutoModelForSequenceClassification from the shipped model/ directory. No ensemble.json is shipped, so it uses a single Member and p_ai = sigmoid(logit/4). The server makes no remote calls and uses no second LM signal. config.json and the safetensors inventory match a Qwen2.5-3B Qwen2ForSequenceClassification with score.weight [1,2048].
✓no retrieval of human textThe serving path has no corpus, index, nearest-neighbour lookup or network access. The train_src/resources tables (confu
The serving path has no corpus, index, nearest-neighbour lookup or network access. The train_src/resources tables (confusables and homoglyphs) are character-substitution tables used only by the training-time augmentation in train_src/augment.py. server.py does not load them.
✓stateless per request scoringThe server.py queue worker groups concurrent requests within a 4 ms window only to pad them into one batch. Each text is
The server.py queue worker groups concurrent requests within a 4 ms window only to pad them into one batch. Each text is tokenized and scored on its own. There is no cache, no history and no cross-request state. Errors return 0.5, and warmup uses fixed strings.
✓no evaluation special casingserver.py has no text hashing, keyword or fingerprint rules, request-order logic or eval-set lookup tables. Every score
server.py has no text hashing, keyword or fingerprint rules, request-order logic or eval-set lookup tables. Every score comes straight from the classifier logit with a fixed temperature.
✓own training data provenanceTraining data is the DetectRL-X binary and ternary data, train split only. download_detectrlx*.py keeps split=='train',
Training data is the DetectRL-X binary and ternary data, train split only. download_detectrlx*.py keeps split=='train', writes those rows and deletes the raw file. The logs show 76,800 train rows written out of 124,800 and the raw file removed. The augmentations (mBERT, mBART, mT5 seq2seq fine-tuned on train-split pairs, and Qwen2.5-7B paraphrases) were generated by the participant from the train split. The dev and selection data are train held-out rows plus the allowed public validation file. DATA_MANIFEST.md declares every source, with licenses and commands. The trajectory shows no access to the evaluation dataset.
✓actual inference uses reviewed weightsThe training command (trajectory#156) and the final training log show Qwen/Qwen2.5-3B fine-tuned with LoRA, merged, and
The training command (trajectory#156) and the final training log show Qwen/Qwen2.5-3B fine-tuned with LoRA, merged, and saved with a newly initialized score head. Trajectory#192-194 show the merged checkpoint transferred with matching sha256 and copied to output/model. The shipped config and tensor inventory match Qwen2.5-3B. server.py defaults MODEL_DIR to the shipped model/ directory, serve.sh does not override AIDET_MODEL_DIR, and no alternate checkpoint is shipped.
✓harness credentials used only by the harnessThe OpenRouter key used for data augmentation in gen_decoder_paraphrase.py was the participant's own key, created with `
The OpenRouter key used for data augmentation in gen_decoder_paraphrase.py was the participant's own key, created with `spin openrouter key create` (trajectory#30). Searches of the trajectory found no use of harness Anthropic or API credentials, environment dumps or BASE_URL overrides. Web searches were ordinary harness tool calls.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed1h 18m into the runtook 12m 12s
startup 22.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 22.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9263037302939515}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 337.2 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed3h 35m into the runtook 3m 10s
startup 27.8 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 27.8 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9825575152637948}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 31.6 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 3passed4h 22m into the runtook 3m 20s
startup 21.5 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 21.5 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9693241982893259}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 46.8 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 4passed8h 50m into the runtook 3m 27s
startup 29.9 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 29.9 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9678992932829918}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 50.4 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
The output holds 90 files (10.17 GB), the same contents the final selfcheck verified. Submitting: