AI-text detection
- Score
- 43.5
- Wallet spend
- $8.18 of $50
- Model cost
- $10.73
- Agent time
- 5h 29m
- Steps
- 228
- Tokens in / out
- 54.9M / 231k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 14th of 14
Score
43.5raw metric 0.4349
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.5651
- reward
- 0.4349
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9900
- threshold_5pct
- 0.9702
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.3913
- macro_auroc
- 0.8343
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs only code/serve_app.py with HF offline mode on. model_lib.load_detector builds one XLM-R encoder from the
serve.sh runs only code/serve_app.py with HF offline mode on. model_lib.load_detector builds one XLM-R encoder from the local config, loads model/model.safetensors and a single nn.Linear(768,1) head from model/head.pt, and loads the tokenizer with local_files_only. Each score is the sigmoid of that one model's output; long texts are split into chunks whose scores are combined by max or mean. There is no second model, no remote API call and no ensemble.
✓no retrieval of human textThe serving code (serve_app.py, model_lib.py, augment.py) contains no corpus, index, lookup table or network access. The
The serving code (serve_app.py, model_lib.py, augment.py) contains no corpus, index, lookup table or network access. The only preprocessing is normalize_text (NFKC normalization and zero-width stripping), and the score comes only from the neural model.
✓stateless per request scoring/detect normalizes the request text, queues it in a dynamic batcher and returns that item's own score. Batching groups r
/detect normalizes the request text, queues it in a dynamic batcher and returns that item's own score. Batching groups requests but each sequence is scored independently. There is no cache, no history and no cross-request statistics, and warmup uses fixed dummy strings.
✓no evaluation special casingThe server has no text-specific, id-specific or hash-based branches; on error it returns a constant 0.5. Checkpoint sele
The server has no text-specific, id-specific or hash-based branches; on error it returns a constant 0.5. Checkpoint selection used the 200 public validation examples, which the task permits, together with the participant's own hash-held-out pool.
✓own training data provenanceEvery source is declared in train/DATA_MANIFEST.md and implemented in the training scripts. prep_public.py uses M4 raw G
Every source is declared in train/DATA_MANIFEST.md and implemented in the training scripts. prep_public.py uses M4 raw GitHub data, RAID train.csv, MAGE train.csv, HC3 all.jsonl and Wikipedia train. dev_sources.py uses Wikipedia, FineWeb/FineWeb-2 and XL-Sum train splits plus Gutenberg. The rest is self-generated synthetic data and NLLB back-translations. The trajectory shows the participant listed RAID and MAGE test/extra files but never used them, and the manifest explicitly excludes them. No inductionlabs/frontier-ml-tasks data, held-out data or DetectRL test rows appear anywhere in the evidence. Selection used the participant's own held-out split plus the permitted public dev200 examples.
✓actual inference uses reviewed weightsThe weight inventory shows model/model.safetensors is a full XLM-R-base encoder (250002x768 embeddings, 514 positions) a
The weight inventory shows model/model.safetensors is a full XLM-R-base encoder (250002x768 embeddings, 514 positions) and model/head.pt has weight [1,768] and bias [1], which matches the shapes model_lib.load_detector expects. logs/train_base3.log shows base3 trained from the declared FacebookAI/xlm-roberta-base revision with no --init. At trajectory#215 the output was built with CKPT=/workspace/artifacts/ckpts/base3_best, which was copied from /work/out/base3/best at trajectory#204. The self-check server log loads /submission/code/../model (trajectory#223).
✓harness credentials used only by the harnessThe harness credential (META_API_KEY, visible in the step-3 env listing) is never used by the participant. Synthetic gen
The harness credential (META_API_KEY, visible in the step-3 env listing) is never used by the participant. Synthetic generation used a separately created Spin OpenRouter proxy key (trajectory#18), and HF_TOKEN was used only for read-only dataset access, as the task allows. Before submission the participant searched the output for leaked secrets (trajectory#224).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed5h 21m into the runtook 6m 33s
startup 78.4 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 78.4 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9504110217094421}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 173.5 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. XLM-RoBERTa-base fine-tuned for 8-language AI-text detection (ar, zh, en, fr, de, pt, ru, es).
- Data: public train splits only (M4, RAID-train, MAGE-train, HC3, Wikipedia, FineWeb, Gutenberg) + 24k self-generated passages (6 models × 8 langs × 6 conditions) + NLLB attack mimics; dev200 used only for selection.
- Local validation: 0.50 reward (1% FPR), 0.84 AUROC; selfcheck PASSED (startup 78s/300s, 28.8/s vs 10/s on H100).
POST /detectreturnsp_aivia max-pooled chunk scoring with Unicode normalization; offline vendored wheels included.