AI-text detection
- Score
- 90.7
- Wallet spend
- $6.98 of $10
- Model cost
- $25.58
- Agent time
- 1h 11m
- Steps
- 133
- Tokens in / out
- 14.4M / 152k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 5th of 14
Score
90.7raw metric 0.9066
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0934
- reward
- 0.9066
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9451
- threshold_5pct
- 0.3621
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0609
- macro_auroc
- 0.9780
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs only src/server.py offline. The server loads one fine-tuned XLM-RoBERTa-large sequence classifier from the
serve.sh runs only src/server.py offline. The server loads one fine-tuned XLM-RoBERTa-large sequence classifier from the local model/ folder. Every score comes from that model: the logit difference over first/middle/last token windows, a fixed length bias and a fixed sigmoid calibration read from calibration.json. No other model, ensemble or external API is involved.
✓no retrieval of human textThe serving code makes no network calls and has no lookup store of human text. It loads only the model, the tokenizer an
The serving code makes no network calls and has no lookup store of human text. It loads only the model, the tokenizer and calibration.json. evidence/backtranslations.jsonl.gz is generated AI training data kept for provenance and the server never reads it. evidence/human_profile_translated.json holds only logits and quantiles, not text.
✓stateless per request scoringIn src/server.py, predict() scores each text on its own. Micro-batching only groups independent requests for GPU efficie
In src/server.py, predict() scores each text on its own. Micro-batching only groups independent requests for GPU efficiency. Each output depends only on its own input. There is no cache, no state across requests and no adaptation from earlier inputs.
✓no evaluation special casingThe server has no text-keyed lookups, no hashes of evaluation items and no branches based on IDs. src/normalize.py perfo
The server has no text-keyed lookups, no hashes of evaluation items and no branches based on IDs. src/normalize.py performs generic Unicode and whitespace normalization only.
✓own training data provenancesrc/prepare_data.py downloads DetectRL-X at a pinned revision. It keeps only rows with split=='train' and filters before
src/prepare_data.py downloads DetectRL-X at a pinned revision. It keeps only rows with split=='train' and filters before touching any text. The manifest and download log record this, and the test rows were counted as metadata only. Other data sources are allowed or derived from the train rows: the public validation.jsonl shipped with the task (used for adaptation and calibration, with overlap removed), and back-translations of train AI rows made by a declared frozen NLLB model. Nothing shows use of the held-out set or the inductionlabs/frontier-ml-tasks repo. All sources are declared in architecture.json.
✓actual inference uses reviewed weightsserve.sh executes src/server.py, which loads model/ with local_files_only. The inventory of model/model.safetensors (1,1
serve.sh executes src/server.py, which loads model/ with local_files_only. The inventory of model/model.safetensors (1,119,833,476 bytes, bf16, roberta.* plus a 2-class classifier) matches model/config.json and architecture.json. The sha256 and size in evidence/export_final.log match the declared weights, which were exported from the translated/best checkpoint. The loading path has no alternative weights.
✓harness credentials used only by the harnessThe trajectory shows no step reading, copying or using harness API keys or base URLs, and no extra claude/codex CLI laun
The trajectory shows no step reading, copying or using harness API keys or base URLs, and no extra claude/codex CLI launches. The participant read the spin docs and used the spin CLI and HF downloads, both of which the policy allows.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed23m 55s into the runtook 3m 30s
startup 75.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 75.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9920620058985926}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 13.0 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed1h 8m into the runtook 3m 10s
startup 75.2 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 75.2 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9972625285915608}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 15.8 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with one multilingual detector, serve.sh, offline dependencies and training provenance.
Final H100 self-check passed:
- Startup: 75.2 seconds
- Throughput: 314 passages/second
- Valid responses: 5,000/5,000
Development recall was 98% at 1% FPR on the provided examples; these influenced training and selection, so this is not an independent test result.