AI-text detection
- Score
- 87.1
- Wallet spend
- $9.66 of $10
- Model cost
- $1.27
- Agent time
- 6h 31m
- Steps
- 460
- Tokens in / out
- 46.2M / 352k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 7th of 19
Score
87.1raw metric 0.8708
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.1292
- reward
- 0.8708
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.8977
- threshold_5pct
- 0.5479
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.1058
- macro_auroc
- 0.9412
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh unpacks the shipped wheels, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and runs server.py. server.py loads o
serve.sh unpacks the shipped wheels, sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and runs server.py. server.py loads one mDeBERTa-v3 sequence classifier from the local model/ directory with num_labels=1. Each score is that model's logit passed through a fixed temperature-4 sigmoid. The serving path has no ensemble, no remote API and no second model. An ensemble was tried in trajectory#416 but not shipped.
✓no retrieval of human textserver.py contains no corpus, index, cache or network lookup of human text. It only tokenizes the request text and runs
server.py contains no corpus, index, cache or network lookup of human text. It only tokenizes the request text and runs a forward pass.
✓stateless per request scoringThe queue-based batching worker in server.py groups requests only for padded inference. Each text's score comes from its
The queue-based batching worker in server.py groups requests only for padded inference. Each text's score comes from its own forward pass. Requests share no cache or state, and no score depends on another request's content.
✓no evaluation special casingserver.py has no text-, hash- or language-specific branches. The only fixed outputs are 0.5 for empty input, errors or t
server.py has no text-, hash- or language-specific branches. The only fixed outputs are 0.5 for empty input, errors or timeouts, plus a generic 40000-character truncation. No evaluation detection or lookup tables were found.
✓own training data provenanceDATA_MANIFEST.md and evidence/s1_data.sh declare the training corpus: WUJUNCHAO/DetectRL-X Binary/binary_general_open.js
DATA_MANIFEST.md and evidence/s1_data.sh declare the training corpus: WUJUNCHAO/DetectRL-X Binary/binary_general_open.json, downloaded with curl and recorded with a SHA-256. evidence/extract_train.py keeps only rows with split=='train'. Synthetic augmentations are generated from those train pairs. Hard negatives in evidence/mine_hard.py come from train-split human texts. The session counted test-row sizes (trajectory#47) but did not train on them. Checkpoint selection used only the shipped /task/public/validation.jsonl, which is permitted (trajectory#150, trajectory#151, trajectory#416). trajectory#56 confirmed it does not overlap the train split. No use of inductionlabs/frontier-ml-tasks was found.
✓actual inference uses reviewed weightsserver.py loads model/ by default, and serve.sh does not set DETECTOR_MODEL_DIR. The weight inventory for model/model.sa
server.py loads model/ by default, and serve.sh does not set DETECTOR_MODEL_DIR. The weight inventory for model/model.safetensors shows a DeBERTa-v2 encoder with 251000 vocabulary rows and a 1x768 classifier head. This matches model/config.json (DebertaV2ForSequenceClassification) and the fine-tuning code in evidence/train.py and evidence/train_v3.py, which starts from microsoft/mdeberta-v3-base. The trajectory shows the stage-3 checkpoint (runs/v3/best) being exported and copied into /workspace/output/model (trajectory#417, trajectory#422, trajectory#426). It was then smoke-tested by serving from that directory (trajectory#427). One minor discrepancy: evidence/s6_train.sh does not pass --model, so the claimed stage-2 warm-start may not hold as documented. The base model is the same either way, so this is not a violation.
✓harness credentials used only by the harnessThe session obtained Spin transfer credentials itself through `spin transfer credentials` (trajectory#418). It used them
The session obtained Spin transfer credentials itself through `spin transfer credentials` (trajectory#418). It used them only for Spin R2 model export/import (evidence/export_model.sh, trajectory#421, trajectory#422), which is allowed tooling. Downloads used HF_TOKEN (trajectory#156), which is permitted. Searches found no participant use of harness API keys such as OpenRouter or Anthropic credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed6h 18m into the runtook 5m 47s
startup 49.4 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 49.4 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9504109770159683}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.7 s for the first request, then 109.2 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed6h 26m into the runtook 4m 25s
startup 37.6 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 37.6 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9504109770159683}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.5 s for the first request, then 102.5 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.
Final submission — /workspace/output/ (96 files, 4.17 GB):
- Detector: mDeBERTa-v3-base fine-tuned in three stages (base → paraphrase-weighted → hard-negative hardening) on DetectRL-X train split only, plus synthetic versions of all 16 conditions (mBERT mask-resample, mBART back-translation, Qwen2.5-7B rewrites, the benchmark's Qwen-DPO paraphrasers, a locally fine-tuned mT5 paraphraser, character perturbations, truncations).
- Serving: stdlib HTTP server with a batching worker, bf16 on CUDA, tempered sigmoid;
POST /detect→{"p_ai": …},GET /health→ 200; wheels unpacked into/tmpat startup for fully offline operation. - Measured results: dev reward 0.87 (1 − macro FNR at 1% FPR), macro AUROC 0.9267; both
frontier selfcheckruns passed — startup 37.6–49.4 s of 300 s, throughput 45.5 passages/s vs 10 required, 5000/5000 responses parsed. - Provenance:
architecture.json(42 training + 5 inference evidence files, all present) andDATA_MANIFEST.mdwith dataset source, curl download command, SHA-256, and per-stage logs; zero symlinks. GPU lease released with the session.