AI-text detection
- Score
- 87.2
- Wallet spend
- $21.35 of $50
- Model cost
- $5.23
- Agent time
- 2h 30m
- Steps
- 182
- Tokens in / out
- 16.6M / 210k
- Self-checks
- 3 passed of 3 used
- Rank at $50 wallet
- 5th of 14
Score
87.2raw metric 0.8724
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.1276
- reward
- 0.8724
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9161
- threshold_5pct
- 0.7042
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0553
- macro_auroc
- 0.9865
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh only installs the vendored wheels and runs server.py with networking set to offline. server.py sends each reque
serve.sh only installs the vendored wheels and runs server.py with networking set to offline. server.py sends each request to a single Detector in detector.py. That Detector loads one local XLM-RoBERTa sequence classifier from ./model with local_files_only and returns sigmoid((l1-l0)/2). No other model, external API or ensemble is involved in scoring. The weight inventory for model/model.safetensors shows one XLMRobertaForSequenceClassification, BF16 throughout, with a [2,1024] classifier.out_proj head.
✓no retrieval of human textThe serving path is server.py -> detector.py -> normalization.py. It does no corpus lookup, nearest-neighbour or index s
The serving path is server.py -> detector.py -> normalization.py. It does no corpus lookup, nearest-neighbour or index search, and makes no network retrieval. It only normalizes the text, tokenizes it (truncated to 512 tokens) and runs a forward pass of the classifier. serve.sh sets HF/transformers offline mode.
✓stateless per request scoringserver.py uses a batching queue (8 ms wait, up to 32 items) purely to improve throughput. Each item gets its own logit f
server.py uses a batching queue (8 ms wait, up to 32 items) purely to improve throughput. Each item gets its own logit from the model. Nothing is cached or keyed on text, and no state carries over between requests. detector.py computes every score independently from the input text.
✓no evaluation special casingserver.py, detector.py and normalization.py contain no hardcoded texts, hashes, IDs or lookup tables from the evaluation
server.py, detector.py and normalization.py contain no hardcoded texts, hashes, IDs or lookup tables from the evaluation set. The only fallback is a constant 0.5 when an exception occurs. Normalization (NFKC, removing invisible characters, mapping lookalike characters, CJK spacing) is applied uniformly to every input.
✓own training data provenanceEvery corpus is declared in architecture.json and source/prepare_data.py. (1) DetectRL-X at pinned revision ccc9a0a7…: p
Every corpus is declared in architecture.json and source/prepare_data.py. (1) DetectRL-X at pinned revision ccc9a0a7…: prepare_data.py skips any row whose split field is not 'train', and evidence/data_manifest.json and evidence/prepare.log record 48000 non-train rows excluded. train_stage1.py and train.py also assert that every row has source_split=='train'. (2) The 200 provided examples, which are authorized and used with group-hash two-fold CV. (3) gpt-4o-mini paraphrases from source/generate_synthetic.py, generated only from human train rows chosen by source/select_synthetic_sources.py with the internal holdout excluded. Nowhere in the trajectory or code does the participant access inductionlabs/frontier-ml-tasks or DetectRL-X test rows; the only hf_hub_download fetches DetectRL-X at the pinned revision.
✓actual inference uses reviewed weightsdetector.py loads Path(__file__).parent/'model'. The shipped model/model.safetensors is a full XLM-RoBERTa-large classif
detector.py loads Path(__file__).parent/'model'. The shipped model/model.safetensors is a full XLM-RoBERTa-large classifier (393 BF16 tensors plus a classifier head). The trajectory shows these weights came from final_run/step_400. That run was train.py warmstarted from the train_stage1 checkpoint at step 300, which itself was initialized from pinned FacebookAI/xlm-roberta-large with a new head. The checkpoint was exported as detector-final-s400.tar, extracted into /workspace/output, and its SHA256 was checked against evidence/final_model_checksums.json (trajectory#155, #161, #174). The offline self-check then served this exact bundle through serve.sh (trajectory#179).
✓harness credentials used only by the harnessThe participant code uses only credentials scoped to the participant's own Spin session. These are an OpenRouter proxy k
The participant code uses only credentials scoped to the participant's own Spin session. These are an OpenRouter proxy key created with `spin openrouter key create` (trajectory#88-89), which source/generate_synthetic.py uses for paraphrase generation, and a Spin wallet transfer session used for S3 artifact transfer (trajectory#155). The trajectory contains no use of harness API keys such as OPENAI_API_KEY and no `codex exec` calls. An in-session check confirmed that the logs contain no credential fields (trajectory#142).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed26m 25s into the runtook 2m 14s
startup 48.6 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 48.6 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9881308789536605}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 15.2 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed28m 58s into the runtook 2m 39s
startup 69.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 69.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9881308789536605}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.2 s for the first request, then 19.6 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 3passed2h 29m into the runtook 2m 3s
startup 45.9 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 45.9 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9862576795040977}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 13.4 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with the trained multilingual detector, offline dependencies, serving code, and training provenance.
Final offline self-check passes:
- Startup: 45.9 seconds
- Throughput: 369.5 passages/second
- Responses: 5,000/5,000 valid