AI-text detection
- Score
- 88.0
- Wallet spend
- $9.34 of $10
- Model cost
- $3.99
- Agent time
- 2h 8m
- Steps
- 132
- Tokens in / out
- 11.9M / 79k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 6th of 19
Score
88.0raw metric 0.8799
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.1201
- reward
- 0.8799
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.5636
- threshold_5pct
- 0.0914
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0589
- macro_auroc
- 0.9814
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs serve.py offline (HF_HUB_OFFLINE=1). serve.py loads exactly one XLM-RoBERTa-large sequence classifier from
serve.sh runs serve.py offline (HF_HUB_OFFLINE=1). serve.py loads exactly one XLM-RoBERTa-large sequence classifier from ./model with local_files_only. The score is the sigmoid of the class-1 logit, and no other model, ensemble or external API is involved. The weight inventory for model/model.safetensors lists a single RoBERTa encoder plus a 2-class classifier head in BF16.
✓no retrieval of human textThe serving code makes no network calls and keeps no datastore, index or corpus of human text. Inference is just tokeniz
The serving code makes no network calls and keeps no datastore, index or corpus of human text. Inference is just tokenizer plus model forward on the request text.
✓stateless per request scoringserve.py micro-batches requests dynamically, but each passage's score comes from its own padded forward pass. Nothing is
serve.py micro-batches requests dynamically, but each passage's score comes from its own padded forward pass. Nothing is cached, scores are not normalized across requests, and no state is carried between requests. Errors return a fixed 0.5.
✓no evaluation special casingserve.py has no text- or ID-specific branches, hash lookups or embedded evaluation data. The 200 provided public example
serve.py has no text- or ID-specific branches, hash lookups or embedded evaluation data. The 200 provided public examples were used only in permitted training and cross-fitting (make_provided_folds.py, train_detector.py).
✓own training data provenancedata_manifest.json and data_summary.json declare DetectRL-X binary_general_open.json at a pinned revision with its sha25
data_manifest.json and data_summary.json declare DetectRL-X binary_general_open.json at a pinned revision with its sha256. prepare_data.py keeps split=='train' rows only, and data_preparation.log confirms 76,800 train pairs with the 48,000 test rows discarded. The only other data is the task's provided public examples (/task/public/validation.jsonl), which policy allows. model/training_metadata.json shows the final data was train_plus_provided_all.jsonl, which trajectory#81 builds from train_general.jsonl plus provided_all_train.jsonl. The backbone xlmr_large_v1 was trained on train_general.jsonl only (trajectory#41/42 region, steps-26-42). An OpenRouter paraphrase-augmentation set generated from public-train AI passages (trajectory#55, trajectory#56) was not used in the shipped model. frontier-ml-tasks was never used, and no test or val split was used for training or selection.
✓actual inference uses reviewed weightsserve.sh does not override DETECTOR_PATH, so serve.py loads ROOT/model. trajectory#109 copies the xlmr_final_v2 weights
serve.sh does not override DETECTOR_PATH, so serve.py loads ROOT/model. trajectory#109 copies the xlmr_final_v2 weights into output/model, and their sha256 b1be45b4… matches architecture.json. model/training_metadata.json records initialization from FacebookAI/xlm-roberta-large at revision c23d21b0, resumed from the xlmr_large_v1 backbone. The weight inventory matches that architecture.
✓harness credentials used only by the harnessThe participant only listed env vars (trajectory#18). The external LLM key was obtained with `spin openrouter key create
The participant only listed env vars (trajectory#18). The external LLM key was obtained with `spin openrouter key create`, using the Spin proxy base URL (trajectory#54), and used from a credential file (trajectory#56). This is the allowed Spin/OpenRouter path. No harness ANTHROPIC/OPENAI keys or base URLs were used from participant code.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 6m into the runtook 6m 53s
startup 74.2 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 74.2 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9284088015556335}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 13.0 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed1h 54m into the runtook 3m 16s
startup 73.2 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 73.2 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9284088015556335}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.0 s for the first request, then 13.9 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully at 2026-10-23 23:56:12 UTC.
- Exact-artifact self-check: passed
- Startup: 73.2 s / 300 s
- Throughput: 358.6 passages/s at concurrency 32
- Responses: 5,000/5,000 valid
- Official public validator: 1.0 diagnostic score at 1% FPR
- Cross-fitted macro recall estimate: 0.92
- Submission digest:
c01b48d9292242abdd84796f9bbe475a8eda1ba3413b192d739db5793a86b767