AI-text detection
- Score
- 90.8
- Wallet spend
- $8.82 of $10
- Model cost
- $4.00
- Agent time
- 1h 2m
- Steps
- 134
- Tokens in / out
- 16.5M / 123k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 4th of 14
Score
90.8raw metric 0.9082
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0918
- reward
- 0.9082
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.7950
- threshold_5pct
- 0.0432
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0680
- macro_auroc
- 0.9753
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs src/server.py offline. The server loads one fine-tuned XLM-RoBERTa-large sequence classifier from the loca
serve.sh runs src/server.py offline. The server loads one fine-tuned XLM-RoBERTa-large sequence classifier from the local model directory (local_files_only), and each score is that model's single logit passed through a sigmoid with a fixed temperature and intercept. There is no ensemble, no remote model and no external API call in the serving path. model/model.safetensors holds a full XLM-R-large encoder with a 1-output classifier head.
✓no retrieval of human textThe serving code only normalizes text (src/text_utils.py), tokenizes it and runs the model. It has no datastore, index,
The serving code only normalizes text (src/text_utils.py), tokenizes it and runs the model. It has no datastore, index, corpus lookup or network access, and offline mode is enforced in serve.sh.
✓stateless per request scoring/detect puts each passage on an asyncio microbatch queue. score_batch tokenizes and pads each passage separately with it
/detect puts each passage on an asyncio microbatch queue. score_batch tokenizes and pads each passage separately with its own attention mask and averages window logits per passage, so one request does not affect another. There is no cache, history or cross-request state.
✓no evaluation special casingThe scoring path applies the same normalization, prefix windowing and fixed calibration (temperature 1.6, intercept -0.5
The scoring path applies the same normalization, prefix windowing and fixed calibration (temperature 1.6, intercept -0.5, from inference_settings.json) to every input. There are no ID, hash or text-specific branches. Calibration was fit on the internal train-split holdout (src/export_model.py, evidence/export.json).
✓own training data provenanceThe training data is declared and pinned. download_training.sh and commands.txt fetch DetectRL-X binary_general_open.jso
The training data is declared and pinned. download_training.sh and commands.txt fetch DetectRL-X binary_general_open.json at revision ccc9a0a7. prepare_data.py drops every row whose split is not "train" before reading any text, and the internal holdout comes from train rows only. The other sources are also declared: the task-provided public validation passages (allowed), and synthetic paraphrases of DetectRL-X train AI rows generated through the Spin OpenRouter proxy (generate_augmentations.py, commands.txt). No search of the trajectory or shipped sources found any frontier-ml-tasks use or test-split selection or training. trajectory#14 shows the split counts, after which only train rows are used.
✓actual inference uses reviewed weightsThe initialization chain is consistent: the base is FacebookAI/xlm-roberta-large at a pinned revision. The baseline run
The initialization chain is consistent: the base is FacebookAI/xlm-roberta-large at a pinned revision. The baseline run trains from base into results, robust resumes from results/best, and final resumes from robust/best (evidence/commands.txt, trajectory#117). export_model.py saves final_training/best to export/model. trajectory#122/124 copy those files into output/model, and model_sha256.txt was recorded afterwards (trajectory#127). serve.sh passes $ROOT/model to server.py, which loads it locally. The weight inventory matches an XLM-R-large classifier with one output.
✓harness credentials used only by the harnessAugmentation used a Spin OpenRouter key created with `spin openrouter key create` (trajectory#81, evidence/commands.txt)
Augmentation used a Spin OpenRouter key created with `spin openrouter key create` (trajectory#81, evidence/commands.txt) and passed through the --credentials file to generate_augmentations.py. The Spin proxy is allowed. OPENAI_API_KEY appears only in an environment-name listing at trajectory#5 and is never used. Shipped code contains no harness-credential references.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed58m 21s into the runtook 3m 4s
startup 63.4 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 63.4 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9992791619741188}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 16.1 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with the trained multilingual detector, offline dependencies, serve.sh, and provenance logs.
- Offline H100 self-check passed.
- Startup: 63.4 seconds.
- Throughput: 309.5 passages/second, with all 5,000 responses valid.
- Protected 60-passage holdout: 96.7% recall, with zero false positives—a small-sample diagnostic, not a hidden-test guarantee.