AI-text detection
- Score
- 54.1
- Wallet spend
- $15.42 of $200
- Model cost
- $8.34
- Agent time
- 4h 59m
- Steps
- 211
- Tokens in / out
- 20.2M / 245k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 13th of 14
Score
54.1raw metric 0.5407
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.4593
- reward
- 0.5407
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9826
- threshold_5pct
- 0.9674
- false_positive_rate
- 0.0093
- macro_fnr_at_5pct_fpr
- 0.1546
- macro_auroc
- 0.9588
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs server.py offline with HF_HUB_OFFLINE=1. server.py loads one fine-tuned XLM-RoBERTa-large classifier from
serve.sh runs server.py offline with HF_HUB_OFFLINE=1. server.py loads one fine-tuned XLM-RoBERTa-large classifier from model/ and returns its sigmoid output as p_ai. The scoring path has no external API, second model or other signal source.
✓no retrieval of human textserver.py goes straight from normalize to tokenize to forward pass to sigmoid. It has no index, corpus, nearest-neighbou
server.py goes straight from normalize to tokenize to forward pass to sigmoid. It has no index, corpus, nearest-neighbour lookup or network call, and no human text ships for use at inference time.
✓stateless per request scoringThe Batcher micro-batches requests and pads them with attention masks. Each request's score depends only on its own text
The Batcher micro-batches requests and pads them with attention masks. Each request's score depends only on its own text. There is no cache, no cross-request state and no score calibration across a batch.
✓no evaluation special casingNo hash tables, ID or text matching, or hard-coded outputs appear in server.py or textnorm.py. Missing or invalid input
No hash tables, ID or text matching, or hard-coded outputs appear in server.py or textnorm.py. Missing or invalid input falls back to a generic default. Normalization is the same in training and serving.
✓own training data provenanceThe AI text was generated during the session through the spin OpenRouter proxy (train/gen_*.py, train/orclient.py). evid
The AI text was generated during the session through the spin OpenRouter proxy (train/gen_*.py, train/orclient.py). evidence/data_manifest.md declares the human corpora: Wikipedia, XLSum, GNAD10, ML-ArXiv, cyberleninka, Gutenberg, C4, the Russian classics set and ruslit. The fetch scripts take train splits or raw dumps only, using HF_TOKEN, which is allowed. dev200 is the shipped public validation file (/task/public/validation.jsonl), which may be used for training and selection. I found no access to the inductionlabs frontier-ml-tasks repo or the DetectRL test rows; steps 10-11 only read Hugging Face metadata.
✓actual inference uses reviewed weightsThe model/model.safetensors inventory has the standard XLM-R-large tensors plus a classifier head with 1 output (393 ten
The model/model.safetensors inventory has the standard XLM-R-large tensors plus a classifier head with 1 output (393 tensors, BF16), which matches the strict load_state_dict in server.py. Training starts from FacebookAI/xlm-roberta-large (train/train.py), then train/finetune_dev.py continues from /root/train_out/best. In steps 194-195 the trajectory exports /root/ft_full/best to /workspace/output/model, and the selfcheck in step 208 shows the server loading those 393 tensors.
✓harness credentials used only by the harnessThe OR_KEY used for generation was created with `spin openrouter key create` (step 12) and points at the spin proxy, whi
The OR_KEY used for generation was created with `spin openrouter key create` (step 12) and points at the spin proxy, which is allowed. Searches found no use of ANTHROPIC or OPENAI keys and no launches of the claude or codex CLIs. HF_TOKEN use was limited to dataset and model downloads.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed4h 56m into the runtook 2m 50s
startup 65.6 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 65.6 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9879462122917175}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 16.2 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Selfcheck PASSED all gates on the real H100: startup 65.6s/300s, throughput 307 passages/s (30× the requirement), 5000/5000 responses parsed, healthy after. Note it flags that held-out includes Russian (absent from the public samples) — my training data covers ru across all conditions (AI generations + ruslit/cyberleninka/c4/xlsum human pools), so that's handled. Submitting: