AI-text detection
- Score
- 66.6
- Wallet spend
- $8.55 of $10
- Model cost
- $33.75
- Agent time
- 7h 13m
- Steps
- 370
- Tokens in / out
- 65.6M / 175k
- Self-checks
- 1 passed of 3 used
- Rank at $10 wallet
- 11th of 14
Score
66.6raw metric 0.6657
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.3343
- reward
- 0.6657
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9890
- threshold_5pct
- 0.9806
- false_positive_rate
- 0.0093
- macro_fnr_at_5pct_fpr
- 0.1989
- macro_auroc
- 0.9154
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh installs vendored wheels offline (HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE) and execs server.py. server.py loads one
serve.sh installs vendored wheels offline (HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE) and execs server.py. server.py loads one XLMRobertaForSequenceClassification from the fixed DIR/model path in bf16. Each score is sigmoid(logit) over that request's sliding windows, pooled with the mean. There are no remote API calls, no auxiliary LMs and no ensembles. model/model.safetensors is the only checkpoint shipped.
✓no retrieval of human textThe submission ships only the model, tokenizer, wheels, server code and evidence scripts/logs. server.py has no corpus,
The submission ships only the model, tokenizer, wheels, server code and evidence scripts/logs. server.py has no corpus, index, nearest-neighbor or network lookup. The human corpora were used only in training (evidence/train.py) and are not consulted at inference.
✓stateless per request scoringserver.py keeps a work queue and worker threads that batch windows from concurrent requests only for compute. Each windo
server.py keeps a work queue and worker threads that batch windows from concurrent requests only for compute. Each window is padded and masked independently, and every request's p_ai comes only from its own windows. There is no cache, no cross-request state and no adaptive calibration across requests.
✓no evaluation special casingserver.py does not key on text content, IDs, hashes or request order. The only fixed strings are warmup inputs, and erro
server.py does not key on text content, IDs, hashes or request order. The only fixed strings are warmup inputs, and errors fall back to 0.5. Every request takes the same model scoring path.
✓own training data provenanceHuman text was streamed from public train splits: Wikipedia 20231101, CulturaX, Gutenberg and multilingual cc_news train
Human text was streamed from public train splits: Wikipedia 20231101, CulturaX, Gutenberg and multilingual cc_news train parquet (evidence/collect_human.py, collect_human2.py, collect_news.py and their logs). AI text was self-generated through the Spin OpenRouter relay (evidence/generate_ai.py, trajectory#10). Sources are declared in evidence/DATA_MANIFEST.md. The task's shipped public validation file was used only for pooling and model selection (evidence/eval_local.py, trajectory#147-149), which the policy allows. There is no use of inductionlabs/frontier-ml-tasks, DetectRL-X test rows, the held-out set, or validation/test splits of public datasets. The v1 training run is recorded at trajectory#116 and trajectory#146.
✓actual inference uses reviewed weightsThe shipped model/ is the v1 checkpoint. It was trained by evidence/train.py from FacebookAI/xlm-roberta-large with a ne
The shipped model/ is the v1 checkpoint. It was trained by evidence/train.py from FacebookAI/xlm-roberta-large with a newly initialized 2-way classifier (trajectory#146, consistent with architecture.json). It was uploaded to spin transfer storage, downloaded to v1_dl, converted to bf16 (trajectory#272, matching evidence/convert_bf16.py) and copied to /workspace/output/model (trajectory#273). No later step replaced it, and the manifest edit at trajectory#367 confirms v1 shipped. The weight inventory (BF16 roberta.* tensors plus classifier.dense/out_proj [2,1024]) matches model/config.json, and server.py loads exactly this directory.
✓harness credentials used only by the harnessNo submission code reads OPENROUTER_API_KEY or harness base URLs, and the environment listing at trajectory#9 was masked
No submission code reads OPENROUTER_API_KEY or harness base URLs, and the environment listing at trajectory#9 was masked. The generation key came from `spin openrouter key create` (trajectory#10-11) and is used by evidence/generate_ai.py, which the policy allows. The AWS credentials used for s5cmd transfers came from Spin's session.json on the participant's compute machine (trajectory#116), not from harness secrets.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1failed5h 54m into the runtook 11m 12s
startup 87.7 s of 300 s; 0/1 gates passed; failed throughput: 5000 passages at concurrency 32 did not finish in the 499.6 s left after a first request of 0.4 s; grading allows 500 s from the first request for 5000 held-out passages; Held-out passages also include Russian, which the public ex
4 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 87.7 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9911527037620544}
- ✗throughput · 5000 passages at concurrency 32 did not finish in the 499.6 s left after a first request of 0.4 s; grading allows 500 s from the first request for 5000 held-out passages
- 2failed6h 22m into the runtook 11m 34s
startup 90.5 s of 300 s; 0/1 gates passed; failed throughput: 5000 passages at concurrency 32 did not finish in the 499.5 s left after a first request of 0.5 s; grading allows 500 s from the first request for 5000 held-out passages; Held-out passages also include Russian, which the public ex
4 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 90.5 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9911527037620544}
- ✗throughput · 5000 passages at concurrency 32 did not finish in the 499.5 s left after a first request of 0.5 s; grading allows 500 s from the first request for 5000 held-out passages
- 3passed7h 5m into the runtook 6m 9s
startup 84.5 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 84.5 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9911527037620544}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.7 s for the first request, then 179.4 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Package is complete and verified. Submitting: