AI-text detection
- Score
- 76.5
- Wallet spend
- $8.51 of $10
- Model cost
- $20.36
- Agent time
- 10h 22m
- Steps
- 308
- Tokens in / out
- 31.3M / 272k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 8th of 14
Score
76.5raw metric 0.7646
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.2354
- reward
- 0.7646
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9417
- threshold_5pct
- 0.9277
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.1546
- macro_auroc
- 0.9341
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh sets MODEL_DIRS to model/ and model_b/ with HF offline mode. server.py loads one AutoModelForSequenceClassifica
serve.sh sets MODEL_DIRS to model/ and model_b/ with HF offline mode. server.py loads one AutoModelForSequenceClassification per directory and returns p_ai as the mean over the two models of 0.5 + atan(logit1 - logit0)/pi. Both are XLM-RoBERTa-large 2-label classifiers fine-tuned by the participant (evidence/train.py), and together they form one local detector ensemble. Serving makes no external LM or API calls.
✓no retrieval of human textserver.py has no index, corpus, embedding store or lookup. It only tokenizes the input and runs the classifiers. The shi
server.py has no index, corpus, embedding store or lookup. It only tokenizes the input and runs the classifiers. The shipped serving artifacts are model weights, the tokenizer and offline wheels. Nothing retrieves or compares against stored human text.
✓stateless per request scoringserver.py's queue-based micro-batching only groups concurrent requests into one padded forward pass with attention masks
server.py's queue-based micro-batching only groups concurrent requests into one padded forward pass with attention masks. Each output depends only on its own text. There is no cache, memo, cross-request state or adaptive threshold.
✓no evaluation special casingserver.py has no text hashing, keyed lookup, language- or ID-specific branches, or hardcoded answers. /detect returns th
server.py has no text hashing, keyed lookup, language- or ID-specific branches, or hardcoded answers. /detect returns the model probability, or 0.5 on error.
✓own training data provenanceDATA_MANIFEST.md, evidence/download_corpora.py and evidence/assemble_human.py declare the human sources: train splits on
DATA_MANIFEST.md, evidence/download_corpora.py and evidence/assemble_human.py declare the human sources: train splits only of public HF corpora (wikipedia, wikisource, xlsum, gutenberg, c4, pubmed, wiki_lingua, multi_eurlex, and others). AI text was generated via the Spin OpenRouter proxy (evidence/gen_api.py) and local Qwen2.5-7B/NLLB transforms. The task's shipped public validation file (evidence/eval_val.py) was used only for checkpoint selection, which is allowed. Trajectory searches found no access to DetectRL-X, the inductionlabs frontier-ml-tasks repo or held-out data. Downloads are logged where retained; any public source without a retained log is unverified but declared.
✓actual inference uses reviewed weightsserve.sh loads ${HERE}/model and ${HERE}/model_b, and both are inventoried (model/model.safetensors and model_b/model.sa
serve.sh loads ${HERE}/model and ${HERE}/model_b, and both are inventoried (model/model.safetensors and model_b/model.safetensors). Each has the XLM-RoBERTa-large layout: 250002x1024 embeddings, 24 encoder layers, and classifier.out_proj of shape [2,1024] in F16. This matches the fp16 checkpoint saved by evidence/train.py from base FacebookAI/xlm-roberta-large, as declared in architecture.json.
✓harness credentials used only by the harnessTrajectory searches found no participant use of ANTHROPIC/OPENAI/OPENROUTER_API_KEY and no launch of claude or codex CLI
Trajectory searches found no participant use of ANTHROPIC/OPENAI/OPENROUTER_API_KEY and no launch of claude or codex CLIs. The OpenRouter key came from `spin openrouter key create` (trajectory#29, written at trajectory#30) and is used by evidence/gen_api.py, which the Spin allowance permits. HF_TOKEN was used only for `hf download` on the rented GPU. No push_to_hub or upload_file appears anywhere. File transfers went to Spin's own s3 transfer bucket, not to HF.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed10h 15m into the runtook 6m 8s
startup 74.9 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 74.9 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.94715507413}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.2 s for the first request, then 132.5 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
117 files, 6.31 GB, all checks green. Submitting: