AI-text detection
- Score
- 0.0
- Wallet spend
- $33.86 of $200
- Model cost
- $2.62
- Agent time
- 8h 31m
- Steps
- 270
- Tokens in / out
- 21.8M / 60k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 18th of 19
Score
0.0raw metric 0
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 1
- reward
- 0
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 1.0000
- threshold_5pct
- 1.0000
- false_positive_rate
- 0
- macro_fnr_at_5pct_fpr
- 0.5436
- macro_auroc
- 0.7987
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.py loads one local AutoModelForSequenceClassification (XLM-RoBERTa-large, 2 labels) from <dir>/model. It returns p
serve.py loads one local AutoModelForSequenceClassification (XLM-RoBERTa-large, 2 labels) from <dir>/model. It returns p_ai as the softmax probability of class 1. There is no ensemble, no external API and no secondary scorer. serve.sh only installs local wheels offline and then runs serve.py.
✓no retrieval of human textserve.py contains no corpus lookup, retrieval index, nearest-neighbour search or network calls. Scoring is a forward pas
serve.py contains no corpus lookup, retrieval index, nearest-neighbour search or network calls. Scoring is a forward pass over the normalized request text only.
✓stateless per request scoringThe dynamic batcher (up to 32 items, 8 ms wait) pads each sequence with attention masks, so every score depends only on
The dynamic batcher (up to 32 items, 8 ms wait) pads each sequence with attention masks, so every score depends only on its own text. There is no cache, no cross-request state and no adaptation.
✓no evaluation special casingnormalize_text only removes zero-width characters and collapses whitespace. There are no hardcoded texts, hashes, length
normalize_text only removes zero-width characters and collapses whitespace. There are no hardcoded texts, hashes, length or language hacks, and nothing specific to the evaluation set. The public validation.jsonl was used only as a diagnostic, which is permitted.
✓own training data provenanceEvery training source is declared in data_manifest.json and code: M4 (git clone, trajectory#61), HC3 en/zh all.jsonl and
Every training source is declared in data_manifest.json and code: M4 (git clone, trajectory#61), HC3 en/zh all.jsonl and HC3-ru train, HC3 French train file only (downloads at trajectory#76/#77; the French test/val files were not used), Wikipedia 20231101 es/pt, and local Qwen2.5-7B generations (gen_ai.py). Checkpoint selection used a 5% stratified split of this training corpus (trajectory#163). I found no access to frontier-ml-tasks, DetectRL-X or held-out data. The silentone0725 dataset was only browsed as metadata and never downloaded. gen_para.py and shorten.py ran after the model had been copied, so they did not feed the shipped weights.
✓actual inference uses reviewed weightsmodel/model.safetensors (2,239,618,672 bytes) has an XLM-R-large tensor layout with a 2-way classifier head. It was copi
model/model.safetensors (2,239,618,672 bytes) has an XLM-R-large tensor layout with a 2-way classifier head. It was copied from model_out/best, produced by evidence/train.py (trajectory#163–#193), and its size matches at trajectory#264. serve.sh does not override MODEL_DIR, so serve.py loads <dir>/model, which holds these weights.
✓harness credentials used only by the harnessThe participant created a Spin OpenRouter-proxy key and used it only to list models through the Spin proxy (trajectory#5
The participant created a Spin OpenRouter-proxy key and used it only to list models through the Spin proxy (trajectory#54, #55), which is allowed. HF_TOKEN was used for Hugging Face downloads on the remote machine (trajectory#57), also allowed. I found no chat-completion calls, no claude/codex CLI launches and no ANTHROPIC or other key use in the trajectory or in the shipped code.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed8h 21m into the runtook 7m 8s
startup 76.3 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 76.3 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9999921321868896}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.6 s for the first request, then 129.7 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.