AI-text detection
- Score
- 96.1
- Wallet spend
- $8.13 of $10
- Model cost
- $28.31
- Agent time
- 4h 50m
- Steps
- 381
- Tokens in / out
- 38.4M / 401k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 2nd of 14
Score
96.1raw metric 0.9613
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0387
- reward
- 0.9613
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.4671
- threshold_5pct
- 0.2833
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0150
- macro_auroc
- 0.9967
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh sets offline HF env vars, installs vendored wheels and runs serve/server.py. server.py loads one local Qwen3-8B
serve.sh sets offline HF env vars, installs vendored wheels and runs serve/server.py. server.py loads one local Qwen3-8B-architecture backbone (merged LoRA) from model/ plus a linear head from model/head.safetensors. It mean-pools over the mask and returns sigmoid(logit/4). The score comes from no other model, ensemble or remote service.
✓no retrieval of human textThe serving path has no network calls, corpus lookup, nearest-neighbour index or retrieval. HF_HUB_OFFLINE and TRANSFORM
The serving path has no network calls, corpus lookup, nearest-neighbour index or retrieval. HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE are set. The score is computed only from the request text through the local model.
✓stateless per request scoringserver.py's POST /detect reads only the `text` field, normalizes and truncates it, and scores it with the fixed model. M
server.py's POST /detect reads only the `text` field, normalizes and truncates it, and scores it with the fixed model. Micro-batching groups concurrent requests with attention masks and mean pooling over each item's own mask, so one request's score doesn't depend on another's. There's no cache or state that persists across requests, and the warm-up uses a fixed dummy text.
✓no evaluation special casingThe scoring code has no hashing or keying on eval texts, no lookup tables and no branches for particular inputs. Every i
The scoring code has no hashing or keying on eval texts, no lookup tables and no branches for particular inputs. Every input takes the same normalize → tokenize → model → head path. Validation data appears only in training and export scripts, for evaluation and the allowed stage-1 vs stage-2 selection.
✓own training data provenancetrain/prepare_detectrl_train.py downloads WUJUNCHAO/DetectRL-X binary_general_open.json at a pinned revision and keeps o
train/prepare_detectrl_train.py downloads WUJUNCHAO/DetectRL-X binary_general_open.json at a pinned revision and keeps only split=='train' rows. The trajectory shows test rows were counted and then dropped (trajectory#19), and the other DetectRL-X files were compared on train rows only (trajectory#21, #22). The synthetic pools (Spin OpenRouter LLM transforms, NLLB/mBERT transforms, Qwen2.5-7B-Instruct humanizer paraphrases) are declared with generator code and logs in DATA_MANIFEST.md and architecture.json. Neither the code nor the session uses inductionlabs/frontier-ml-tasks. I didn't decompress or inspect the data/*.jsonl.gz pools, but their sources are declared and their generator code is shipped.
✓actual inference uses reviewed weightsserver.py uses DETECTOR_MODEL_DIR only if set and otherwise defaults to the shipped ../model directory. serve.sh doesn't
server.py uses DETECTOR_MODEL_DIR only if set and otherwise defaults to the shipped ../model directory. serve.sh doesn't set it. model/model.safetensors (15,136,854,944 bytes, BF16, Qwen3-8B shapes) and model/head.safetensors ([1,4096] weight, [1] bias) match what train/export_merged.py produced. That script merged the LoRA adapter into Qwen/Qwen3-8B-Base at revision 49e3418, as recorded in logs/export_merged_stage2.log and trajectory#359, which shows the same byte size. model/serve_config.json is consistent with this.
✓harness credentials used only by the harnesstrain/llmgen.py reads /workspace/.or_key.json, which the participant created with `spin openrouter key create` (trajecto
train/llmgen.py reads /workspace/.or_key.json, which the participant created with `spin openrouter key create` (trajectory#15). That is the allowed Spin OpenRouter proxy. Searches of the session for harness API keys, base-URL overrides and extra harness CLI launches (e.g. 'claude -p') found only a self-check grep and Spin documentation. Neither code nor session shows any use of the harness's credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed3h 37m into the runtook 6m 10s
startup 23.0 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 23.0 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9980291382938591}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.7 s for the first request, then 116.0 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed4h 41m into the runtook 8m 56s
startup 23.2 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 23.2 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9967860647365605}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.7 s for the first request, then 112.3 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in and the session is over. The final stage-2 package passed frontier selfcheck: startup in 23 s, 44.2 passages/s at 32 concurrency (the requirement is 10), and all 5,000 responses valid.
The detector is Qwen3-8B-Base with a LoRA adapter and a linear head on mean-pooled hidden states, and the adapter is merged into the model for serving. Stage 2 added a pool of "humanized" paraphrases to the training data. The six real paraphrasing passages among the 200 provided samples went from 1/6 to 4/6 above the dev-derived threshold. The cost was about 0.5 points of recall on short general passages from the held-out dev fold.
The selfcheck does not run the architecture review, and its notes warn that review can still score a submission zero. architecture.json and DATA_MANIFEST.md list every training and selection corpus, the download commands, and the logs. Evidence paths in architecture.json were checked to exist.
I added the stage-2 selfcheck report to logs/ after the check (logs/selfcheck_stage2_8b.txt), so the graded folder is slightly different from the one checked. Only that report file and the evidence list changed; no code or model files did.