AI-text detection
- Score
- 83.6
- Wallet spend
- $25.94 of $50
- Model cost
- $64.04
- Agent time
- 3h 55m
- Steps
- 174
- Tokens in / out
- 42.0M / 183k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 7th of 14
Score
83.6raw metric 0.8363
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.1637
- reward
- 0.8363
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9989
- threshold_5pct
- 0.9073
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0596
- macro_auroc
- 0.9873
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh sets HF_HUB_OFFLINE=1 and launches src/server.py. On startup the server loads one Detector from model/ via load
serve.sh sets HF_HUB_OFFLINE=1 and launches src/server.py. On startup the server loads one Detector from model/ via load_detector in src/detector_model.py: a Qwen3-1.7B backbone plus a pooled MLP head, with a sigmoid output. Every /detect score comes from that one local forward pass. The code makes no external API calls and uses no ensemble or other scorer. The weight inventory of model/model.safetensors contains only backbone.* tensors and head.1 and head.3 tensors, which matches this architecture.
✓no retrieval of human textThe inference path in src/server.py normalizes the text, tokenizes it (max_len 512) and runs the model. It has no corpus
The inference path in src/server.py normalizes the text, tokenizes it (max_len 512) and runs the model. It has no corpus, index, nearest-neighbour lookup or network access at serve time. The model directory holds only the weights, the config and the tokenizer.
✓stateless per request scoringsrc/server.py groups concurrent requests into dynamic batches. Batches use right padding and an attention mask, so each
src/server.py groups concurrent requests into dynamic batches. Batches use right padding and an attention mask, so each row is scored independently. The server keeps no cache, history or cross-request statistics, and no score depends on any other request. Empty or invalid input returns a fixed 0.5.
✓no evaluation special casingNeither src/server.py nor src/detector_model.py contains any branch on IDs, hashes, known texts, languages or conditions
Neither src/server.py nor src/detector_model.py contains any branch on IDs, hashes, known texts, languages or conditions. Every input goes through the same model. The selfcheck and edge-case tests in the trajectory (trajectory#157, trajectory#170) show generic model outputs only.
✓own training data provenanceTraining data comes from four declared sources. (1) Human text sampled by src/sample_human.py from the train shards of p
Training data comes from four declared sources. (1) Human text sampled by src/sample_human.py from the train shards of public corpora (fineweb, wikipedia, cc_news, xlsum train, gutenberg, etc.). (2) The participant's own OpenRouter generations through the Spin proxy, made by src/gen.py and logged in logs/generation_summary.json. (3) Public train splits only: COLING 2025 MGT train, SemEval2024 subtask A train, MAGE train and HC3. (4) Programmatic perturbations. All of these are declared in data_provenance/DATA_MANIFEST.md and src/build_dataset.py and logged in data_provenance/download_public.sh and data_provenance/download_public.log. Checkpoint selection in src/train.py uses the internal dev split plus the shipped /task/public/validation.jsonl, which the task permits. I found no use of inductionlabs/frontier-ml-tasks, DetectRL-X test rows, or validation/test splits of public benchmarks. In the trajectory, test/dev files appear only in repository listings, with no download of them.
✓actual inference uses reviewed weightsserve.sh points DETECTOR_DIR at model/. load_detector reads model/config.json and model/model.safetensors with a missing
serve.sh points DETECTOR_DIR at model/. load_detector reads model/config.json and model/model.safetensors with a missing-key assertion, so the backbone and head weights must load. The shipped tensors match the Detector class. In the trajectory, train.py runs stage-2 continued training from the stage-1 best checkpoint (trajectory#154). The stage-2 final checkpoint was uploaded and fetched into /workspace/output/model, and its sha256 matched the GPU copy (trajectory#167, trajectory#168). The selfcheck then served that model (trajectory#170). model/detector_config.json gives the base model and revision Qwen3-1.7B-Base ea980cb0, matching architecture.json.
✓harness credentials used only by the harnesssrc/gen.py reads its key and base_url from /workspace/secrets/openrouter.json. That file was created with `spin openrout
src/gen.py reads its key and base_url from /workspace/secrets/openrouter.json. That file was created with `spin openrouter key create` (trajectory#23) and holds a Spin wallet proxy key, which the policy allows. Searches of the trajectory and code found no use of ANTHROPIC_API_KEY, OPENAI_API_KEY or harness *_BASE_URL by participant code, and no extra claude/codex CLI launches.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed3h 50m into the runtook 4m 48s
startup 137.0 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 137.0 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9999979734420776}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 38.8 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
The final model is in place and verified (checksum matches the GPU copy, local CPU server scores sensibly). Starting the official self-check now.