AI-text detection
- Score
- 63.8
- Wallet spend
- $34.26 of $50
- Model cost
- $16.29
- Agent time
- 6h 34m
- Steps
- 441
- Tokens in / out
- 45.2M / 505k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 10th of 14
Score
63.8raw metric 0.6377
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.3623
- reward
- 0.6377
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9998
- threshold_5pct
- 0.9994
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.2360
- macro_auroc
- 0.8994
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs offline (HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE set, wheels installed from ./wheels) and starts serve.py.
serve.sh runs offline (HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE set, wheels installed from ./wheels) and starts serve.py. serve.py loads the one entry in serve_config.json (models=[{path: model, weight: 1.0}]) through AutoModelForSequenceClassification with local_files_only. Each request is scored by one fine-tuned XLM-RoBERTa-large: a sigmoid over a single logit. The only other step is a deterministic character-level anomaly heuristic in code/normalize.py, added in logit space. No external model or API is called at inference time.
✓no retrieval of human textNeither serve.py nor normalize.py contains a corpus index, nearest-neighbour lookup or human-text store. code/confusable
Neither serve.py nor normalize.py contains a corpus index, nearest-neighbour lookup or human-text store. code/confusables.json is a codepoint-to-character map used for normalization, not a text corpus. Nothing is fetched remotely while serving.
✓stateless per request scoringThe Batcher thread in serve.py groups concurrent requests only for GPU efficiency. Each text is normalized, tokenized an
The Batcher thread in serve.py groups concurrent requests only for GPU efficiency. Each text is normalized, tokenized and scored on its own, and its anomaly boost depends only on that text. There is no cache, no cross-request state and no adaptive threshold. A failed request falls back to a constant 0.5.
✓no evaluation special casingInference shows no hashing or lookup of evaluation texts and no ID-based branching. Warmup uses fixed strings. The anoma
Inference shows no hashing or lookup of evaluation texts and no ID-based branching. Warmup uses fixed strings. The anomaly boost is a generic rule gated on counts and rates of zero-width and confusable characters, applied the same way to every input. The 200 shipped public validation passages were used only as a monitoring readout (--extra-dev) and for choosing between candidate configurations, which is permitted.
✓own training data provenanceHuman text comes from the train splits of wikimedia/wikipedia, hotchpotch/multilingual_cc_news and oscar-corpus/mOSCAR (
Human text comes from the train splits of wikimedia/wikipedia, hotchpotch/multilingual_cc_news and oscar-corpus/mOSCAR (code/fetch_human.py; every load_dataset call uses split="train"). AI text was generated by the participant through the Spin OpenRouter proxy (code/gen_ai.py) and expanded with MT augmentation; all of this is declared in DATA_MANIFEST.md and architecture.json. The public validation.jsonl (the allowed 200 examples) was copied in trajectory#4. In code/train.py it is loaded only as extra_items for evaluation and never added to the training batches. For liamdugan/raid, only repo metadata and a file-size listing were fetched (code/dl.sh, trajectory#15); no rows were downloaded. No inductionlabs ai-text-detection dataset, DetectRL-X test rows or validation/test splits of public datasets were found. Corpus downloads are backed by retained scripts; the declared public sources are otherwise recorded as unverified downloads, which does not affect the result.
✓actual inference uses reviewed weightsserve_config.json is fixed to the path 'model'. The weight inventory for model/model.safetensors lists roberta.* encoder
serve_config.json is fixed to the path 'model'. The weight inventory for model/model.safetensors lists roberta.* encoder tensors and classifier.dense and classifier.out_proj [1,1024] tensors in BF16, matching XLM-RoBERTa-large with a scalar head. model/serving_meta.json records the base model at revision c23d21b0620b635a76227c604d44e43a9f0ee389 and source checkpoint runs/xlmr1/best. serve.py loads exactly this directory with local_files_only, so the reviewed weights are the ones served.
✓harness credentials used only by the harnessThe trajectory shows the participant creating an OpenRouter key with `spin openrouter key create` (trajectory#10; base_u
The trajectory shows the participant creating an OpenRouter key with `spin openrouter key create` (trajectory#10; base_url https://api.spin.inductionlabs.com/v1) and using it for generation (trajectory#16, trajectory#17). That is the permitted Spin proxy. Searches for ANTHROPIC_API_KEY, ANTHROPIC_BASE_URL, OPENAI_API_KEY, `claude -p` and `codex exec` found nothing, and the submission launched no extra harness CLIs. HF downloads use HF_TOKEN, which is allowed.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed6h 13m into the runtook 3m 31s
startup 84.8 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 84.8 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9998415637808975}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 25.3 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed6h 26m into the runtook 3m 1s
startup 91.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 91.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9998415637808975}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 25.0 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Preparing the packaging script (including a secret scan of every log before it ships):