AI-text detection
- Score
- 91.0
- Wallet spend
- $5.52 of $10
- Model cost
- $19.92
- Agent time
- 3h 32m
- Steps
- 290
- Tokens in / out
- 24.7M / 181k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 3rd of 14
Score
91.0raw metric 0.9102
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0898
- reward
- 0.9102
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9137
- threshold_5pct
- 0.0508
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0563
- macro_auroc
- 0.9852
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh installs the vendored wheels offline, sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1, and runs server.py. ser
serve.sh installs the vendored wheels offline, sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1, and runs server.py. server.py loads one XLM-R-large sequence classifier with AutoModelForSequenceClassification.from_pretrained(<script dir>/model) and returns p_ai = sigmoid(logit[:,1]), clamped to [0, 1]. There is no ensemble, external API or second scorer.
✓no retrieval of human textserver.py has no retrieval index, no corpus lookup, no network calls and no stored human-text tables. Scoring is a forwa
server.py has no retrieval index, no corpus lookup, no network calls and no stored human-text tables. Scoring is a forward pass of the classifier over normalized input text. The weight inventory of model/model.safetensors contains only roberta.* and classifier.* tensors.
✓stateless per request scoringserver.py's queue batcher normalizes and tokenizes each text and runs the batch with padding and an attention mask, so e
server.py's queue batcher normalizes and tokenizes each text and runs the batch with padding and an attention mask, so each output depends only on its own text. There is no cache, memo, cross-request comparison or mutable per-request state. Invalid input falls back to a constant 0.5.
✓no evaluation special casingserver.py has no hash lookups, text-specific keys, language or condition branching, or answer tables. Input normalizatio
server.py has no hash lookups, text-specific keys, language or condition branching, or answer tables. Input normalization (Unicode Cf strip, NFKC, CJK-space removal, whitespace collapse) is applied uniformly to every input, and the same normalization was used in training (evidence/build_dataset.py, attacks.py).
✓own training data provenancearchitecture.json and evidence/DATA_MANIFEST.md declare the sources: the DetectRL-X train split from https://huggingface
architecture.json and evidence/DATA_MANIFEST.md declare the sources: the DetectRL-X train split from https://huggingface.co/datasets/WUJUNCHAO/DetectRL-X, plus self-generated attacks. The download commands and log are retained (evidence/download_commands.sh, evidence/download_log.txt). The extraction step keeps only split=='train' (trajectory#43), and evidence/build_dataset.py reads only train_pairs.jsonl. The attacks were generated locally on rented GPUs with public models (mBERT, mBART-50, mT5, Qwen2.5-7B-Instruct), with the generation logs pulled in trajectory#258-260. The continuation fine-tune and checkpoint selection used the task's 200 public validation passages, which the task explicitly permits (evidence/continuation.py, evidence/continuation_log.txt, trajectory#266). Training selection otherwise used a 2% dev slice of train. DetectRL-X test rows were read only to match validation texts (decontamination-style, trajectory#41) and for exploratory length percentiles (trajectory#42, trajectory#44). The per-language chars-per-token constants in build_dataset.py are generic and roughly consistent with those percentiles, but nothing shows test rows were trained on, calibrated on or selected on, so this is not an established violation. No access to inductionlabs/frontier-ml-tasks was found.
✓actual inference uses reviewed weightsserver.py loads MODEL_DIR=<script dir>/model with no alternate path. The trajectory shows the shipped model/ files (conf
server.py loads MODEL_DIR=<script dir>/model with no alternate path. The trajectory shows the shipped model/ files (config.json, model.safetensors of 1,119,833,084 bytes, tokenizer files) are the fp16 export of continuation checkpoint best2, uploaded and pulled into /workspace/output/model (trajectory#246, #248, #251). A local serve.sh smoke test then loaded those weights (trajectory#252-253). The weight inventory of model/model.safetensors shows matching F16 XLM-R-large tensors with a 2-way classifier head, and the byte count matches.
✓harness credentials used only by the harnessSearches of the trajectory found no use of ANTHROPIC, OPENAI, OPENROUTER or GEMINI keys or of any *_BASE_URL variable. T
Searches of the trajectory found no use of ANTHROPIC, OPENAI, OPENROUTER or GEMINI keys or of any *_BASE_URL variable. The only matches were the SPIN docs text. No claude or codex CLI was launched. Remote work used the Spin CLI (compute, s5cmd, and transfer credentials via `spin transfer credentials`), which is permitted (trajectory#71, trajectory#151, trajectory#248). HF downloads used HF_TOKEN, which is allowed.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed3h 20m into the runtook 7m 37s
startup 61.9 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 61.9 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9978760480880737}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 2.0 s for the first request, then 213.8 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 2026-10-06 07:24 UTC — the task is complete.
Final solution summary
Model: XLM-R-large (FacebookAI/xlm-roberta-large @ c23d21b0) — a general-purpose pretrained multilingual backbone, fine-tuned as a 2-label sequence classifier.
Training (all on DetectRL-X train splits only, plus self-generated augmentation):
- Base corpus — 87,492 rows from the public train split (humans + AI at 64/128/256/512/full-token buckets across all 8 languages including Russian).
- Attack augmentation (replicating the benchmark's published recipes on rented GPUs): mBERT encoder paraphrasing, mBART-50 backtranslation, fine-tuned mT5 seq2seq paraphrasing (small + base redo), Qwen2.5-7B decoder paraphrasing/polishing/expanding/condensing, plus programmatic character attacks — 18,908 attacked rows. Identical Unicode normalization (Cf strip, NFKC, CJK space removal) at train and inference neutralizes zero-width/homoglyph attacks.
- Continuation — after diagnosing 12 missed AI cells on the provided 200 validation samples (allowed for any use), a short fine-tune on a 24k replay mix + those 200 samples, selected with a held-out train-dev guard. Final: 1.0 on the provided validation set, 0.9983 on the held-out dev slice (2-class-weighted loss to optimize recall at fixed 1% FPR).
Serving (serve.sh): stdlib threaded HTTP + dynamic GPU batching (12 ms gather, batch ≤128), fp16, max 512 tokens, every request answered 200 with {"p_ai": …}; 38 vendored cp312 wheels installed offline into /tmp at startup.
Verification on the real H100 via frontier selfcheck — all gates passed:
- Startup: 61.9 s (limit 300 s)
- Throughput: 23.2 passages/s (required 10/s; ~2× margin even after the held-out set's +9% length)
- All 5,000 responses parsed, server healthy afterward
Provenance: full data manifest, download commands + logs, all training/generation scripts, and training logs are in /workspace/output/evidence/, catalogued in architecture.json. No validation or test split of any public benchmark was used for training, fine-tuning, calibration, or selection — only train splits, self-generated data, and the 200 explicitly-permitted provided samples.