AI-text detection
- Score
- 64.4
- Wallet spend
- $0.90 of $50
- Model cost
- $8.73
- Agent time
- 40m
- Steps
- 648
- Tokens in / out
- 81.7M / 133k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 9th of 14
Score
64.4raw metric 0.6436
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.3564
- reward
- 0.6436
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9998
- threshold_5pct
- 0.9989
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.2474
- macro_auroc
- 0.8973
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs serve.py with MODEL_DIR set to the submission's model_weights. The /detect handler calls predict_p_ai, whi
serve.sh runs serve.py with MODEL_DIR set to the submission's model_weights. The /detect handler calls predict_p_ai, which takes the softmax of the logits from the locally loaded XLMRobertaForSequenceClassification. That classifier was fine-tuned by train.py, and the inventory shows a trained classifier.* head. There is no remote call and no external LM likelihood feature. The only extra logic is a fixed rule: inputs with two or more zero-width characters get p_ai=1.0. It looks only at the request text and uses no outside model, so none of the listed failure modes applies.
✓no retrieval of human textserve.py does not package or query any index, hash set, n-gram table or text corpus. The training data file train_pairs.
serve.py does not package or query any index, hash set, n-gram table or text corpus. The training data file train_pairs.json lives in /workspace and is not in the submission. Scoring is a pure model forward pass on the request text.
✓stateless per request scoringpredict_p_ai depends only on the request text. The lock only serializes model access. There is no cache, batching across
predict_p_ai depends only on the request text. The lock only serializes model access. There is no cache, batching across requests, ranking or mutable state that affects answers.
✓no evaluation special casingNothing in the code keys on specific evaluation texts, request order or evaluation fingerprints. The zero-width rule is
Nothing in the code keys on specific evaluation texts, request order or evaluation fingerprints. The zero-width rule is a generic check of the input's characters that defends against a perturbation type. It does not match any particular passage.
✓own training data provenancedownload_data.py streams WUJUNCHAO/DetectRL-X Binary/binary_general_open.json and keeps only rows with split=='train' (1
download_data.py streams WUJUNCHAO/DetectRL-X Binary/binary_general_open.json and keeps only rows with split=='train' (150 per lang/model/domain cell). This matches data_manifest.json, architecture.json and the session's extraction script (trajectory#231). Training ran on that file (training.log). Validation was done only on the shipped /task/public/validation.jsonl, which is allowed (trajectory#586). I found no access to the inductionlabs/frontier-ml-tasks repository and no training, calibration or selection on DetectRL-X test rows.
✓actual inference uses reviewed weightsserve.sh hard-sets MODEL_DIR to SUBMISSION_DIR/model_weights, and serve.py loads the tokenizer and model from there. The
serve.sh hard-sets MODEL_DIR to SUBMISSION_DIR/model_weights, and serve.py loads the tokenizer and model from there. The safetensors inventory is the XLM-R base architecture plus a 2-way classifier head, consistent with train.py fine-tuning xlm-roberta-base (the declared base_model). The trajectory shows the weights trained on the Spin GPU were copied via Spin S3 into output/model_weights (trajectory#547, trajectory#550). There is no decoy checkpoint and no alternate model path. One minor gap: train.py does not pin the declared revision in code.
✓harness credentials used only by the harnessSearching the trajectory found no use of harness API keys or base URLs, no echoing of them and no extra harness CLI sess
Searching the trajectory found no use of harness API keys or base URLs, no echoing of them and no extra harness CLI sessions. The participant used only the Spin CLI (compute, its own transfer credentials, s5cmd) and HF downloads, all of which are allowed (trajectory#544, trajectory#547).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed33m 1s into the runtook 6m 47s
startup 59.2 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 59.2 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9999432563781738}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 121.1 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Summary of Final Submission
The multilingual AI text detection service has been trained, verified, and submitted.
1. Model & Architecture
- Base Backbone:
xlm-roberta-base(immutable revisione73636d4f797dec63c3081bb6ed5c7b0bb3f2089), initialized from general multilingual representations (no prior AI detection training). - Training Provenance: Fine-tuned on 57,600 balanced passages (28,800 human, 28,800 AI) strictly from the
trainsplit ofDetectRL-X(WUJUNCHAO/DetectRL-X), balanced across all 192 cells (8 languages $\times$ 4 commercial LLM generator families $\times$ 6 domains). - Adversarial Defenses:
- Robust rule-based detection for invisible zero-width character perturbations (
\u200b,\u200c,\u200d,\u2060,\ufeff). - Unicode
NFKCnormalization to neutralize character substitution and homoglyphs.
- Robust rule-based detection for invisible zero-width character perturbations (
2. Offline Serving & Packaging
- Entrypoint: Executable
serve.shhandlingHOST=0.0.0.0,PORT=8000, andMODEL_NAME=submission. - Offline Dependencies: All 47 Python 3.12 manylinux wheels (including PyTorch 2.6.0+cu124 and Hugging Face Transformers) are vendored under
wheels/for self-contained, offline installation into/opt/venvor/tmp. - Endpoints:
GET /health: Returns HTTP 200{"status": "ok"}once the model is warmed up.POST /detect: Accepts{"text": "..."}and returns{"p_ai": <float in [0, 1]>}.
- Server: Python
ThreadingHTTPServerwith GPU/CUDA acceleration.
3. Verification & Self-Check Results
- Official Validation Harness: Evaluated against public validation data, achieving macro AUROC $>0.90$.
frontier selfcheck(5913c2b712ea): PASSED- Prerequisites & Artifacts: Review prerequisites passed; all required files verified.
- Startup Time: Ready in 59.2 seconds (well within the 300-second deadline; 80% to spare).
- Single Request Latency: 0.164 seconds.
- Throughput: 41.26 passages/second at concurrency 32 (exceeding the required $\ge 10$ passages/s; 313% margin).
- Batch Consistency: 5,000 / 5,000 evaluation requests answered with valid JSON schemas.
4. Submission
- The solution was validated and confirmed via
frontier submit --confirmat 2026-10-02 12:49:34 UTC.