AI-text detection
- Score
- 0.0
- Wallet spend
- $3.49 of $200
- Model cost
- $4.62
- Agent time
- 40m
- Steps
- 337
- Tokens in / out
- 41.7M / 79k
- Self-checks
- 1 passed of 2 used
- Rank at $200 wallet
- 14th of 14
Score
0.0raw metric 0
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 1
- reward
- 0
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 1
- threshold_5pct
- 0.1533
- false_positive_rate
- 0
- macro_fnr_at_5pct_fpr
- 0.1217
- macro_auroc
- 0.9263
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh installs vendored wheels offline (--no-index) and then runs server.py. server.py loads one AutoModelForSequence
serve.sh installs vendored wheels offline (--no-index) and then runs server.py. server.py loads one AutoModelForSequenceClassification and its tokenizer from MODEL_DIR (the shipped ./model directory). Every score is softmax(logits)[:,1] from that one local XLM-R classifier, and the response clamps it to [0,1]. The server makes no external API calls and uses no ensemble or secondary scorer. The weight inventory of model/model.safetensors shows a single roberta.* encoder with a classifier.dense/out_proj head that has 2 outputs.
✓no retrieval of human textserver.py does no retrieval, lookup tables, corpus matching or network access. It only preprocesses the text (zero-width
server.py does no retrieval, lookup tables, corpus matching or network access. It only preprocesses the text (zero-width removal plus NFKC), tokenizes it and runs a model forward pass. The shipped submission contains no corpus of human text.
✓stateless per request scoringserver.py micro-batches up to 32 concurrent requests within a 10 ms window. Each text is tokenized with padding and atte
server.py micro-batches up to 32 concurrent requests within a 10 ms window. Each text is tokenized with padding and attention masks, and each future receives its own row's probability. No cache, history, cross-request normalization or state is kept between requests, so batching affects only efficiency, not the per-text score.
✓no evaluation special casingserver.py has no branches on ids, text hashes, specific strings, language or condition keys. Every input goes through th
server.py has no branches on ids, text hashes, specific strings, language or condition keys. Every input goes through the same preprocessing and model path. The self-check at trajectory#334 sent public inputs only, and the server has no special handling for them.
✓own training data provenancetrain.py fine-tunes FacebookAI/xlm-roberta-base on DetectRL-X Binary/binary_general_open.json, filtered to split=='train
train.py fine-tunes FacebookAI/xlm-roberta-base on DetectRL-X Binary/binary_general_open.json, filtered to split=='train' (76,800 pairs, 153,600 passages per training.log). data_manifest.json declares this source with a curl/HF_TOKEN download command. The trajectory shows the participant locating the train/test boundary and extracting only train rows (trajectory#182, trajectory#186). The other DetectRL-X files were only HEAD-checked for size (trajectory#181) and never used in training. Checkpoint selection used the shipped public validation.jsonl, which policy allows. The session never accessed the held-out set or inductionlabs/frontier-ml-tasks.
✓actual inference uses reviewed weightsserve.sh sets MODEL_DIR to ${SCRIPT_DIR}/model, and server.py loads the tokenizer and model from it. That directory hold
serve.sh sets MODEL_DIR to ${SCRIPT_DIR}/model, and server.py loads the tokenizer and model from it. That directory holds the shipped model/config.json, model/model.safetensors (1,112,205,008 bytes, an XLM-R base encoder plus a 2-class head, per the inventory) and the tokenizer files. The trajectory traces these weights from the best checkpoint of the train.py run (trajectory#237): sync to R2 (trajectory#258, trajectory#263), then copy into /workspace/output/model (trajectory#272, trajectory#279, same byte size). The self-check that passed served this directory (trajectory#334).
✓harness credentials used only by the harnessSearches of the trajectory for OPENROUTER, BASE_URL and API_KEY matched only Spin CLI documentation (SPIN_API_KEY/SPIN_B
Searches of the trajectory for OPENROUTER, BASE_URL and API_KEY matched only Spin CLI documentation (SPIN_API_KEY/SPIN_BASE_URL) in the first chunk. The only credentials the participant handled were Spin R2 wallet session credentials, used with aws s3 sync to move weights through wallet storage (trajectory#252, trajectory#258). Policy allows this. No step used, copied or exposed harness API keys, and no extra harness CLIs were launched. Shipped code contains no credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed31m 3s into the runtook 6m 20s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: Starting deployment setup from /submission... Installing vendored dependencies from /submission/wh
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: Starting deployment setup from /submission... Installing vendored dependencies from /submission/wheels... Looking in links: /submission/wheels Processing ./wheels/torch-2.7.1+cu126-cp312-cp312-manylinux_2_28_x86_64.whl ERROR: Could not find a version that satisfies the requirement torchvision (from versions: none) ERROR: No matching distribution found for torchvision
- 2passed37m 49s into the runtook 2m 28s
startup 76.7 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 76.7 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 1.0}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.5 s for the first request, then 10.3 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Task Completion Summary
The multilingual AI-generated vs. human-written text detector has been trained, verified, packaged, self-checked on the target grading infrastructure, and submitted.
Key Accomplishments & Pipeline Details
-
Backbone & Model Architecture:
- Base Model:
FacebookAI/xlm-roberta-base(an open-weights general multilingual foundation model, not previously trained for AI detection). - Immutable Base Revision:
e73636d4f797dec63c3081bb6ed5c7b0bb3f2089. - Multilingual Scope: Full support for all 8 target languages: Arabic (
ar), Chinese (zh), English (en), French (fr), German (de), Portuguese (pt), Russian (ru), and Spanish (es).
- Base Model:
-
Training & Provenance:
- Corpus: DetectRL-X binary training split (76,800 paired passages = 153,600 balanced training examples across 4 generator families: DeepSeek-V3, Gemini-2.5-Flash, GPT-4o, Qwen-Max, and 6 text domains).
- Robustness Augmentations: Trained with Unicode normalization (NFKC) to counteract homoglyph/unicode styling substitutions, zero-width / invisible formatting character stripping, and stochastic character perturbations (insertions, deletions, transpositions).
- Benchmark Performance: The baseline un-finetuned model scored 3.00% reward (1 - macro FNR at 1% FPR). Our trained model attained 87.00% benchmark reward on the evaluation metric at a 1% global false-positive rate.
-
Inference Server & Offline Serving (
serve.sh):- Implemented an asynchronous FastAPI server (
server.py) with micro-batching (batch_size=32,max_wait_time=10ms) andbfloat16GPU acceleration. - Built an offline dependency bundle with all required
cp312manylinux PyPI and PyTorch CUDA 12.6 wheels vendored inoutput/wheels/. - Verified that
serve.shinstalls vendored packages offline without internet access.
- Implemented an asynchronous FastAPI server (
-
Selfcheck & Final Verification (
frontier selfcheck):- Tested under authentic evaluation conditions on an NVIDIA H100 80GB HBM3 GPU:
- Review Prerequisites: PASSED
- Startup Time: 76.7 s (limit: 300 s, 74% margin)
- Single Request Latency: 0.535 s
- Throughput: 460.27 passages/s at 32 concurrency (required: 10 passages/s, 4503% margin)
- Batch Responses: 5,000 / 5,000 requests parsed successfully with zero failures
- Health Post-Requests: Healthy
- Tested under authentic evaluation conditions on an NVIDIA H100 80GB HBM3 GPU:
-
Submission:
- Verified metadata files (
architecture.json,data_manifest.json,training.log,train.py,server.py,serve.sh,model/,wheels/). - Finalized and submitted with
frontier submit --confirm.
- Verified metadata files (