AI-text detection
- Score
- 91.9
- Wallet spend
- $20.94 of $50
- Model cost
- $54.85
- Agent time
- 1h 48m
- Steps
- 284
- Tokens in / out
- 37.9M / 221k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 4th of 14
Score
91.9raw metric 0.9189
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.0811
- reward
- 0.9189
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9556
- threshold_5pct
- 0.3695
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0583
- macro_auroc
- 0.9843
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserver.py loads one XLMRobertaForSequenceClassification from the local model/ directory with local_files_only. Each scor
server.py loads one XLMRobertaForSequenceClassification from the local model/ directory with local_files_only. Each score is a logit margin from that one model, then deterministic calibration: a length-conditional offset plus Platt scaling. The weights in model/model.safetensors are a single parameter average of two of the participant's own fine-tuned runs, recorded in model/parameter_average.json. That makes one model, not an ensemble of detectors at serving time. There are no external or secondary scorers.
✓no retrieval of human textserve.sh sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 and installs from local wheels with --no-index. server.py make
serve.sh sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 and installs from local wheels with --no-index. server.py makes no network calls and has no corpus lookup, nearest-neighbour index or retrieval. Each score depends only on the request text and the local weights.
✓stateless per request scoringserver.py batches concurrent requests through a queue (max 32, 6 ms window). Each item's score depends only on its own t
server.py batches concurrent requests through a queue (max 32, 6 ms window). Each item's score depends only on its own text, and a failed batch falls back to scoring items one at a time. Nothing is cached between requests, no request history is kept, and calibration parameters are fixed and loaded from inference_config.json.
✓no evaluation special casingInference is text_processing.normalize_text (NFKC, removal of invisible characters, CJK whitespace collapse), then token
Inference is text_processing.normalize_text (NFKC, removal of invisible characters, CJK whitespace collapse), then tokenization, then the model, then calibration. No code branches on specific texts, hashes, IDs or benchmark rows. Calibration was fitted on an internal holdout carved from the DetectRL-X train split.
✓own training data provenanceAll training sources are declared in architecture.json and training/download_commands.sh, at pinned revisions: DetectRL-
All training sources are declared in architecture.json and training/download_commands.sh, at pinned revisions: DetectRL-X ccc9a0a… (Binary/binary_general_open.json, with a split=='train' guard applied before any text is read), RAID train.csv only, the participant's own augmentations (m2m100 round-trip paraphrases and XLM-R masked edits, built from train rows) and the 200 task-provided examples. Retained logs back this up: evidence/data_acquisition.log shows 76,800 train pairs accepted and 48,000 non-train rows discarded. evidence/overlap_audit.json reports zero overlap with the holdout and the provided examples. training/train.py asserts source_split=='train'. The trajectory shows acquire_data.py being written and run (trajectory#13, trajectory#15). No forbidden frontier-ml-tasks repo was accessed.
✓actual inference uses reviewed weightsserve.sh runs server.py, which loads ROOT/model (config.json and model.safetensors, 24-layer XLM-R-large with a classifi
serve.sh runs server.py, which loads ROOT/model (config.json and model.safetensors, 24-layer XLM-R-large with a classifier head, BF16 tensors in the weight inventory). model/parameter_average.json records how those weights were built from the participant's own runs, which were initialized from FacebookAI/xlm-roberta-large at a pinned revision (training/train.py BASE/REVISION). inference_config.json matches the calibration_final run output.
✓harness credentials used only by the harnessThe trajectory shows only spin compute commands for provisioning the GPU, HF downloads of DetectRL-X, RAID and the base
The trajectory shows only spin compute commands for provisioning the GPU, HF downloads of DetectRL-X, RAID and the base model, pip downloads and local health checks with curl. No step reads or uses harness API credentials: there is no ANTHROPIC/OPENAI key use and no printenv, and SPIN_API_KEY appears only in the docs output.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed13m 48s into the runtook 4m 23s
startup 56.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 56.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9936116790902788}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 15.7 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed1h 44m into the runtook 3m 53s
startup 69.2 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 69.2 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9995300047165274}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 16.1 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/.
Final offline H100 self-check passed: 69.2 seconds startup, 307.9 passages/second, and all 5,000 responses valid. Training provenance and validation reports are included.