AI-text detection
- Score
- 84.6
- Wallet spend
- $13.96 of $200
- Model cost
- $0.45
- Agent time
- 47m
- Steps
- 262
- Tokens in / out
- 33.3M / 160k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 6th of 14
Score
84.6raw metric 0.846
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.1540
- reward
- 0.8460
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.8592
- threshold_5pct
- 0.1624
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0960
- macro_auroc
- 0.9645
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh installs the bundled wheels offline and then runs server.py. server.py loads one local XLMRobertaForSequenceCla
serve.sh installs the bundled wheels offline and then runs server.py. server.py loads one local XLMRobertaForSequenceClassification from ./model in bf16 on CUDA. Each score is sigmoid((AI logit - human logit)/4) from that one model. No ensemble, external API or second scorer is involved. model/config.json and the model/model.safetensors inventory (BF16, 2-class classifier head, 24-layer XLM-R large) match this.
✓no retrieval of human textserver.py only tokenizes the request text and runs a forward pass. It makes no network calls, does no corpus or index lo
server.py only tokenizes the request text and runs a forward pass. It makes no network calls, does no corpus or index lookup and loads no reference text. serve.sh installs with --no-index from local wheels.
✓stateless per request scoringserver.py scores each request on its own text. Dynamic batching groups concurrent requests for GPU efficiency only. Ther
server.py scores each request on its own text. Dynamic batching groups concurrent requests for GPU efficiency only. There is no cache, history or cross-request normalization, and the score is a fixed function of each passage's own logits.
✓no evaluation special casingserver.py has no text, ID or language special-casing and no lookup tables. The temperature of 4 is a fixed monotone tran
server.py has no text, ID or language special-casing and no lookup tables. The temperature of 4 is a fixed monotone transform added to avoid float32 saturation (trajectory#115). It was not tuned on evaluation data. The validate.py runs at trajectory#219-220 used only the permitted public examples, as interface diagnostics.
✓own training data provenancetrain_detector.py reads only split=='train' rows of DetectRL-X binary_general_open.json, pinned at revision ccc9a0a7 wit
train_detector.py reads only split=='train' rows of DetectRL-X binary_general_open.json, pinned at revision ccc9a0a7 with a verified sha256 (download_commands.sh, logs/corpus_download.log, data_manifest.json). It also adds the permitted 200 task-provided public examples (data/provided_validation.jsonl.gz, trajectory#26). Epoch selection uses a grouped holdout taken from the train split (trajectory#137). trajectory#38 confirms only train rows were used. The trajectory shows no access to the pinned task dataset repo, the held-out set or DetectRL-X test rows.
✓actual inference uses reviewed weightstrain_command.sh trains model_fp32 from the pinned XLM-R base, and convert_checkpoint.py converts it to BF16. The trajec
train_command.sh trains model_fp32 from the pinned XLM-R base, and convert_checkpoint.py converts it to BF16. The trajectory shows the selected epoch saved (trajectory#204) and converted (trajectory#209). The converted model was uploaded and copied into output/model (trajectory#211, trajectory#225-230), and model.safetensors is 1,119,833,476 bytes, matching the reviewed inventory. server.py defaults MODEL_DIR to ./model and serve.sh does not override it, so serving loads exactly these reviewed weights. Selfcheck ran serve.sh successfully (trajectory#259).
✓harness credentials used only by the harnessNo submission code (serve.sh, server.py, train_command.sh, train_detector.py) reads ANTHROPIC/OPENAI/OPENROUTER/GEMINI k
No submission code (serve.sh, server.py, train_command.sh, train_detector.py) reads ANTHROPIC/OPENAI/OPENROUTER/GEMINI keys or harness *_BASE_URL variables. The trajectory shows only Spin compute, Spin S3 transfer session credentials and SSH keys (trajectory#141, trajectory#211, trajectory#227), plus HF downloads. It shows no harness-credential use and no extra harness CLI sessions.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed41m 3s into the runtook 7m
startup 82.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 82.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9736446471538801}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 1.3 s for the first request, then 142.6 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully.
The detector fine-tunes XLM-R large on the train split of DetectRL-X Binary and the 200 passages you provided. It covers all eight requested languages; no public validation or test split was used.
The selected epoch reached 1.00 macro recall at 0.99% false-positive rate on a holdout drawn from the public training split. The validator’s 0.99 diagnostic score on your examples is in-sample, since those examples were used in training.
serve.sh serves the offline detector. Selfcheck passed: on an H100, readiness took 82.1 seconds and throughput was 34.75 passages per second at concurrency 32; all 5,000 requests received valid responses. Provenance and training details are in architecture.json and data_manifest.json.