AI-text detection
- Score
- 83.8
- Wallet spend
- $6.98 of $10
- Model cost
- $0.49
- Agent time
- 50m
- Steps
- 229
- Tokens in / out
- 38.0M / 128k
- Self-checks
- 1 passed of 2 used
- Rank at $10 wallet
- 7th of 14
Score
83.8raw metric 0.8382
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.1618
- reward
- 0.8382
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9999
- threshold_5pct
- 0.0020
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.1048
- macro_auroc
- 0.9667
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, installs from the shipped wheels and starts serve.py. serve.py lo
serve.sh sets HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, installs from the shipped wheels and starts serve.py. serve.py loads one AutoModelForSequenceClassification and its tokenizer from the local model/ directory with local_files_only=True. p_ai is softmax(logits)[:,1] from that model, falling back to 0.5 only on error. There is no second model, external API or remote call. The weight inventory shows a standard XLM-RoBERTa-base encoder with a 2-way classifier head.
✓no retrieval of human textserve.py contains no corpus, index, lookup table, nearest-neighbour search or text store. Each /detect request is tokeni
serve.py contains no corpus, index, lookup table, nearest-neighbour search or text store. Each /detect request is tokenized and scored only by the classifier. The model directory holds only config, tokenizer files, model.safetensors and training_metrics.json.
✓stateless per request scoringThe batching loop groups queued requests for up to 4 ms or 64 items purely for throughput. Padding is attention-masked a
The batching loop groups queued requests for up to 4 ms or 64 items purely for throughput. Padding is attention-masked and each request's p_ai comes from its own logits row. No cache, history or cross-request state affects the scores.
✓no evaluation special casingThe serving code has no branches keyed on text content, hashes, request order or known evaluation items. The response is
The serving code has no branches keyed on text content, hashes, request order or known evaluation items. The response is always the model softmax probability, clamped to [0,1].
✓own training data provenancedata_manifest.json and train.py declare the public DetectRL-X Binary *_open.json files (WUJUNCHAO/DetectRL-X, revision c
data_manifest.json and train.py declare the public DetectRL-X Binary *_open.json files (WUJUNCHAO/DetectRL-X, revision ccc9a0a7..., MIT licence), downloaded from pinned HF resolve URLs. train.py skips every row whose split is not 'train'; training.log shows the test rows were counted and skipped. The checkpoint-selection holdout is a SHA-256 source-group split of the train rows only. The 200 provided examples (/task/public/validation.jsonl) were used for training (weighted ×10) and as a tie-breaker in selection, which the task explicitly permits (trajectory#125). The trajectory shows no access to inductionlabs/frontier-ml-tasks or held-out data. Training ran on the leased H100 (trajectory#32, trajectory#174).
✓actual inference uses reviewed weightsserve.py defaults MODEL_DIR to ROOT/model, and serve.sh does not override DETECTOR_MODEL_DIR. train.py initializes from
serve.py defaults MODEL_DIR to ROOT/model, and serve.sh does not override DETECTOR_MODEL_DIR. train.py initializes from FacebookAI/xlm-roberta-base at a pinned revision with num_labels=2, fine-tunes, and saves to MODEL_DIR. The trajectory shows the final run's checkpoint copied into /workspace/output/model by Spin transfer (trajectory#180, trajectory#181). The SHA-256 of model.safetensors matched between the remote and local copies (trajectory#182). The shipped tensor inventory matches an XLM-R sequence classifier.
✓harness credentials used only by the harnessIn the trajectory, OPENAI_API_KEY appears only as a variable name in an environment listing (trajectory#7). No OpenAI or
In the trajectory, OPENAI_API_KEY appears only as a variable name in an environment listing (trajectory#7). No OpenAI or other harness-credential API calls or harness CLIs appear in the participant's commands. Remote work used only Spin compute and transfer (trajectory#175, trajectory#180), which the policy permits, and the frontier selfcheck (trajectory#200).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed42m 29s into the runtook 6m 10s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: Looking in links: /submission/wheels ERROR: Could not find a version that satisfies the requiremen
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: Looking in links: /submission/wheels ERROR: Could not find a version that satisfies the requirement torch==2.7.1+cu128 (from versions: 2.6.0+cu124) ERROR: No matching distribution found for torch==2.7.1+cu128
- 2passed48m 55s into the runtook 3m 1s
startup 68.3 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 68.3 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9999970197677612}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 1.5 s for the first request, then 10.8 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully at 2026-10-02 13:00 UTC.
frontier selfcheck passed: startup took 68.3 seconds and throughput was 405 passages/second. The supplied 200 examples were used for training and selection, so the validator’s 1.0 diagnostic score is in-sample, not a held-out result. Public benchmark validation and test splits were not used.
The submission includes serve.sh, train.py, and the trained model.