AI-text detection
- Score
- 36.5
- Wallet spend
- $41.88 of $50
- Model cost
- $5.62
- Agent time
- 10h 25m
- Steps
- 358
- Tokens in / out
- 44.6M / 85k
- Self-checks
- 1 passed of 2 used
- Rank at $50 wallet
- 18th of 19
Score
36.5raw metric 0.3652
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.6348
- reward
- 0.3652
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 1.0000
- threshold_5pct
- 0.9996
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.4632
- macro_auroc
- 0.8372
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh runs `uvicorn inference:app` and nothing else. At startup, inference.py loads one AutoModelForSequenceClassific
serve.sh runs `uvicorn inference:app` and nothing else. At startup, inference.py loads one AutoModelForSequenceClassification from the local model/ directory. /detect returns softmax[:,1] of that one model as p_ai. There is no ensemble, no external API and no second scorer.
✓no retrieval of human textinference.py makes no network calls and loads no reference corpus, index or lookup table. Each request is just tokenized
inference.py makes no network calls and loads no reference corpus, index or lookup table. Each request is just tokenized and passed through the classifier. The serving install is offline: pip uses --no-index with the vendored wheels.
✓stateless per request scoringEach /detect call tokenizes only that request's text and runs one forward pass under inference_mode. No state is cached
Each /detect call tokenizes only that request's text and runs one forward pass under inference_mode. No state is cached or carried between requests. The global variables hold only the model, the tokenizer and the device.
✓no evaluation special casinginference.py has no hash lookups, ID or text matching, or hard-coded outputs. The score is always the model softmax clam
inference.py has no hash lookups, ID or text matching, or hard-coded outputs. The score is always the model softmax clamped to [0,1]. Appending the 200 public validation examples to training (trajectory#283) is explicitly allowed. They are learned weights, not lookup rules.
✓own training data provenancedata_manifest.json declares every corpus. The trajectory shows these splits were used: SemEval subtaskA_multilingual tra
data_manifest.json declares every corpus. The trajectory shows these splits were used: SemEval subtaskA_multilingual train split only; HC3 all.jsonl and HC3-ru train; HC3-French train_full_examples only, not its test or val files; KFUPM Arabic; kanwal; Wikipedia train; Qwen2.5-7B generated locally; news corpora; train streams of xsum, amazon and gutenberg, plus a c4 shard; and the allowed /task/public/validation.jsonl. RAID and iis test files appear only in directory listings and were never used for training. There is no access to inductionlabs/frontier-ml-tasks or DetectRL-X. Searches found no matches, and checkpoint selection used Trainer's own 5% split of the training data. The kanwal split-filtering code is clipped in the trajectory, so no violation is established there. The shipped training scripts are absent, and some download commands are recorded only in the trajectory, so parts of the provenance are unverified rather than violations.
✓actual inference uses reviewed weightsinference.py defaults MODEL_DIR to BASE_DIR/model, and serve.sh does not override it. The model/model.safetensors invent
inference.py defaults MODEL_DIR to BASE_DIR/model, and serve.sh does not override it. The model/model.safetensors inventory shows an XLM-R large encoder with a classifier.dense head and a 2-way classifier.out_proj, 2239618672 bytes. That matches the final checkpoint from a fresh fine-tune of FacebookAI/xlm-roberta-large: base downloaded at trajectory#240, training restarted from the base at trajectory#314, final dir saved at trajectory#323. The checkpoint was copied from /tmp/model_out/final into the output model/ at trajectory#327-328. The base revision sha was checked at trajectory#335. The self-check at trajectory#353 shows the server serving from this model.
✓harness credentials used only by the harnessThe trajectory shows only HF_TOKEN in use, a read-only download credential, including on Spin machines. It also shows Sp
The trajectory shows only HF_TOKEN in use, a read-only download credential, including on Spin machines. It also shows Spin CLI and proxy operations, including Spin proxy key creation at trajectory#42, and Spin-issued S3 transfer credentials for moving the model at trajectory#326-328. Searches found no use, copying or echo of ANTHROPIC, OPENAI, OPENROUTER or GEMINI keys or harness base URLs, and no extra harness CLIs were launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed10h 7m into the runtook 7m 5s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: (empty)
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: (empty)
- 2passed10h 18m into the runtook 6m 12s
startup 112.5 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 112.5 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9999843835830688}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.4 s for the first request, then 107.0 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
Submission complete.