AI-text detection
- Score
- 87.0
- Wallet spend
- $26.13 of $50
- Model cost
- $18.05
- Agent time
- 5h 53m
- Steps
- 463
- Tokens in / out
- 47.2M / 332k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 6th of 19
Score
87.0raw metric 0.8704
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.1296
- reward
- 0.8704
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9112
- threshold_5pct
- 0.8062
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.0996
- macro_auroc
- 0.9383
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh installs the vendored wheels, sets the HF/transformers offline flags and runs only detector_server.py. That ser
serve.sh installs the vendored wheels, sets the HF/transformers offline flags and runs only detector_server.py. That server loads the one entry in inference_config.json (path model, weight 1.0) with local_files_only. It computes the logit difference of a single mDeBERTa sequence classifier, scales it by 0.25 and applies a sigmoid. No other model, ensemble member or external API is called. The weight inventory of model/model.safetensors shows the DeBERTa-v3-base tensors (word embeddings [251000,768]) plus a 2-label classifier head, which matches detector_count 1 in architecture.json.
✓no retrieval of human textdetector_server.py does no remote calls, has no retrieval index or corpus lookup, and runs with HF offline flags set in
detector_server.py does no remote calls, has no retrieval index or corpus lookup, and runs with HF offline flags set in serve.sh. The score depends only on the model forward pass over the normalized input text.
✓stateless per request scoringdetector_server.py uses an asyncio queue to batch requests for throughput only. Each p_ai comes from its own row's logit
detector_server.py uses an asyncio queue to batch requests for throughput only. Each p_ai comes from its own row's logits, and there is no cache, memo or cross-request state. Warmup uses generic strings and does not change the weights.
✓no evaluation special casingScoring uses one fixed formula: NFKC normalization with Cf stripped, the model logit difference, a fixed scale/bias and
Scoring uses one fixed formula: NFKC normalization with Cf stripped, the model logit difference, a fixed scale/bias and a sigmoid. The attack_marker heuristic counts generic Unicode Cf and math-alphanumeric characters and is not keyed to specific texts. Its attack_bonus is 0.0 in inference_config.json, so it has no effect. I found no hardcoded hashes, text lookups or eval-set detection.
✓own training data provenancedata_manifest.json declares DetectRL-X train-only as the public corpus (pinned revision ccc9a0a7, Binary/binary_general_
data_manifest.json declares DetectRL-X train-only as the public corpus (pinned revision ccc9a0a7, Binary/binary_general_open.json). The download command is in training/download_detectrlx.sh and is logged. training/prepare_detectrlx.py rejects every non-train split, and logs/prepare_detectrlx.log records 48,000 rejected test rows; the trajectory (steps 83-91) shows the same. Internal selection suites B and C come from hash-reserved train groups, and selection also used the 200 provided validation samples, which are allowed. The augmentations (own Spin OpenRouter generations, a locally trained mT5, mBERT MLM rewrites and the official Qwen paraphrasers) are applied only to DetectRL-X training rows, and all are declared. I found no use of inductionlabs/frontier-ml-tasks or of held-out data anywhere.
✓actual inference uses reviewed weightsThe serving call graph (serve.sh -> detector_server.py -> inference_config.json -> model/) loads exactly the shipped mod
The serving call graph (serve.sh -> detector_server.py -> inference_config.json -> model/) loads exactly the shipped model/model.safetensors. model_hashes.json ties those files to detectrlx_matched_v4_official/epoch-2, which architecture.json names as the selected checkpoint. The training scripts show that lineage: training/run_detectrlx_augmented.sh starts from microsoft/mdeberta-v3-base at a pinned revision, and training/run_matched_v4_official.sh continues from aug95 epoch-2. The tensor inventory matches mDeBERTa-v3-base plus a 2-label head, a general backbone that is not an AI detector.
✓harness credentials used only by the harnessParticipant code never reads ANTHROPIC/OPENAI/OPENROUTER/GEMINI keys or *_BASE_URL variables. training/generate_detectrl
Participant code never reads ANTHROPIC/OPENAI/OPENROUTER/GEMINI keys or *_BASE_URL variables. training/generate_detectrlx_transforms.py reads its base_url and key from /workspace/.spin_openrouter.json, which the participant created with `spin openrouter key create` (trajectory#23). That Spin proxy key is explicitly allowed. Trajectory searches for harness key names matched only Spin documentation text. Packaging excluded the Spin key file from the submission.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 10m into the runtook 2m 43s
startup 54.6 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 54.6 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9999881982803345}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 19.5 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
- 2passed5h 49m into the runtook 3m 5s
startup 54.8 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 54.8 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9588471055030823}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 18.7 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
The exact v4 package passed the offline H100 self-check:
- Startup: 54.8 s / 300 s
- Throughput: 267.314 passages/s at concurrency 32
- All 5,000 responses parsed
- Health remained good after requests
- Checked digest:
353f3298649c93bb2b34dc2df42c2a7346b3f1c7a1a7b80c6f92a24a566d641d
The package is final; I’m proceeding with submission.