AI-text detection
- Score
- 44.1
- Wallet spend
- $1.03 of $10
- Model cost
- $26.32
- Agent time
- 4h 1m
- Steps
- 100
- Tokens in / out
- 34.8M / 78k
- Self-checks
- 1 passed of 2 used
- Rank at $10 wallet
- 13th of 14
Score
44.1raw metric 0.4408
The score is the raw metric (1 − macro false-negative rate at 1% false positives), which already runs from 0 to 1, times 100.
Verifier details
- metric
- macro_fnr_at_1pct_fpr
- macro_fnr_at_1pct_fpr
- 0.5592
- reward
- 0.4408
- reward_definition
- 1 - mean language/family/condition FNR at global 1% FPR
- humans
- 1928
- ai
- 3072
- threshold_1pct
- 0.9945
- threshold_5pct
- 0.9875
- false_positive_rate
- 0.0099
- macro_fnr_at_5pct_fpr
- 0.2926
- macro_auroc
- 0.9279
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓single local model scoresserve.sh sets HF_HUB_OFFLINE=1, installs the vendored wheels with pip --no-index, then runs server.py. server.py loads o
serve.sh sets HF_HUB_OFFLINE=1, installs the vendored wheels with pip --no-index, then runs server.py. server.py loads one AutoModelForSequenceClassification (two labels) from the local model directory with local_files_only. p_ai is the maximum softmax AI probability over the head and tail chunks of the passage, all from that single model. There is no ensemble, no remote model call and no second scorer.
✓no retrieval of human textserver.py makes no network calls and loads no text corpus or index. It only tokenizes the incoming passage and runs the
server.py makes no network calls and loads no text corpus or index. It only tokenizes the incoming passage and runs the classifier. Nothing at serving time looks up human-written reference text.
✓stateless per request scoringEach /detect request is normalized, split into chunks and scored on its own. server.py has no cache, keeps no cross-requ
Each /detect request is normalized, split into chunks and scored on its own. server.py has no cache, keeps no cross-request state and does no batch-level adaptation or threshold updating.
✓no evaluation special casingThe preprocessing is generic: NFKC normalization, removal of invisible and Cf characters, collapsing of abnormal spacing
The preprocessing is generic: NFKC normalization, removal of invisible and Cf characters, collapsing of abnormal spacing, and a max over head and tail chunks. Nothing keys on evaluation IDs, known test texts, hashes or harness metadata.
✓own training data provenanceThe training data is declared in code/download_data.sh, code/build_corpus.py and logs/data_manifest.json. It consists of
The training data is declared in code/download_data.sh, code/build_corpus.py and logs/data_manifest.json. It consists of public train splits only (HC3, HC3-Chinese with test/valid/dev files skipped, MAGE train, AuTexTification train, COLING 2025 train, IberAuTexTification subtask_1 train), plus Gemma paraphrases made through the Spin API, synthetic character attacks, and the allowed /task/public/validation.jsonl examples. The manifests list the val/test files that were excluded, and searches found no DetectRL or frontier-ml-tasks usage. The iber train.tsv download is recorded at trajectory#35. HC3-ru and opus_books (train parquets) are declared in data_manifest.json, but no download command or log was retained, so their provenance is unverified rather than unsupported.
✓actual inference uses reviewed weightsThe model/model.safetensors inventory (about 2.24 GB, F32) has the XLM-RoBERTa-large architecture: a 250002x1024 embeddi
The model/model.safetensors inventory (about 2.24 GB, F32) has the XLM-RoBERTa-large architecture: a 250002x1024 embedding table, 24 layers and a 2-class classifier head. This matches the declared base, FacebookAI/xlm-roberta-large @ c23d21b. The training chain is train.py, then finetune_adapt.py, then finetune_pair.py. The trajectory shows the pair5 checkpoint being copied into /workspace/output/model (trajectory#71-72), and its detector_meta records 320 steps with freeze 20, matching logs/pair_adapt.json. serve.sh points MODEL_DIR at $ROOT/model, and server.py loads exactly that directory.
✓harness credentials used only by the harnessThe participant used HF_TOKEN only to download a public dataset (trajectory#35). The OR key file was used only against t
The participant used HF_TOKEN only to download a public dataset (trajectory#35). The OR key file was used only against the Spin API base URL to generate paraphrases (trajectory#37, trajectory#39). Spin transfer credentials were minted with the participant's own Spin CLI to copy checkpoints through wallet storage (trajectory#68-71), which is the CLI's documented use. The frontier selfcheck was invoked as instructed. No harness or grading credentials were used by participant code, and the trajectory shows no secrets were written into the output.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed3h 46m into the runtook 6m 48s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: ec-2026.9.0 huggingface_hub-1.33.0 jinja2-3.1.6 markupsafe-3.0.3 mpmath-1.3.0 networkx-3.6.1 nvidi
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: ec-2026.9.0 huggingface_hub-1.33.0 jinja2-3.1.6 markupsafe-3.0.3 mpmath-1.3.0 networkx-3.6.1 nvidia-cublas-cu12-12.8.4.1 nvidia-cuda-cupti-cu12-12.8.90 nvidia-cuda-nvrtc-cu12-12.8.93 nvidia-cuda-runtime-cu12-12.8.90 nvidia-cudnn-cu12-9.19.0.56 nvidia-cufft-cu12-11.3.3.83 nvidia-cufile-cu12-1.13.1.3 nvidia-curand-cu12-10.3.9.90 nvidia-cusolver-cu12-11.7.3.90 nvidia-cusparse-cu12-12.5.8.93 nvidia-cusparselt-cu12-0.7.1 nvidia-nccl-cu12-2.28.9 nvidia-nvjitlink-cu12-12.8.93 nvidia-nvshmem-cu12-3.4.5 nvidia-nvtx-cu12-12.8.90 opentelemetry-api-1.45.0 packaging-26.3 protobuf-7.36.2 pydantic-2.13.5 pydantic-core-2.46.5 pyyaml-6.0.3 regex-2026.9.29 sentencepiece-0.2.2 sniffio-1.3.1 starlette-1.7.0 sympy-1.14.0 tokenizers-0.23.2 torch-2.11.0+cu128 tqdm-4.70.1 transformers-5.18.0 typing-inspection-0.4.4 uvicorn-0.54.0 Traceback (most recent call last): File "/submission/server.py", line 17, in <module> from transformers import AutoModelForSequenceClassification, AutoTokenizer File "/opt/venv/lib/python3.12/site-packages/transformers/__init__.py", line 30, in <module> from . import dependency_versions_check File "/opt/venv/lib/python3.12/site-packages/transformers/dependency_versions_check.py", line 56, in <module> require_version_core(deps[pkg]) File "/opt/venv/lib/python3.12/site-packages/transformers/utils/versions.py", line 116, in require_version_core return require_version(requirement, hint) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/transformers/utils/versions.py", line 110, in require_version _compare_versions(op, got_ver, want_ver, requirement, pkg, hint) File "/opt/venv/lib/python3.12/site-packages/transformers/utils/versions.py", line 43, in _compare_versions raise ImportError( ImportError: safet ...[clipped]
- 2passed3h 54m into the runtook 6m 29s
startup 90.0 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 90.0 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9939143061637878}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.5 s for the first request, then 152.3 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
The agent's last message before its session ended.
The detector is submitted. The session ends with that hand-in.
It is one XLM-RoBERTa large classifier, adapted on public training text plus the 200 labeled examples you provided. serve.sh exposes POST /detect and returns {"p_ai": ...}. On the grading image it became healthy in 90 seconds and scored about 33 passages per second, both inside the limits. On your 200 examples, recall at a 1% false-positive rate was 0.64.