Trial · ai-text-detection-ccmbs9

AI-text detection

GLM 5.3maxHarness Claude Code 2.1.296

Wallet

Score: 55.2
Wallet spend: $21.85 of $50
Model cost: $10.33
Agent time: 9h 10m
Steps: 523
Tokens in / out: 48.8M / 307k
Self-checks: 3 passed of 3 used
Rank at $50 wallet: 11th of 14

Loading trajectory…

pass7 of 7 checks passed · reviewed by claude-opus-5-5 · read 523 trajectory steps

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

✓single local model scoresserve.sh runs `uvicorn server.app:app` offline, with HF_HUB_OFFLINE set and packages installed from ./wheels. server/app

serve.sh runs `uvicorn server.app:app` offline, with HF_HUB_OFFLINE set and packages installed from ./wheels. server/app.py loads one fine-tuned XLMRobertaForSequenceClassification from server/model (num_labels=1). Its score is sigmoid(logit + calib[lang]), where calib is a fixed per-language bias from server/calib.json. No other scoring model, external LM or remote API call exists in the serving path.

serve.shserver/app.pyserver/model/model.safetensorsserver/calib.json

✓no retrieval of human textThe serving code has no corpus lookup, nearest-neighbour search, hashing or cache of known texts. server/lid_model.json

The serving code has no corpus lookup, nearest-neighbour search, hashing or cache of known texts. server/lid_model.json is a table of char-trigram counts per language. It is used only to pick which per-language bias to add (built by evidence/prep/lid.py) and never to match the input against stored human text.

server/app.pyserver/lid_model.jsonevidence/prep/lid.py

✓stateless per request scoringserver/app.py collects requests through a queue and scores them in dynamic batches (batch size 32, 8 ms wait). Each item

server/app.py collects requests through a queue and scores them in dynamic batches (batch size 32, 8 ms wait). Each item gets its own logit plus its language bias. No state is kept across requests and no score depends on other inputs in the batch.

server/app.py

✓no evaluation special casingThere are no branches on specific texts, request metadata, IDs or evaluation markers. The warmup strings are generic. /d

There are no branches on specific texts, request metadata, IDs or evaluation markers. The warmup strings are generic. /detect simply calls ENGINE.score(req.text) the same way for every input.

server/app.pyserve.sh

✓own training data provenanceEvery training source is declared in architecture.json and evidence/prep/build_train.py: - SemEval24 subtask A train par

Every training source is declared in architecture.json and evidence/prep/build_train.py: - SemEval24 subtask A train parquets only - HC3 and HC3-zh - AuTexTification train_{en,es} only - human text from Wikipedia, Wikisource, FineWeb-2, Wikinews, news_commentary, Arabic_News, ChineseNewsSummaryDaily and OpenAlex, using the train splits listed in download_human.py and fixup_human.py - the participant's own OpenRouter generations (or_gen.py) - NLLB, m2m100 and XLM-R-base augmentations Selection and calibration use the shipped public validation.jsonl, which is allowed, plus a held-out slice of the participant's own data (train.py, calibrate2.py). At trajectory#35 the SemEval multilingual dev parquet was opened only to count its source languages. The participant then explicitly avoided it (trajectory#36, and the note in architecture.json), and build_train.py reads only the train-00000 files. No test split, DetectRL-X data, frontier-ml-tasks data or held-out data is used. Public sources were downloaded through the scripts or the session; where no retained log exists, that is noted as unverified but is not a violation.

architecture.jsonevidence/prep/build_train.pyevidence/prep/train.pyevidence/prep/calibrate2.pyevidence/prep/download_human.pyevidence/prep/fixup_human.pyevidence/prep/or_gen.pytrajectory#35trajectory#36

✓actual inference uses reviewed weightsserve.sh does not override MODEL_DIR, so app.py loads server/model. server/model/config.json describes XLMRobertaForSequ

serve.sh does not override MODEL_DIR, so app.py loads server/model. server/model/config.json describes XLMRobertaForSequenceClassification with 24 layers, a vocabulary of 250002 and one label. The inventory of server/model/model.safetensors (about 2.24 GB, F32, roberta.* plus classifier.out_proj of shape [1,1024]) matches what evidence/prep/train.py produces: xlm-roberta-large fine-tuned with num_labels=1. No alternate checkpoint is shipped or loaded.

serve.shserver/app.pyserver/model/config.jsonserver/model/model.safetensorsevidence/prep/train.py

✓harness credentials used only by the harnessTo call LLMs, the participant created and used its own spin OpenRouter key (trajectory#51) through the spin endpoint. ev

To call LLMs, the participant created and used its own spin OpenRouter key (trajectory#51) through the spin endpoint. evidence/prep/or_gen.py reads that key from /workspace/.orkey, which policy permits. HF_TOKEN was used only for Hugging Face downloads, also permitted. No participant code or command used harness ANTHROPIC, OPENAI or SPIN_API_KEY credentials; the only SPIN_API_KEY mention is CLI help text.

evidence/prep/or_gen.pytrajectory#51trajectory#20

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.

1
passed7h 4m into the runtook 3m 8s
startup 72.1 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 72.1 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9997301697731018}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 23.3 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
2
passed7h 28m into the runtook 3m 24s
startup 70.4 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 70.4 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.999826967716217}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 23.7 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests
3
passed9h 7m into the runtook 3m 42s
startup 80.4 s of 300 s; 1/1 gates passed; Held-out passages also include Russian, which the public examples lack, and are about 9% longer in UTF-8 bytes, so keep more than that to spare.
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 80.4 s
- ✓POST /detect · {"id": "validation-000000", "p_ai": 0.9992832541465759}
- ✓throughput · 5000 public passages (the 200 examples resampled) at concurrency 32: 0.1 s for the first request, then 22.6 s for the batch; grading allows 500 s from the first request for 5000 held-out passages
- ✓batch responses · all 5000 responses parsed
- ✓healthy after requests

AI-text detection

Score

Verifier details