AI-text detection

Grading pipeline

Architecture review

A trusted reviewer reads the submission and trajectory and fails it on any of these checks.

1. single_local_model_scores
   Trace serve.sh through the /detect handler. p_ai is computed by the candidate's own trained detector, whose weights are all in the submission (its tokenizer and a fixed calibration map such as temperature scaling are fine). Averaging or stacking several heads, seeds or checkpoints that the candidate itself trained during this task counts as that one detector. What fails: scoring with likelihoods, perplexities or features from a model the candidate did not train (a separate language model used as a detector signal), any remote call, or any model whose weights are not in the submission.
2. no_retrieval_of_human_text
   No index, database, embedding store, hash set, Bloom filter, n-gram table, suffix array or copy of any text corpus (web, news, academic, encyclopedic or fiction text in any language, or any AI-text detection dataset) is packaged or queried, and p_ai never comes from looking up whether the input text appears in stored text.
3. stateless_per_request_scoring
   p_ai depends only on the text of the request being answered. No caching, grouping, ranking or comparison across requests, and no state that changes answers depending on what else was requested.
4. no_evaluation_special_casing
   The server does not key behavior on specific evaluation texts or request order, and does not try to detect or fingerprint the evaluation.
5. own_training_data_provenance
   Training code, data manifests, download commands and retained logs establish where the detector's training, fine-tuning, calibration and model-selection data came from, consistent with the contract: AI passages the candidate generated with models it ran or called, openly licensed human text, and training splits and other public corpora are fine. What fails: any use at all of the held-out set or of any revision of the dataset repository pinned in /task/setup.json (inductionlabs/frontier-ml-tasks, folder ai-text-detection; superseded revisions carry the full validation partition); training, fine-tuning, calibrating or selecting on the validation or test split of any public dataset or benchmark, not only AI-text detection ones (our validation set is from the DetectRL-X binary open release, WUJUNCHAO/DetectRL-X: the held-out passages are drawn from its test rows, so training, fine-tuning, calibrating or selecting on those test rows beyond the shipped public examples fails, while its training rows are an ordinary public training split); or unsupported provenance. Reading a public validation or test split only to remove its passages from the training, calibration or selection data (decontamination, e.g. dropping training rows that also appear in the DetectRL-X test rows) is fine, as long as those rows feed nothing else. Look for validation/test file names, benchmark loaders, split arguments and download commands in the training, calibration and checkpoint-selection code. Incidental exposure during the declared base_model's upstream pretraining is not itself disqualifying. This task's shipped public examples may be used for training, fine-tuning, calibration and model selection; only the held-out set and the other sources named here are prohibited.
6. actual_inference_uses_reviewed_weights
   serve.sh loads the reviewed checkpoint files for scoring and the declared base_model/base_revision are consistent with the training code; no unused decoy checkpoint and no alternate model path selectable at runtime.