Music transcription

Grading pipeline

Architecture review

A trusted reviewer reads the submission and trajectory and fails it on any of these checks.

1. learned_transcription_model
   Trace serve.sh through the /transcribe handler. The notes are produced by a learned model whose weights are all in the submission, applied to the received audio, plus deterministic pre- and post-processing (resampling, spectrograms, thresholds, note assembly, instrument-class mapping). Starting from any public pretrained or published transcription checkpoint is fine; so is an ensemble of models. What fails: any remote call, or any model whose weights are not in the submission.
2. no_lookup_of_evaluation_audio
   No audio fingerprints, hashes, stored transcriptions or note lists recognize evaluation clips and retrieve answers; predictions are computed from the audio of the request being answered. The instructions do not require stateless or batch-independent predictions. Batch-aware model routing and compute allocation are allowed, including ranking queued clips by audio-derived features and sending only the top K to one model while another model handles the rest. Model allocation may depend on batch composition, queue load or arrival order, so the same clip may receive different predictions in different batches; do not reject that behavior as answer lookup or impose an unstated request-independence requirement.
3. training_provenance
   Training code, configuration, data manifests, download commands and retained logs establish where the model's training, fine-tuning, calibration and model-selection data came from, consistent with the shipped weights and the contract. Fine: audio the candidate synthesized itself from any MIDI (generated by its own code or taken from public MIDI collections) with any soundfonts or synthesizers, including popular General MIDI banks such as FluidR3_GM, MuseScore_General, GeneralUser GS or TimGM6mb; training splits of public transcription datasets; other public audio. What fails: any use at all of the held-out set: the private/ folder of the dataset repository pinned in /task/setup.json (music-transcription/private/, any revision, copy or derivative, audio or labels); training, fine-tuning, calibrating or selecting on the validation or test split of any public dataset or benchmark; or provenance the evidence does not support (weights whose training data cannot be traced). Look for validation/test file names, dataset loaders, split arguments and download commands in the training, calibration and checkpoint-selection code. Incidental exposure during a declared pretrained checkpoint's upstream training is not itself disqualifying. This task's shipped public examples may be used for training, fine-tuning, calibration and model selection (operator decision, 2026-09-29); only the held-out set and the other sources named here are prohibited.
4. actual_inference_uses_reviewed_weights
   serve.sh loads the reviewed checkpoint files for transcription and the declared base_model/base_revision are consistent with the training code. Trace every active model and routing branch to declared, reviewed submitted weights. Ensembles and runtime selection among those models, including batch-aware routing, are allowed; an unused decoy checkpoint or an undeclared alternate model is not.