Music transcription
- Score
- 0.0
- Wallet spend
- $10.80 of $50
- Model cost
- $30.55
- Agent time
- 7h 5m
- Steps
- 387
- Tokens in / out
- 94.7M / 245k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 6th of 14
Score
0.0raw metric 0.289
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.3350
- onset_offset_f1
- 0.0862
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py, which builds TranscriptionModelV2 from model.py. This is a CRNN with 4 conv blocks, a 2-layer B
serve.sh runs server.py, which builds TranscriptionModelV2 from model.py. This is a CRNN with 4 conv blocks, a 2-layer BiGRU and onset and onset-conditioned frame heads, giving 9 classes x 88 pitches. The server loads model.pt and runs the model on log-mel features computed from the request audio. amt_common.decode_notes then turns the sigmoid outputs into notes using per-class thresholds and peak-picking. The model.pt tensor inventory matches the V2 architecture exactly in fp16, so this is a learned transcription model.
✓no lookup of evaluation audioThe /transcribe handler in server.py decodes the base64 audio, resamples it, computes mel features and runs the model, t
The /transcribe handler in server.py decodes the base64 audio, resamples it, computes mel features and runs the model, then decodes notes. It makes no remote calls, has no hash or fingerprint tables and reads no stored transcriptions or label files. The only shipped artifacts are the weights and threshold config, and amt_common.py contains only feature, target and decoding code.
✓training provenanceThe training data is Slakh2100 train split only. training/setup_vm2.sh keeps the commands that download projectlosangele
The training data is Slakh2100 train split only. training/setup_vm2.sh keeps the commands that download projectlosangeles/Slakh2100 from HF and extract with --wildcards '*/train/*'. training/prep_slakh.py keeps only train MIDIs. The FluidSynth re-renders use the publicly declared FluidR3_GM soundfont (installed with apt) and MuseScore_General (fetched with curl). Both training logs show 3867 manifest entries, which equals 1289 train tracks x 3 renderings. The test-split paths seen in the trajectory are only archive listings, and extraction was restricted to train. run3 was trained from scratch: logs/train3.log has no init line. run6 was initialized from run3 final (logs/train6.log). The shipped soup averages run3 final with run6 epochs 0-5 (trajectory#369). The provided public clips (valdata.tgz built from /task/public) were used only for soup and threshold selection (trajectory#371), which the operator policy allows. run4, which trained on those clips, was rejected and not shipped. No use of private/ data was found.
✓actual inference uses reviewed weightsIn config.json, model is 'v2' and weights is 'model.pt', and server.py load_model reads exactly that file into Transcrip
In config.json, model is 'v2' and weights is 'model.pt', and server.py load_model reads exactly that file into TranscriptionModelV2 via load_state_dict. trajectory#373 shows package_model.py writing the soup checkpoint to /workspace/output/model.pt and writing config.json. The thresholds in config.json match the soup tuning output at trajectory#371. The packaged server was then validated at trajectory#377. The reviewed model.pt inventory matches the V2 architecture.
✓harness credentials used only by the harnessSearches of the trajectory for harness API keys, base URLs, OpenRouter/Anthropic/OpenAI and printenv found matches only
Searches of the trajectory for harness API keys, base URLs, OpenRouter/Anthropic/OpenAI and printenv found matches only in platform help text. The only credentials used were HF_TOKEN for public dataset downloads and the Spin S3 transfer credentials for the participant's own storage (training/train.py put_s3, setup_vm2.sh). Neither is a harness credential, and no model API calls were made by the participant's code.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed5h 49m into the runtook 8m 25s
startup 3.7 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.7 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 22.0 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.01, "offset": 0.07, "pitch": 55, "instrument": "piano"}, {"onset": 0.02, "offset": 0.86, "pitch": 65, "instrument": "piano"}, {"onset": 0.08, "offset": 0.19, "pitch": 59, "instrument": "piano"}, {"onset": 0.09, "offset": 0.16, "pitch": 5 ...[clipped]
- ✓healthy after requests
- 2passed7h 2m into the runtook 2m 28s
startup 3.7 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.7 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 22.4 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.01, "offset": 0.86, "pitch": 65, "instrument": "piano"}, {"onset": 0.09, "offset": 0.86, "pitch": 65, "instrument": "piano"}, {"onset": 0.09, "offset": 0.58, "pitch": 71, "instrument": "piano"}, {"onset": 0.24, "offset": 0.39, "pitch": 6 ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully at 2026-10-13 01:23:51 UTC.
Result: A from-scratch multi-instrument transcription model in /workspace/output/ (22 files, 12.3 MB — well under the 1 GB limit).
Model: CRNN (mel-spectrogram conv blocks + BiGRU) with an onset head and an onset-conditioned frame head, predicting 9 instrument classes × 88 pitches at 100 fps; notes decoded by per-class threshold peak-picking (model.py, amt_common.py, server.py).
Training data (provenance in TRAINING.md): Slakh2100 train split only (1,289 tracks; validation/test/omitted never touched), plus FluidSynth re-renders of the same train MIDIs with two GM soundfonts (FluidR3_GM, MuseScore_General) with program-substitution/gain augmentation. Continued training (run6) was averaged with the base run into a checkpoint soup.
Key decisions verified empirically:
- Fine-tuning on the 10 provided clips scored 0.85 on those clips but a controlled 8-train/2-held-out test showed no transfer (0.360 vs 0.380), so the shipped model contains no provided-clip training — clips were used only for selection and threshold calibration.
- Score on the provided clips via the official
validate.py: macro onset F1 0.335 (piano 0.50, pipe 0.58, bass 0.45, reed 0.40, guitar 0.36, strings 0.31, organ 0.24, brass 0.11, mallets 0.06).
Runtime (frontier selfcheck: PASSED): startup 3.7 s (limit 300 s), 363× real-time (required 50×; 271 clips in 22.4 s vs 162.6 s allowed), all response-shape gates passed, edge cases (silence/short/noise) return valid empty/note lists.