Music transcription
- Score
- 0.0
- Wallet spend
- $23.02 of $50
- Model cost
- $4.39
- Agent time
- 7h 57m
- Steps
- 1,457
- Tokens in / out
- 121.5M / 1.3M
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 6th of 19
Score
0.0raw metric 0.185
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.3215
- onset_offset_f1
- 0.0057
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh execs server.py with the model dir and sets HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE. server.py builds amnet.AMNet(w
serve.sh execs server.py with the model dir and sets HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE. server.py builds amnet.AMNet(width=48, n_blocks=7), loads model.safetensors with load_state_dict, and for each request runs deterministic STFT harmonic-salience features (amnet.compute_features), then the CNN forward pass, then sigmoid, then peak-picking with per-class thresholds from model_meta.json. The model is a trained 178k-parameter dilated temporal CNN that produces onset and frame logits for 9 classes × 88 pitches. There are no remote API calls and no rule-based or hard-coded note output.
✓no lookup of evaluation audioThe /transcribe handler base64-decodes the FLAC, reads it with soundfile, downmixes and resamples the waveform, and pass
The /transcribe handler base64-decodes the FLAC, reads it with soundfile, downmixes and resamples the waveform, and passes it through features → model → decoder. The server has no hashing, fingerprinting, audio-matching index, stored note tables or network access. Predictions depend only on the request waveform and the model weights. The trajectory's Slakh track-identification probes were run during development against the 10 provided public clips; none of that code or data ships in the server path.
✓training provenancedata_manifest.json, README.md and architecture.json declare the training corpus: Lakh MIDI lmd_full (curl download from
data_manifest.json, README.md and architecture.json declare the training corpus: Lakh MIDI lmd_full (curl download from the Columbia host), rendered with FluidSynth 2.2.5 and five apt-installed GM soundfonts (render_data2.py, render_data3.py; logs/render.log). The result is 31,846 training clips plus 250 held-out rendered clips used for validation. The LMD download command is declared, but no download log was retained, so the download itself is unverified. Slakh2100 metadata (fetch_md5.py) was used only to exclude matching LMD md5s before rendering (render.log reports 1144 files excluded). That is exclusion, not training. Thresholds were calibrated and checkpoints selected on the 10 provided public clips, which the task allows. During exploration the participant downloaded Slakh MIDI and one Slakh validation-split mix to work out where the provided clips came from. Nothing in the trajectory shows those files entering training, calibration or selection of the shipped model. I found no access to music-transcription/private in the trajectory.
✓actual inference uses reviewed weightsmodel_meta.json lists checkpoints ['model.safetensors'] with width 48 and 7 blocks. server.py loads exactly that file in
model_meta.json lists checkpoints ['model.safetensors'] with width 48 and 7 blocks. server.py loads exactly that file into AMNet and uses it for every request. The tensor inventory of model.safetensors (blocks.0-6 c1/c2/n1/n2, mix1/2, norm1/2, pitch_emb[88,48], onset/frame heads [9,48,1,1], biases [9,88], all F32, 718,272 bytes) matches the AMNet(48,7) definition in amnet.py. export_model.py converts the train.py checkpoint (run F step 4000, trained from scratch through the D→E→F chain) to this safetensors file. The trajectory shows the file in the final output listing and passing the selfcheck.
✓harness credentials used only by the harnessTrajectory searches for ANTHROPIC, API_KEY, BASE_URL, openrouter, 'claude -p', codex and printenv found only Spin CLI he
Trajectory searches for ANTHROPIC, API_KEY, BASE_URL, openrouter, 'claude -p', codex and printenv found only Spin CLI help text. I found no participant use or copying of harness credentials and no extra model sessions. The only token use was HF_TOKEN for Hugging Face downloads, plus Spin GPU compute; both are allowed. The shipped server runs offline.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed6h 52m into the runtook 57s
startup 3.0 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.0 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 17.1 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0813, "offset": 0.1313, "pitch": 83, "instrument": "organ"}, {"onset": 0.4063, "offset": 0.4563, "pitch": 77, "instrument": "mallets"}, {"onset": 1.0217, "offset": 1.0717, "pitch": 81, "instrument": "mallets"}, {"onset": 1.0217, "offset" ...[clipped]
- ✓healthy after requests
- 2passed7h 37m into the runtook 15m 10s
startup 4.6 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 4.6 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 20.1 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0813, "offset": 0.1313, "pitch": 83, "instrument": "organ"}, {"onset": 0.4063, "offset": 0.4563, "pitch": 77, "instrument": "mallets"}, {"onset": 1.0217, "offset": 1.0717, "pitch": 81, "instrument": "mallets"}, {"onset": 1.0217, "offset" ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted at 03:32:36 UTC, ahead of the 04:22 deadline.
Final submission (/workspace/output, 56 files, 1.3 MB):
| Model | 0.18 M-parameter dilated temporal CNN (width 48, 7 blocks) over a 4×88 harmonic-salience STFT front end, trained from scratch |
| Data | Lakh MIDI Dataset rendered to 32,346 FLAC clips with FluidSynth + five General MIDI soundfonts (31,846 train / 250 held out); Slakh-matched LMD files excluded before rendering |
| Runs | D (8,918 steps) → E (30,882, combined corpus) → F (4,498, lr 1.2e-5) → shipped: run F step 4,000; run G trained as a check and rejected |
| Decoding | per-class probability thresholds [0.95, 0.95, 0.5, 0.85, 0.9, 0.95, 0.85, 0.95, 0.95], 50 ms duplicate suppression |
| Public validation | macro onset F1 0.1924 (piano .293, mallets .206, organ .102, guitar .119, bass .087, strings .124, brass .172, reed .284, pipe .346), 0 failed requests |
| Speed | RTF 98.5 locally, 403.9× in the selfcheck (limit 50×) — 271 clips in 20.1 s of the 162.6 s allowance |
| Startup | 4.6 s (limit 300 s), offline, no vendored wheels needed |
| Selfcheck | 2 checks run, both passed all gates |
Every model-selection and calibration decision was made with a leave-one-clip-out protocol on the ten provided clips, and the rejected alternatives are documented in logs/decoder_experiments.md: clip-level class gate (LOO 0.117 vs 0.155), fine-tuning on the provided clips (0.128 vs 0.169), per-class pitch-range prior (0.132 vs 0.158), winner-take-all decoding, temporal smoothing, posterior normalisation, frame-head filtering, weight soups and probability-map ensembles. The shipped weights were never trained on the provided clips, and no public validation or test split was used for training, calibration or selection.