Music transcription
- Score
- 0.0
- Wallet spend
- $8.39 of $10
- Model cost
- $4.23
- Agent time
- 3h 13m
- Steps
- 212
- Tokens in / out
- 66.1M / 341k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 9th of 14
Score
0.0raw metric 0.3491
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.4759
- onset_offset_f1
- 0.1074
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs `exec python server.py`. server.py builds amt.model.TranscriptionModel and loads weights/model.safetensors
serve.sh runs `exec python server.py`. server.py builds amt.model.TranscriptionModel and loads weights/model.safetensors into it with strict=True. The model is a learned audio-to-notes network: a log-frequency spectrogram, a convolutional front end, 10 conformer-style encoder blocks (d=512, 8 heads) and a Linear(512, 4608) onset/activity head over 9 classes x 128 pitches. decode_notes then picks peaks using per-class thresholds. The tensor inventory (blocks.*, front.layers.*, project.weight [512,8448], head.weight [4608,512], F32, about 270 MB) matches that architecture. architecture.json declares base_model none / from-scratch, and train.py initialises randomly, so this is a genuine learned model trained by the participant.
✓no lookup of evaluation audioserver.py's transcribe path decodes the base64 FLAC with soundfile, downmixes to mono, resamples to 16 kHz, runs a forwa
server.py's transcribe path decodes the base64 FLAC with soundfile, downmixes to mono, resamples to 16 kHz, runs a forward pass of the model and decodes peaks. It does no hashing, caching by input, retrieval from stored outputs, or network calls. amt/model.py contains only signal processing and neural layers. No evaluation-audio lookup or answer table was found in the code or weights.
✓training provenancedata_manifest.json and architecture.json declare the training corpus: Slakh2100 redux MIDI and 16 kHz mix audio from the
data_manifest.json and architecture.json declare the training corpus: Slakh2100 redux MIDI and 16 kHz mix audio from the public HF mirror projectlosangeles/Slakh2100, train folder only, with retained curl download commands. extract_train_audio.py and extract_train_midi.py keep only train-folder files, and the logs agree: 1289 tracks, 88.9 h, skipped_nontrain=151. The 10 provided /task/public validation clips were used only for monitoring and threshold tuning (train.py, tune_decode.py), which the operator allows. The continuation run is initialised from the participant's own main-run EMA (run_gpu_train.sh). The trajectory (steps 207 and 212) agrees: no validation, test or omitted Slakh folders were read, and no undeclared corpus was used.
✓actual inference uses reviewed weightsserver.py loads weights/config.json and weights/model.safetensors, the inventoried file, into TranscriptionModel with st
server.py loads weights/config.json and weights/model.safetensors, the inventoried file, into TranscriptionModel with strict=True, and that model serves every /transcribe request. export_model.py writes these files from the final EMA checkpoint (step 8625, matching config.json and architecture.json). In the trajectory, the final selfcheck c2527b1c281b passed with the config thresholds visible in its server log (step 194). Step 201 shows the sha256 values and config contents, and steps 210-211 confirm that weights and code were unchanged after the selfcheck; only documentation was edited afterwards.
✓harness credentials used only by the harnessThe trajectory has no use of harness or model-API credentials: searches for ANTHROPIC, OPENAI, API_KEY usage, printenv a
The trajectory has no use of harness or model-API credentials: searches for ANTHROPIC, OPENAI, API_KEY usage, printenv and claude invocations found nothing beyond Spin documentation text. HF_TOKEN was the read-only download token given to the participant in the task prompt (trajectory step 1) and was used only for downloads. The Spin transfer session credentials used by s3_transfer.py and boto3 are the participant's own Spin allowance (step 206 memory note). Step 197 shows a credential scan of the output folder that found only key names, not secret values.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed1h 3m into the runtook 1m 31s
startup 3.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 10.6 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 18.37, "offset": 18.4, "pitch": 33, "instrument": "piano"}, {"onset": 17.74, "offset": 17.77, "pitch": 35, "instrument": "piano"}, {"onset": 7.12, "offset": 7.15, "pitch": 36, "instrument": "piano"}, {"onset": 12.12, "offset": 12.15, "pitc ...[clipped]
- ✓healthy after requests
- 2passed1h 58m into the runtook 1m 51s
startup 4.7 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 4.7 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 11.6 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 7.9, "offset": 7.93, "pitch": 31, "instrument": "piano"}, {"onset": 5.55, "offset": 5.58, "pitch": 32, "instrument": "piano"}, {"onset": 5.25, "offset": 5.28, "pitch": 33, "instrument": "piano"}, {"onset": 7.12, "offset": 7.15, "pitch": 36 ...[clipped]
- ✓healthy after requests
- 3passed3h 4m into the runtook 2m 45s
startup 5.0 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 5.0 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 12.2 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37, "offset": 3.67, "pitch": 37, "instrument": "piano"}, {"onset": 3.06, "offset": 3.34, "pitch": 38, "instrument": "piano"}, {"onset": 2.75, "offset": 3.18, "pitch": 40, "instrument": "piano"}, {"onset": 8.06, "offset": 8.4, "pitch": 40 ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The solution is submitted. It is a from-scratch model trained only on Slakh2100's train split. On the 10 validation clips it scores 0.357 macro onset F1 with per-class thresholds (0.325 with one global threshold). The graded score will be lower: the per-class thresholds were chosen on those same clips.
Submission
serve.shrunsserver.py, withPOST /transcribeandGET /health.- Weights are an EMA checkpoint (step 8625) in
weights/, with per-class thresholds inweights/config.json. The folder is about 398 MB. - The final
frontier selfcheckpassed on the exact weights and code: startup 5 s against 300 s allowed, 666x real time against 50x required, 271 clips in 12 s, and healthy after the load.
Model and data
- The model is a 67.5M-parameter conv front end with a conformer-style encoder, using 20 ms tokens that each predict two 10 ms frames. It has onset and activity heads over 9 classes × 128 pitches.
- Training labels come from the General MIDI program families in Slakh2100 (redux)
trainMIDI: 1289 tracks, 88.9 hours of 16 kHz mixes. Thevalidation,testandomittedfolders were never read. - The 10 provided validation clips were used only for monitoring and threshold selection, which you allowed.
Judgement calls to review
- Mapping to the nine classes: I used the GM program families, which match your class descriptions. Strings includes harp and timpani. Choir and voices, synth pads and leads, and percussion are unlabelled.
- Mallets: the model gets zero F1 on validation at every threshold, so its threshold of 0.3 is arbitrary. Mallets is 1/9 of the score.
- Per-class thresholds: a split-half check on the final model gives 0.322 held-out with per-class thresholds versus 0.313 with one global threshold. The gain is real but smaller than the in-sample number.
- Licence: the HF mirror's card says CC BY-NC-SA 4.0; the original Slakh2100 is CC BY 4.0. Check before any commercial use.
Limits
- Validation plateaued around 0.33 late in training, and the model is weak on organ, brass and mallets.
- Training crops scored about 0.41 against 0.24 on validation at the first checkpoint, so there is a generalisation gap. More training did not close it.
- The GPU budget was the constraint: $8.39 of $10 was spent on one H100 PCIe lease (48-minute run plus 90-minute continuation).
Provenance: architecture.json, data_manifest.json, the training and tuning logs, and the data-prep scripts are in /workspace/output/. The architecture review did not run during selfcheck, so these files matter.