Music transcription
- Score
- 0.0
- Wallet spend
- $3.19 of $10
- Model cost
- $1.56
- Agent time
- 11h 4m
- Steps
- 622
- Tokens in / out
- 60.3M / 508k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 10th of 19
Score
0.0raw metric 0.09071
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.1891
- onset_offset_f1
- 0.0206
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py. server.py builds the CRNN from model.py (5 conv blocks with BN and GELU, a Linear projection, a
serve.sh runs server.py. server.py builds the CRNN from model.py (5 conv blocks with BN and GELU, a Linear projection, a 2-layer BiGRU, and frame and onset heads of 9x128) and loads its weights from weights.pt with torch.load(weights_only=True) and load_state_dict. Each request goes through the Transcriber in infer.py: FIR decimation, log-mel from frontend.py, normalisation, then the model forward pass. Notes are then peak-picked from the model's probabilities using the thresholds in decode_config.json. The transcription is produced by a learned neural model, and the decoder is deterministic post-processing of its outputs.
✓no lookup of evaluation audioserver.py and infer.py compute predictions only from the decoded request audio. There are no audio hashes, fingerprint t
server.py and infer.py compute predictions only from the decoded request audio. There are no audio hashes, fingerprint tables, stored note lists or remote calls in the serving path. decode_config.json holds only scalar thresholds and per-class scales. The 10 public clips were used only to calibrate decoder thresholds, which is allowed, and no per-clip answers are stored.
✓training provenancearchitecture.json declares the training source: the public Slakh2100-redux MIDI train split (HF aashishbishow/slakh2100_
architecture.json declares the training source: the public Slakh2100-redux MIDI train split (HF aashishbishow/slakh2100_midi_only_redux), rendered with FluidSynth GM soundfonts. Of the 1281 rendered tracks, 1231 were used for training and 50 held out as dev. The validation and test splits are declared unused. The trajectory supports this: - The HF snapshot used allow_patterns=['train/**','README*'] (trajectory#53 region, steps 38-53). - The tarballs sent to the GPU box contained only train/ (slakh_full.tgz in steps 93-125, slakh_all.tgz in trajectory#143). - make_split.py takes every 26th train track for dev. - The monitoring set val1 was built from dev renders, which are train-split tracks. - The final run chain r4 -> r6 -> r7 trained with train3.py on /root/data/mixes, i.e. train renders (trajectory#565). The public task clips came from /task/public via the participant's own Spin prefix (trajectory#298). They were used only for evaluation and threshold calibration, which is allowed. dl_slakh.py also fetches the 'omitted' subset, but it is declared unused and render.py and train3.py default to the train split. I found no use of music-transcription/private/. The HF listing in logs/hf_listing.log is only a listing of split names. Download commands are partly retained, so the exact download is partially unverified, but the source is declared.
✓actual inference uses reviewed weightsThere is one checkpoint, weights.pt. Its tensor inventory (model.blocks.0-4, proj, rnn l0/l1 plus reverse, frame_head, o
There is one checkpoint, weights.pt. Its tensor inventory (model.blocks.0-4, proj, rnn l0/l1 plus reverse, frame_head, onset_head) matches the parameter shapes in model.py exactly, and server.py loads that file into the CRNN it serves. The trajectory shows final_r7.pt exported from /root/runs/r7/last.pt and copied to /workspace/output/weights.pt, with md5 checks against the r7 run (trajectory#597, trajectory#608). train3.py initialises the CRNN from scratch and resumes only from its own checkpoints, which is consistent with base_model 'none' in architecture.json.
✓harness credentials used only by the harnessThe participant's tools used only Spin compute and Spin transfer credentials: creds.json and session.json from `spin tra
The participant's tools used only Spin compute and Spin transfer credentials: creds.json and session.json from `spin transfer credentials`, used with boto3 and s5cmd against the participant's own spin prefix. HF_TOKEN was used for read-only downloads, and every upload went to the Spin bucket. The claude-router and ANTHROPIC/OPENROUTER key names appear only in a `ps aux` listing of the harness's own process (trajectory#141). The participant never read, copied or used them. The validation CLI's judge options were never passed, and validate.py was run without any judge key or base URL (trajectory#303). No extra harness CLIs were launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed2h 39m into the runtook 2m
startup 3.4 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.4 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 24.4 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": []}
- ✓healthy after requests
- 2passed5h 57m into the runtook 3m 2s
startup 3.5 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.5 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 37.1 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.5572789115646258, "offset": 0.603718820861678, "pitch": 50, "instrument": "piano"}, {"onset": 1.0216780045351475, "offset": 1.0681179138321995, "pitch": 50, "instrument": "piano"}, {"onset": 1.18421768707483, "offset": 1.230657596371882, ...[clipped]
- ✓healthy after requests
- 3passed11h 0m into the runtook 1m 47s
startup 3.2 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.2 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 24.2 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 28.374784580498865, "offset": 28.955283446712016, "pitch": 36, "instrument": "piano"}, {"onset": 0.25541950113378686, "offset": 0.8359183673469388, "pitch": 43, "instrument": "piano"}, {"onset": 0.5572789115646258, "offset": 0.835918367346 ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in.
Final result: a from-scratch CRNN (13.2 M params) that takes a 30 s FLAC and returns per-note onset/offset/pitch/instrument across the nine classes.
- Model: 5 conv blocks over a 192-bin log-mel (22.05 kHz, 11.6 ms hop), one 2× frequency pool and one 2× time pool, a 2-layer BiGRU, and two 9×128 heads (frame activity + onset), upsampled to the frame grid.
- Training: 30,000 steps on re-rendered Slakh2100-redux train-split MIDI only (1,231 tracks, 91 h, five GM soundfonts, synthetic augmentation), in four chained cosine cycles; class-balanced window sampling and a class-frequency-weighted BCE onset loss.
- Decoding: local-maximum peak picking on the onset head at threshold 0.8 with per-class scales tuned on my 48 held-out dev clips and cross-checked on the provided public clips.
Verification: selfcheck passed on the grading infrastructure — 20 files / 52.8 MB (limit 1 GB), 3.2 s startup (limit 300 s), 336× real time (limit 50×), valid POST, healthy after the load. The exact shipped package scored 0.1276 macro onset F1 on the 10 public clips when served over HTTP end-to-end.
Two things worth flagging honestly: the compute lease expired right after the selfcheck, so a final 6,000-step continuation I had started (which would plausibly have added a few thousandths) was lost — the shipped model is the validated step-30,000 checkpoint. And the absolute score is modest; diagnostics showed the model localises onsets well (96% of predicted peaks fall within 46 ms of a real onset) but its instrument/pitch identification is the limiting factor, so the headroom is in model quality rather than decoding.