Music transcription
- Score
- 0.0
- Wallet spend
- $24.43 of $50
- Model cost
- $7.77
- Agent time
- 4h 35m
- Steps
- 327
- Tokens in / out
- 120.5M / 426k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 6th of 14
Score
0.0raw metric 0.5139
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.6732
- onset_offset_f1
- 0.2644
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs src/server.py and points it at weights/. The server builds a single A.Transcriber from weights/config.json
serve.sh runs src/server.py and points it at weights/. The server builds a single A.Transcriber from weights/config.json and loads weights/model.safetensors. The model is a log-frequency STFT front end, a conv stem, 10 transformer blocks and a linear onset/active head. Notes come from deterministic peak picking in amt_common.decode_notes. The server path makes no remote model calls and uses no rule-based substitute. The checkpoint's tensor inventory (34.5M params: stem, blocks, norm, head) matches the declared architecture.
✓no lookup of evaluation audioThe server decodes the request FLAC, downmixes to mono, resamples to 16 kHz and runs the model forward pass, then decode
The server decodes the request FLAC, downmixes to mono, resamples to 16 kHz and runs the model forward pass, then decodes the notes. There is no hashing, fingerprinting, filename matching, or stored note table anywhere in src/server.py or src/amt_common.py. Every prediction is computed from the audio in the request.
✓training provenanceTraining data is the Slakh2100 train split only, in two renditions: 1,289 orig-mix tracks and 1,270 SF2 tracks. Both com
Training data is the Slakh2100 train split only, in two renditions: 1,289 orig-mix tracks and 1,270 SF2 tracks. Both come from the public HF mirror projectlosangeles/Slakh2100. The URLs, sizes and sha256 values are declared in provenance/data_manifest.json and download_commands.sh, and the build logs are retained. src/build_data.py's regex only keeps train/TrackNNNNN members, so validation/test tar members are filtered out. Training also samples six of the task's provided public validation examples, which is allowed. Selection used four provided examples plus 40 train-split dev songs (src/evaluate.py, src/dev_report.py). Searches found no use of private/ held-out data. In the trajectory, listing other datasets' validation/test trees through the HF API was metadata only and fed nothing into training. Train.py initializes from scratch, which matches base_model none.
✓actual inference uses reviewed weightsserve.sh passes $HERE/weights to server.py, and the server loads exactly weights/config.json, weights/decode.json and we
serve.sh passes $HERE/weights to server.py, and the server loads exactly weights/config.json, weights/decode.json and weights/model.safetensors into the reviewed architecture. trajectory#299 shows the g3r step-28k EMA checkpoint being copied to weights/model.safetensors, with an md5 check that matches. The HTTP evaluation in trajectory#300 then reproduced the selection score through the serving path. The weight inventory matches the architecture, with no extra tensors.
✓harness credentials used only by the harnessSearches of the trajectory and code for API keys, base URLs, Anthropic/OpenRouter usage, printenv and claude -p found no
Searches of the trajectory and code for API keys, base URLs, Anthropic/OpenRouter usage, printenv and claude -p found no misuse of harness credentials. HF_TOKEN was used only for Hugging Face downloads and API reads, and spin transfer credentials only for transfers. Both are permitted. The shipped code makes no external API calls.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed2h 30m into the runtook 10m 59s
startup 3.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 13.0 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37, "offset": 3.69, "pitch": 37, "instrument": "piano"}, {"onset": 3.05, "offset": 3.37, "pitch": 38, "instrument": "piano"}, {"onset": 2.75, "offset": 3.07, "pitch": 40, "instrument": "piano"}, {"onset": 8.05, "offset": 8.37, "pitch": 4 ...[clipped]
- ✓healthy after requests
- 2passed4h 34m into the runtook 1m 2s
startup 2.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 2.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 14.4 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37, "offset": 3.69, "pitch": 37, "instrument": "piano"}, {"onset": 3.05, "offset": 3.37, "pitch": 38, "instrument": "piano"}, {"onset": 2.75, "offset": 3.07, "pitch": 40, "instrument": "piano"}, {"onset": 8.05, "offset": 8.37, "pitch": 4 ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in: /workspace/output (45 files, 138.5 MB) was submitted at 10:22 UTC, and two selfchecks on this exact folder passed.
What it is. A from-scratch multi-instrument transcriber, 34.6M parameters. A 30 s clip gets a log-frequency spectrogram, a convolutional stem, and a 10-layer transformer over the whole clip at 50 frames/s. It predicts onsets and note activity for each of the nine classes × 128 pitches. Decoding is peak-picking with threshold 0.5. The served model is the EMA weights at step 28,000 of a 50k-step run.
Selfcheck results (265adf9e6b96, H100):
- startup 2.8 s of 300 s allowed
- 271 clips in 14.4 s, against a limit of 162.6 s (563× real time)
- every response HTTP 200 with the documented JSON
Scores I measured (official metric code, with caveats):
- Four held-out validation clips: 0.322 (thr 0.5). The served HTTP path reproduces this exactly.
- Orig-rendition dev (40 Slakh train songs, never trained on): 0.579 (thr 0.5).
- Per class, the held-out clips are weakest on pipe and mallets (F1 0) and reed (about 0.2–0.26). Organ, brass and bass also drop well below their dev values.
Choices I made that you should check:
- Training data. Slakh2100 train split only, in two renditions: the original 16 kHz mixes and the SoundFont rendition. I also used six of your ten validation clips for training, which you allowed. Four clips are held out for checkpoint and threshold selection.
- Label mapping. I mapped General MIDI programs to your nine classes by family: strings include programs 40–51, choir/voice and synth/ethnic/percussive/FX are unlabelled. The class descriptions don't pin this down, so the mapping is my interpretation.
- Checkpoint selection. Before scoring the final model, I set a rule: 0.6 × holdout + 0.4 × dev. Under it, step 28k scored best, narrowly. Holdout F1 falls after about 18k steps while dev keeps rising, which suggests specialisation to the training renditions. Four clips can't resolve differences of about 0.01, so treat the 28k choice as a judgement call.
- Snapshot ensemble. Averaging checkpoints gave at most +0.008 on the holdout, so I kept one model.
Things to know:
- The architecture review did not run during selfcheck, and it can still zero the submission. The provenance bundle is in
provenance/: sources with archive hashes, download commands, build and training logs, and the selection report. - The Hugging Face mirror I used declares CC BY-NC-SA 4.0. I didn't verify the original Slakh2100 terms. If the weights are used beyond this task, check that licence.
- I printed the first ~60 characters of a temporary transfer session's secret and token once in my terminal output by mistake. The session was scoped to one wallet prefix and expires at 12:39 UTC.
- Spend was about $24.4 of the $50 budget (two rented H100 leases).