Music transcription
- Score
- 0.0
- Wallet spend
- $13.37 of $50
- Model cost
- $37.60
- Agent time
- 8h 1m
- Steps
- 412
- Tokens in / out
- 77.1M / 199k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 6th of 14
Score
0.0raw metric 0.4619
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.5070
- onset_offset_f1
- 0.0302
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py. load_model() calls torch.load on model.pt, builds the Transcriber CRNN defined in model.py and
serve.sh runs server.py. load_model() calls torch.load on model.pt, builds the Transcriber CRNN defined in model.py and loads the checkpoint's state dict. Each request runs a real forward pass and a sigmoid over the full mel spectrogram. decode.py then does deterministic peak-picking and NMS on those outputs and applies the per-class thresholds. The model.pt tensor inventory (94,222,133 bytes, tensors under model.*) matches the architecture in model.py, so the transcriptions come from a learned network.
✓no lookup of evaluation audioThe server.py request path is: base64 decode, soundfile read, mono, resample to 16 kHz, pad or trim to 30 s, model forwa
The server.py request path is: base64 decode, soundfile read, mono, resample to 16 kHz, pad or trim to 30 s, model forward, then decode.py note extraction. There is no hashing, no fingerprint or lookup table, no network call and no access to stored transcripts. logs/fingerprint.log is a training-time analysis that matched public example clips to soundfont timbres. The policy permits that use of public examples, and nothing at serving time uses it.
✓training provenanceThe model was trained from scratch (base_model none) on fluidsynth renders of Lakh MIDI lmd_full, made with GM soundfont
The model was trained from scratch (base_model none) on fluidsynth renders of Lakh MIDI lmd_full, made with GM soundfonts. architecture.json and data_manifest.md declare these sources. logs/download.log and the trajectory show the retained curl commands: the HF AghaTizi/lakh_lmd_full mirror, the osuosl FluidR3 soundfont, HF jbilcke/soundfonts and the Debian timgm6mb package. render.py makes an internal synthetic validation split (i % 97 == 0), and train.py uses that split for model selection. The training logs record the full5, full6 and full7 chain, where each run initializes from the previous one. Only the 10 permitted public example clips were used to calibrate thresholds.json. I found no use of the private set or of public validation/test splits.
✓actual inference uses reviewed weightsIn server.py, load_model() loads model.pt from the submission directory, falling back to ck['model'] because there is no
In server.py, load_model() loads model.pt from the submission directory, falling back to ck['model'] because there is no ema key. The inventory's tensor names and shapes match model.py's Transcriber. Per the trajectory, package.py exported the full7 EMA weights, which were copied to output/model.pt. The md5 and the 94,222,133-byte size match the reviewed inventory, and load_model was verified before submission.
✓harness credentials used only by the harnessThe participant session shows no use of harness credentials such as API keys, base URLs or setup.json, and no environmen
The participant session shows no use of harness credentials such as API keys, base URLs or setup.json, and no environment dumps. HF_TOKEN was only sent as a bearer header on curl downloads of public soundfonts, which the policy allows. Nothing was uploaded with it. The 'openrouter' string appears only in agent metadata and tool help text.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed4h 54m into the runtook 1m 31s
startup 2.7 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 2.7 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 27.1 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 11.2, "offset": 13.92, "pitch": 59, "instrument": "piano"}, {"onset": 8.38, "offset": 30.0, "pitch": 64, "instrument": "piano"}, {"onset": 21.2, "offset": 23.92, "pitch": 59, "instrument": "piano"}, {"onset": 29.64, "offset": 30.0, "pitch" ...[clipped]
- ✓healthy after requests
- 2passed7h 12m into the runtook 2m 9s
startup 4.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 4.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 25.8 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 19.34, "offset": 29.34, "pitch": 79, "instrument": "piano"}, {"onset": 26.66, "offset": 29.34, "pitch": 79, "instrument": "piano"}, {"onset": 17.76, "offset": 18.68, "pitch": 47, "instrument": "piano"}, {"onset": 26.04, "offset": 29.34, "p ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Final package verified end-to-end. Submitting.