Music transcription
- Score
- 73.4
- Wallet spend
- $46.01 of $50
- Model cost
- $153.20
- Agent time
- 5h 43m
- Steps
- 670
- Tokens in / out
- 85.6M / 781k
- Self-checks
- 4 passed of 4 used
- Rank at $50 wallet
- 2nd of 14
Score
73.4raw metric 0.8809
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.9155
- onset_offset_f1
- 0.5784
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh installs the bundled wheels offline, turns on HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and runs server.py. The
serve.sh installs the bundled wheels offline, turns on HF_HUB_OFFLINE and TRANSFORMERS_OFFLINE, and runs server.py. The server loads a learned ensemble: 9 dense onset/frame networks (MT3-encoder OnsetModel, HarmonicModel, HarmonicWaveModel and YourMT3-encoder YourOnset), an autoregressive MT3 decoder and a channel autoregressive decoder. Each is loaded with torch.load(weights_only=True) and load_state_dict. Notes come from the network outputs: sigmoid onset/frame scores are blended, then autoregressive boosting and GPU decoding are applied. The rules are not hand-written, and pretrained public bases (MR-MT3, YourMT3) are allowed.
✓no lookup of evaluation audioThe /transcribe handler in server.py decodes the base64 FLAC, resamples it to 16 kHz and runs it through the model pipel
The /transcribe handler in server.py decodes the base64 FLAC, resamples it to 16 kHz and runs it through the model pipeline. I found no audio hashing, fingerprinting, reference tables or remote calls. Serving runs offline with HF_HUB_OFFLINE=1. The trajectory shows no access to music-transcription/private.
✓training provenancedata_manifest.json and architecture.json declare every training and selection source: (1) procedural GM mixtures from ev
data_manifest.json and architecture.json declare every training and selection source: (1) procedural GM mixtures from evidence/synthesize.py; (2) the ten supplied public examples, which are permitted; (3) the Slakh2100 MIDI-only redux, train split only (aashishbishow/slakh2100_midi_only_redux at a pinned revision), with 1161 compositions for training and 128 reserved train compositions for selection. evidence/fetch_midi.py lists only the train/ tree and asserts path.startswith('train/'). evidence/corpus_split_audit.json shows no composition overlap. evidence/download_sources.sh keeps the download commands and sha256 checks for the soundfonts and base checkpoints. In the trajectory, the participant listed other Slakh repos, some of which contain test/ directories (trajectory#38), but I found no download or use of any validation/test split. Calibration used only the provided public validation examples.
✓actual inference uses reviewed weightsconfig.json points to model_0..8.pt, autoregressive.pt and channel_autoregressive.pt. Weight inventories were supplied f
config.json points to model_0..8.pt, autoregressive.pt and channel_autoregressive.pt. Weight inventories were supplied for all eleven files. server.py's Transcriber builds each model from config.json and loads exactly these files. architecture.json's selected_checkpoints map them to their source checkpoints and sha256 values. YourOnset is built with checkpoint=None, so its weights come only from the shipped state dicts. The serving call graph runs predict() on these models.
✓harness credentials used only by the harnessI searched the trajectory for harness API keys, base URLs, OpenRouter, Anthropic/Claude and nested codex exec CLI sessio
I searched the trajectory for harness API keys, base URLs, OpenRouter, Anthropic/Claude and nested codex exec CLI sessions, and found no use by the participant. The only credential used was the read-only HF_TOKEN for Hugging Face downloads, which the policy allows (trajectory#5 through trajectory#17). Spin compute and SSH leases are participant compute tools, not harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed20m 32s into the runtook 3m 29s
startup 8.6 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 8.6 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 27.6 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.09028941917419434, "offset": 0.272, "pitch": 65, "instrument": "piano"}, {"onset": 0.08436895513534547, "offset": 0.112, "pitch": 83, "instrument": "piano"}, {"onset": 0.10085515356063843, "offset": 0.64, "pitch": 77, "instrument": "pian ...[clipped]
- ✓healthy after requests
- 2passed1h 5m into the runtook 1m 21s
startup 7.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 7.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 32.1 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 16.806542788267137, "offset": 17.128, "pitch": 65, "instrument": "piano"}, {"onset": 27.120819597423075, "offset": 27.488, "pitch": 50, "instrument": "piano"}, {"onset": 15.867677015304565, "offset": 16.224, "pitch": 50, "instrument": "pia ...[clipped]
- ✓healthy after requests
- 3passed1h 40m into the runtook 2m 26s
startup 22.3 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 22.3 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 66.4 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 6.33669267976284, "offset": 6.656000000000001, "pitch": 66, "instrument": "piano"}, {"onset": 24.150104095220566, "offset": 24.312, "pitch": 79, "instrument": "piano"}, {"onset": 14.308, "offset": 14.568, "pitch": 76, "instrument": "piano" ...[clipped]
- ✓healthy after requests
- 4passed5h 36m into the runtook 7m 14s
startup 25.0 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 25.0 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 73.8 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 14.932475090026855, "offset": 15.248001098632812, "pitch": 65, "instrument": "piano"}, {"onset": 3.0584921836853027, "offset": 3.3680002689361572, "pitch": 38, "instrument": "piano"}, {"onset": 17.119583129882812, "offset": 17.768001556396 ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/.
Final offline selfcheck passed:
- Startup: 25.0 seconds
- Throughput: 110.2× real time
- Submission size: 469.6 MB