Music transcription
- Score
- 73.1
- Wallet spend
- $199.32 of $200
- Model cost
- $14.29
- Agent time
- 9h 3m
- Steps
- 373
- Tokens in / out
- 45.7M / 465k
- Self-checks
- 3 passed of 3 used
- Rank at $200 wallet
- 3rd of 14
Score
73.1raw metric 0.8799
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.9130
- onset_offset_f1
- 0.5537
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py. server.py loads ensemble.json through inference.load_engine. Each checkpoint is loaded with tor
serve.sh runs server.py. server.py loads ensemble.json through inference.load_engine. Each checkpoint is loaded with torch.load into model.Transcriber, a learned CNN with residual blocks, context layers, an 18-channel onset/activity head and a 9-class presence head, built on analytic harmonic log-frequency features. Its sigmoid outputs are combined by class weight and then decoded by deterministic peak picking. The inventories for weights.pt and onset_weights.pt are complete and contain the tensors model.py expects: 512 channels, 14 blocks, stem 512x49x5x5 and head 18x1024. So transcription comes from trained network weights, not rules or a lookup.
✓no lookup of evaluation audioserver.py decodes the base64 audio, downmixes to mono, clips it to 30 s, resamples it and passes it only to engine.trans
server.py decodes the base64 audio, downmixes to mono, clips it to 30 s, resamples it and passes it only to engine.transcribe. inference.py runs the network and decode() with the fixed thresholds in calibration.json. There is no hashing, fingerprinting, reference table, file lookup or network call on input audio, and serve.sh sets offline environment variables. I found nothing that reads music-transcription/private/.
✓training provenancetraining/data_manifest.json, training/download_commands.sh and training/commands.txt declare every training source. (1)
training/data_manifest.json, training/download_commands.sh and training/commands.txt declare every training source. (1) Procedurally generated note sequences, rendered with public soundfonts (FluidR3, TimGM6mb, GeneralUser GS, SGM); download URLs and hashes are given. (2) Slakh2100 Redux MIDI, restricted to training tracks only. training/prepare_slakh.py extracts only /train/ members that Redux retains and that were never moved to, or duplicated in, validation or test splits. (3) The 10 task examples in /task/public/validation.jsonl, which the policy permits for any use; training/synthdata.py loads them. The Slakh curl download is recorded in the trajectory. training/initialization.json records run1 as from-scratch default initialization, followed by a hashed parent chain. Calibration and selection used only the task examples and synthesized/Slakh-train probes (training/README.md, architecture.json). No val/test split or private set was used. The YourMT3 checkpoint the agent tried early on was only scored on the public examples and then abandoned; nothing shipped uses it or its outputs.
✓actual inference uses reviewed weightsserver.py loads ensemble.json, which lists weights.pt and onset_weights.pt with class weights. inference.EnsembleEngine
server.py loads ensemble.json, which lists weights.pt and onset_weights.pt with class weights. inference.EnsembleEngine torch.loads exactly those files into Transcriber and calls load_state_dict. These are the reviewed checkpoints whose inventories match model.py. training/current_checkpoint.json records their sha256 values and source logs. The trajectory shows the averaged big2 checkpoint and the onset continuation being copied to output/weights.pt and output/onset_weights.pt, and the hashes being verified afterwards. architecture.json declares the same two weight files.
✓harness credentials used only by the harnessI searched the trajectory for ANTHROPIC, OPENAI, API_KEY, BASE_URL and setup.json and found nothing. Remote work went on
I searched the trajectory for ANTHROPIC, OPENAI, API_KEY, BASE_URL and setup.json and found nothing. Remote work went only through the participant's Spin compute/s3 tooling, and downloads came from public URLs and Hugging Face. No step reads, copies or uses harness credentials, and the shipped code contains no credential handling.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed27m 21s into the runtook 4m 27s
startup 8.5 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 8.5 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 13.6 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.38, "offset": 4.02, "pitch": 37, "instrument": "piano"}, {"onset": 3.08, "offset": 3.38, "pitch": 38, "instrument": "piano"}, {"onset": 2.76, "offset": 3.08, "pitch": 40, "instrument": "piano"}, {"onset": 8.08, "offset": 8.66, "pitch": 4 ...[clipped]
- ✓healthy after requests
- 2passed2h 17m into the runtook 3m 38s
startup 80.7 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 80.7 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 44.8 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.38, "offset": 3.7, "pitch": 37, "instrument": "piano"}, {"onset": 3.08, "offset": 3.38, "pitch": 38, "instrument": "piano"}, {"onset": 2.76, "offset": 3.08, "pitch": 40, "instrument": "piano"}, {"onset": 8.08, "offset": 8.46, "pitch": 40 ...[clipped]
- ✓healthy after requests
- 3passed8h 56m into the runtook 7m 30s
startup 85.6 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 85.6 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 80.5 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.36, "offset": 3.7, "pitch": 37, "instrument": "piano"}, {"onset": 3.06, "offset": 3.38, "pitch": 38, "instrument": "piano"}, {"onset": 2.74, "offset": 3.06, "pitch": 40, "instrument": "piano"}, {"onset": 8.06, "offset": 8.38, "pitch": 40 ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with the trained model, serve.sh, endpoint code, and training provenance.
- Provided-example onset F1: 0.9969; not a held-out score.
- Offline H100 self-check: passed.
- Startup: 85.6 seconds.
- Throughput: 101× real time.
- Submission size: 565.8 MB.