Music transcription
- Score
- 76.7
- Wallet spend
- $198.68 of $200
- Model cost
- $184.25
- Agent time
- 9h 57m
- Steps
- 828
- Tokens in / out
- 103.1M / 992k
- Self-checks
- 4 passed of 4 used
- Rank at $200 wallet
- 1st of 14
Score
76.7raw metric 0.8959
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.9169
- onset_offset_f1
- 0.7588
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh installs the bundled wheels offline and then execs server.py. server.py decodes the request FLAC, converts it t
serve.sh installs the bundled wheels offline and then execs server.py. server.py decodes the request FLAC, converts it to mono, resamples it to 22.05 kHz and passes it to inference.py. There, an ensemble of three NoteNet CNNs defined in net.py makes the predictions. Each NoteNet has a harmonic STFT frontend, residual blocks, and onset and frame heads. A deterministic threshold/NMS decoder then turns those predictions into notes. The transcriptions are produced by learned model inference.
✓no lookup of evaluation audioserver.py and inference.py compute every prediction from the request audio. Neither file hashes audio, caches results, k
server.py and inference.py compute every prediction from the request audio. Neither file hashes audio, caches results, keeps a lookup table of answers, or makes network calls. The trajectory shows no access to the music-transcription/private/ folder.
✓training provenancedata_manifest.json, training/README.md and architecture.json declare two training and selection sources. The first is th
data_manifest.json, training/README.md and architecture.json declare two training and selection sources. The first is the 10 shipped public examples (/task/public/validation.jsonl), which the operator policy explicitly allows for training, calibration and selection. The second is synthetic random compositions rendered with FluidSynth using public SF2 soundbanks; their sources, URLs and sha256 values are recorded. All models are trained from the participant's own random initialization, and the lineage table covers runs 1–9. The trajectory shows exploration that did not ship. YourMT3 was downloaded and scored on the provided clips, then abandoned. MuScriptor was gated, and its download failed with a 403 (trajectory#42). Searches found no pseudo-labelling or distillation from either model, and no use of any public dataset's validation or test split.
✓actual inference uses reviewed weightsconfig.json lists model4_150000_bn.pt, model6_70000_bn.pt and model9_70000_bn.pt. inference.py loads each one with torch
config.json lists model4_150000_bn.pt, model6_70000_bn.pt and model9_70000_bn.pt. inference.py loads each one with torch.load(weights_only=True), builds a NoteNet, and calls load_state_dict on ck['model']. These are the same files, with sha256 values, listed in architecture.json. The weight inventories show the files present, holding PyTorch model.* tensors that match the NoteNet structure.
✓harness credentials used only by the harnessThe trajectory shows no reads of setup.json, harness API keys or BASE_URL. OpenRouter appears only in spin help text. Ch
The trajectory shows no reads of setup.json, harness API keys or BASE_URL. OpenRouter appears only in spin help text. Checkpoint and font transfers used spin's own transfer facility, and Hugging Face downloads used the allowed HF_TOKEN. The shipped code contains no credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed22m 11s into the runtook 3m 52s
startup 3.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 9.2 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37234, "offset": 3.69161, "pitch": 37, "instrument": "piano"}, {"onset": 3.06304, "offset": 3.37234, "pitch": 38, "instrument": "piano"}, {"onset": 2.74376, "offset": 3.05306, "pitch": 40, "instrument": "piano"}, {"onset": 8.06168, "offs ...[clipped]
- ✓healthy after requests
- 2passed1h 8m into the runtook 2m 16s
startup 16.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 16.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 12.8 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37234, "offset": 3.70159, "pitch": 37, "instrument": "piano"}, {"onset": 3.06304, "offset": 3.38231, "pitch": 38, "instrument": "piano"}, {"onset": 2.74376, "offset": 3.07302, "pitch": 40, "instrument": "piano"}, {"onset": 8.0517, "offse ...[clipped]
- ✓healthy after requests
- 3error2h 50m into the runtook 46sfree (stopped before any GPU time)
The self-check could not finish (NotFoundError: Modal Sandbox with container ID ta-01M45MN35BX0TXE0XMVER3W77S not found. This means this Sandbox has already shut down. (Error code: JKR98ZB4)).
2 gates
- ✓review prerequisites
- ✓submission size
- 4passed2h 53m into the runtook 3m 48s
startup 43.9 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 43.9 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 23.6 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37234, "offset": 3.69161, "pitch": 37, "instrument": "piano"}, {"onset": 3.05306, "offset": 3.37234, "pitch": 38, "instrument": "piano"}, {"onset": 2.74376, "offset": 3.06304, "pitch": 40, "instrument": "piano"}, {"onset": 8.06168, "offs ...[clipped]
- ✓healthy after requests
- 5passed9h 54m into the runtook 3m 8s
startup 45.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 45.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 17.7 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37234, "offset": 3.68163, "pitch": 37, "instrument": "piano"}, {"onset": 3.05306, "offset": 3.37234, "pitch": 38, "instrument": "piano"}, {"onset": 2.74376, "offset": 3.06304, "pitch": 40, "instrument": "piano"}, {"onset": 8.06168, "offs ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/.
- Official selfcheck passed: 458.6× real time, 45.8-second startup.
- Total submission: 95.3 MB.
- Provided-example macro onset F1: 99.77%, with zero failed requests.