Music transcription
- Score
- 0.0
- Wallet spend
- $10.47 of $50
- Model cost
- $31.49
- Agent time
- 2h 19m
- Steps
- 87
- Tokens in / out
- 40.8M / 129k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 6th of 14
Score
0.0raw metric 0.4644
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.6204
- onset_offset_f1
- 0.3747
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh starts server.py. server.py builds the Transcriber in engine.py, which loads the MuScriptor-small MIDI-token de
serve.sh starts server.py. server.py builds the Transcriber in engine.py, which loads the MuScriptor-small MIDI-token decoder (muscriptor/transcription_model.py, from weights/muscriptor-small.safetensors) and a pure-torch AST AudioSet classifier (ast_model.py, from weights/ast/model.safetensors). Notes come from mel-conditioned greedy decoding by the learned decoder over 5 s windows with a 2.5 s hop, with one pass per instrument group the AST router picks. Both tensor inventories have the expected learned-model shapes: dim 768, 14 layers, card 1393 for the decoder, and the standard AST tensors.
✓no lookup of evaluation audioengine.py, ast_model.py and server.py compute every prediction from the decoded request audio: AST routing, then conditi
engine.py, ast_model.py and server.py compute every prediction from the decoded request audio: AST routing, then conditioned decoding, then the event-to-note mapping. There is no hashing, fingerprinting, stored note table or network access, and the download helpers are stubbed out (muscriptor/utils/download.py). The self-check output (trajectory#82) shows timed requests were sent through the normal path. The only fallbacks are generic (an empty note list on error, and piano when the router picks no class), not lookups.
✓training provenanceNo training or fine-tuning was run on the shipped model; the trajectory has no optimizer or training run feeding the shi
No training or fine-tuning was run on the shipped model; the trajectory has no optimizer or training run feeding the shipped weights. provenance/data_manifest.json and provenance/routing_log.txt say calibration and selection (routing windows, token cap, small versus medium) used only the 10 provided /task/public validation clips, which CONTRACT.md explicitly permits (trajectory#12, trajectory#79, trajectory#83). The checkpoints are declared public sources: cocktailpeanut/muscriptor-small (a mirror of MuScriptor/muscriptor-small) and MIT/ast-finetuned-audioset-10-10-0.4593. provenance/download_log.txt keeps the curl and git commands for them. The downloads themselves are unverified beyond that retained log, which does not by itself fail the check. No forbidden validation/test split or private data was used.
✓actual inference uses reviewed weightsserver.py passes the local paths weights/muscriptor-small.safetensors and weights/ast to the Transcriber. In muscriptor/
server.py passes the local paths weights/muscriptor-small.safetensors and weights/ast to the Transcriber. In muscriptor/transcription_model.py, _resolve_source passes the local path through, download is stubbed, the config is read from weights/config.json (small: dim 768, 12 heads, 14 layers, card 1393), and safetensors load_file is followed by a strict load_state_dict. ast_model.py raises on any key mismatch. The reviewed inventories match those files (411888600 and 346404948 bytes). The medium checkpoint used in development was removed before submission (trajectory#79 shows medium exists False), so the shipped server loads only the reviewed weights.
✓harness credentials used only by the harnessSearches of the whole trajectory for ANTHROPIC, OPENAI, API_KEY, BASE_URL and claude found nothing. The participant used
Searches of the whole trajectory for ANTHROPIC, OPENAI, API_KEY, BASE_URL and claude found nothing. The participant used only the Spin CLI (spin compute exec/upload, spin s5cmd with the Spin session file) and the frontier selfcheck/submit commands (trajectory#16, trajectory#17, trajectory#21, trajectory#27, trajectory#80, trajectory#86). Policy permits all of these. No extra harness CLI was launched, and harness credentials were not read or used by the participant's code.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed2h 15m into the runtook 2m 25s
startup 10.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 10.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 80.2 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.09, "offset": 0.4, "pitch": 71, "instrument": "piano"}, {"onset": 0.09, "offset": 0.4, "pitch": 77, "instrument": "piano"}, {"onset": 0.09, "offset": 0.4, "pitch": 83, "instrument": "piano"}, {"onset": 0.24, "offset": 0.56, "pitch": 43, ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The transcription server is submitted. It starts in about 10 seconds and clears the 50× real-time gate.
The server is MuScriptor-small plus an AudioSet classifier. The classifier picks up to three instrument classes per clip, and each class is decoded with its own instrument condition. On the ten provided clips that scores 0.384 note-onset F1. Bass and mallets stay at zero because the classifier rarely selects them.
The graded self-check passed: startup in 10.1 seconds, the full public pass at 101× real time, and the submission is 759 MB.