Music transcription
- Score
- 0.0
- Wallet spend
- $27.00 of $50
- Model cost
- $25.66
- Agent time
- 9h 7m
- Steps
- 806
- Tokens in / out
- 81.0M / 809k
- Self-checks
- 3 passed of 3 used
- Rank at $50 wallet
- 6th of 14
Score
0.0raw metric 0.3621
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.4562
- onset_offset_f1
- 0.0986
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh execs serve.py. serve.py loads an ensemble of 8 Transcriber networks defined in model.py: a CNN trunk followed
serve.sh execs serve.py. serve.py loads an ensemble of 8 Transcriber networks defined in model.py: a CNN trunk followed by a dilated TCN, with onset and frame heads for 9 instruments × 128 pitches. The models were trained from scratch per architecture.json. Inference runs a log-mel GPU frontend, then the networks, then averages their logits, followed by deterministic peak picking in post.py. This is a learned transcription model.
✓no lookup of evaluation audioserve.py decodes the posted FLAC with soundfile and runs it through the networks. Per-request level normalization uses o
serve.py decodes the posted FLAC with soundfile and runs it through the networks. Per-request level normalization uses only that request's own median log-mel together with fixed constants in serving_config.json. Postprocessing is threshold-based peak picking and duplicate merging. No hashing, network calls, stored transcriptions or audio fingerprint lookup appears anywhere in the serving path.
✓training provenanceDATA_MANIFEST.md and architecture.json declare the training data: the Slakh2100 Redux train split from the HF mirrors Dr
DATA_MANIFEST.md and architecture.json declare the training data: the Slakh2100 Redux train split from the HF mirrors DreamyWanderer/Slakh2100-FLAC-Redux-Reduced and aashishbishow/slakh2100_midi_only_redux. code/download.py restricts allow_patterns to data/train/*, and code/prep.py filters to the train track range. Epoch selection used an md5-hash 10% holdout taken from the train split (code/train.py, code/evalrun.py). Thresholds were calibrated on the 10 supplied public clips, which the task explicitly permits. The trajectory lists the test and validation folders in the HF tree (trajectory#12) but downloads only data/train. Download and prep logs are retained (logs/download.log, logs/prep.log).
✓actual inference uses reviewed weightsserving_config.json lists weights/m1.pth through weights/m8.pth. serve.py's Engine calls torch.load and load_state_dict
serving_config.json lists weights/m1.pth through weights/m8.pth. serve.py's Engine calls torch.load and load_state_dict into Transcriber for each one, and infer() averages logits across all eight. logs/selfcheck_report.json shows the server loading all 8 weights. In the trajectory, the participant checked the md5 mapping between the training-run best checkpoints and the shipped weights.
✓harness credentials used only by the harnessI searched the trajectory for private-data access and for harness credentials (ANTHROPIC, API_KEY, BASE_URL, OPENROUTER,
I searched the trajectory for private-data access and for harness credentials (ANTHROPIC, API_KEY, BASE_URL, OPENROUTER, setup.json). No step used harness credentials; the only matches were Spin CLI documentation output. The HF_TOKEN given to the participant was used only for permitted dataset downloads.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed2h 7m into the runtook 4m 37s
startup 7.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 7.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 21.7 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 25.24, "offset": 25.55, "pitch": 30, "instrument": "piano"}, {"onset": 25.56, "offset": 25.68, "pitch": 30, "instrument": "piano"}, {"onset": 0.24, "offset": 0.65, "pitch": 31, "instrument": "piano"}, {"onset": 0.56, "offset": 0.65, "pitch ...[clipped]
- ✓healthy after requests
- 2passed8h 48m into the runtook 5m 8s
startup 7.4 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 7.4 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 22.7 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37, "offset": 3.84, "pitch": 37, "instrument": "piano"}, {"onset": 3.06, "offset": 3.51, "pitch": 38, "instrument": "piano"}, {"onset": 2.75, "offset": 3.18, "pitch": 40, "instrument": "piano"}, {"onset": 8.06, "offset": 8.64, "pitch": 4 ...[clipped]
- ✓healthy after requests
- 3passed9h 0m into the runtook 1m 25s
startup 8.5 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 8.5 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 23.2 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37, "offset": 3.84, "pitch": 37, "instrument": "piano"}, {"onset": 3.06, "offset": 3.51, "pitch": 38, "instrument": "piano"}, {"onset": 2.75, "offset": 3.18, "pitch": 40, "instrument": "piano"}, {"onset": 8.06, "offset": 8.64, "pitch": 4 ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
It reports exactly what I verified (154 files, 488.3 MB). Confirming: