Music transcription
- Score
- 0.0
- Wallet spend
- $2.49 of $10
- Model cost
- $7.39
- Agent time
- 7h 20m
- Steps
- 208
- Tokens in / out
- 39.4M / 177k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 9th of 14
Score
0.0raw metric 0.3636
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.5465
- onset_offset_f1
- 0.1979
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh starts server.py offline. For each request, server.py decodes the FLAC, resamples it to 16 kHz and computes a 1
serve.sh starts server.py offline. For each request, server.py decodes the FLAC, resamples it to 16 kHz and computes a 128-band log-mel (features.py). It runs two neural networks, both instances of the AMTNet class in model.py: a CNN front end, a 6-layer Transformer, and onset and frame heads of 9x128. It averages their sigmoid outputs and turns them into notes with deterministic peak picking and NMS (decode.py), using per-class thresholds. The weight inventories for weights.pt and weights2.pt are complete and show real trained tensors whose names and shapes match AMTNet. This is a learned audio-to-notes model, not a rule-based or retrieval system.
✓no lookup of evaluation audioThe serving path in server.py, decode.py and features.py has no hashing, no lookup tables keyed on audio, no network cal
The serving path in server.py, decode.py and features.py has no hashing, no lookup tables keyed on audio, no network calls and no stored per-clip outputs. Every output comes from the model's activations on the input audio. thresholds.json holds only per-class scalar thresholds and min_sep_frames. Searches of the trajectory found no access to music-transcription/private.
✓training provenanceTraining data is the Slakh2100 train split only: the HF mirror DreamyWanderer/Slakh2100-FLAC-Redux-Reduced, a mirror of
Training data is the Slakh2100 train split only: the HF mirror DreamyWanderer/Slakh2100-FLAC-Redux-Reduced, a mirror of zenodo 4599666 (CC-BY-4.0). download_slakh.py fetches it with snapshot_download and allow_patterns=['data/train/*']. slakh_download.log shows 2578 files and data_prep.log shows 1289 tracks, which match the train split. The trajectory confirms this: step 17 only listed the mirror's file names (the validation/test files were never fetched), the probe download in step 18 took train Track00001, and the step that launches download_slakh.py is recorded in trajectory/steps-33-62. The only other data is the 10 task-provided public clips, staged from /task/public (trajectory#39). The operator allows them for training, early stopping and threshold calibration (train.py, calibrate_ens.py, calibration_ens.log). Both models were trained from scratch: train.py builds a fresh AMTNet and --resume restores only its own checkpoints. architecture.json declares base 'none' / 'from-scratch', and searches found no download of pretrained weights such as YourMT3. DATA_MANIFEST.md records the commands, including seeds 0 and 1 and the --eq option.
✓actual inference uses reviewed weightsserver.py loads weights.pt and weights2.pt with load_state_dict(ck['model']) into AMTNet, and uses both models for every
server.py loads weights.pt and weights2.pt with load_state_dict(ck['model']) into AMTNet, and uses both models for every transcription. Both inventories are complete PyTorch checkpoints whose 'model.' keys match AMTNet's parameters. In the trajectory, assemble.sh copies out/best.pt to weights.pt and out2/best2.pt to weights2.pt, along with thresholds_ens.json and the logs (trajectory/steps-63-83, trajectory/steps-195-208). Those are the checkpoints produced by the train.py runs logged in train.log and train2.log.
✓harness credentials used only by the harnessNo step reads or reuses the harness's model-API credentials: searches for API_KEY, BASE_URL, openrouter, claude and code
No step reads or reuses the harness's model-API credentials: searches for API_KEY, BASE_URL, openrouter, claude and codex found nothing beyond the agent model name and help text. The only credential handled is a wallet-scoped R2 transfer session, prefix-limited to 'amt'. The participant created it with 'spin transfer credentials' (trajectory#39) and used it on the rented GPU only to pull its own staged validation tarball (trajectory#41, trajectory#42). This is the documented use of the spin tooling, which is allowed. The obfuscated chr() key names in that command are odd but do not misuse harness credentials. HF downloads used public repositories.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed3h 47m into the runtook 1m 14s
startup 3.4 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.4 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 13.6 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0, "offset": 0.45, "pitch": 59, "instrument": "piano"}, {"onset": 0.01, "offset": 0.42, "pitch": 55, "instrument": "piano"}, {"onset": 0.01, "offset": 0.35, "pitch": 65, "instrument": "piano"}, {"onset": 0.01, "offset": 0.89, "pitch": 77 ...[clipped]
- ✓healthy after requests
- 2passed7h 14m into the runtook 1m 10s
startup 3.6 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.6 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 18.5 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0, "offset": 0.43, "pitch": 55, "instrument": "piano"}, {"onset": 0.0, "offset": 0.26, "pitch": 59, "instrument": "piano"}, {"onset": 0.0, "offset": 0.88, "pitch": 77, "instrument": "piano"}, {"onset": 0.01, "offset": 0.24, "pitch": 65, ...[clipped]
- ✓healthy after requests
- 3passed7h 17m into the runtook 2m 12s
startup 3.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 18.5 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0, "offset": 0.43, "pitch": 55, "instrument": "piano"}, {"onset": 0.0, "offset": 0.26, "pitch": 59, "instrument": "piano"}, {"onset": 0.0, "offset": 0.88, "pitch": 77, "instrument": "piano"}, {"onset": 0.01, "offset": 0.24, "pitch": 65, ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. Final summary:
Result: 2-model ensemble transcriber, selfcheck PASSED on the submitted digest (30 files, 60.6 MB): startup 3.8s / 300s, 438x real-time / 50x required, valid response shapes. Local HTTP validation on the 10 provided clips: macro onset-F1 0.926.
Approach:
- Compact AMTNet (7.5M params: conv stack + 6-layer Transformer, 9×128 onset/frame heads, 10ms log-mel) trained from scratch on Slakh2100 train split only (1289 tracks, GM program→class mapping; drums and non-family programs excluded from labels), plus the 10 provided clips as oversampled training tracks.
- Model B adds log-mel EQ augmentation; inference averages both models' probabilities, then per-class thresholded peak-picking (thresholds calibrated on the provided clips: 0.915 single → 0.926 ensemble).
- Server is stdlib + preinstalled torch/torchaudio/soundfile/numpy only — no vendored deps, offline-safe.
Provenance: No Slakh validation/test data was ever downloaded or used (allow_patterns=["data/train/*"]); download/prep/training/calibration logs and a data manifest ship in /workspace/output.