Music transcription
- Score
- 0.0
- Wallet spend
- $10.40 of $200
- Model cost
- $5.17
- Agent time
- 4h 30m
- Steps
- 265
- Tokens in / out
- 24.8M / 167k
- Self-checks
- 2 passed of 2 used
- Rank at $200 wallet
- 9th of 14
Score
0.0raw metric 0.5439
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.5738
- onset_offset_f1
- 0.0300
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py, which loads model.pt and model2.pt into a local CNN+Transformer Net (model.py). The POST handle
serve.sh runs server.py, which loads model.pt and model2.pt into a local CNN+Transformer Net (model.py). The POST handler decodes the FLAC in the request, resamples it to 44.1 kHz, computes a log-mel spectrogram and averages the sigmoid onset/frame outputs of the two local models. It then turns those outputs into notes with deterministic peak-picking, thresholds and class mapping. The server makes no remote calls, and the weight inventory shows every Net parameter is stored in the shipped checkpoint.
✓no lookup of evaluation audioserver.py contains no hashing, fingerprinting or stored note lists. Every prediction comes from model output on the audi
server.py contains no hashing, fingerprinting or stored note lists. Every prediction comes from model output on the audio of the current request. The only per-class data stored in the checkpoints is the onset thresholds, a deterministic post-processing parameter.
✓training provenanceThe training data is MIDI from the Lakh MIDI Dataset, rendered to audio with FluidSynth and General MIDI soundfonts. The
The training data is MIDI from the Lakh MIDI Dataset, rendered to audio with FluidSynth and General MIDI soundfonts. The download command is declared (data_manifest.json, architecture.json), and the trajectory shows lmd_full.tar.gz being fetched. render.py reads only LMD_ROOT. The 'real' set used in train.py, tune.py and ensemble.py for checkpoint selection and threshold tuning is /root/data/real/validation.jsonl. The trajectory shows it was copied from /task/public/audio, the task's 10 shipped public examples, which may be used. The internal validation split is held-out synthetic renders. The participant looked at MT3 and Slakh listings during the session but never downloaded or used them. No access to the private/ held-out folder or to any public dataset's validation or test split was found. The soundfont downloads are declared only in general terms, so that part is recorded as unverified rather than failing.
✓actual inference uses reviewed weightsserver.py loads model.pt and, when it exists, model2.pt, and averages their outputs. Both are shipped, and their tensor
server.py loads model.pt and, when it exists, model2.pt, and averages their outputs. Both are shipped, and their tensor inventories match the Net state_dict plus the mel mean/std statistics. The trajectory shows package.py building these files from m1 best (epoch 12) and m2 best (epoch 14, a fine-tune resumed from m1). Their md5 sums were checked against the machine copies before they went into /workspace/output. This matches architecture.json, which declares a from-scratch base with no pretrained model. There is no decoy or undeclared model.
✓harness credentials used only by the harnessSearches of the trajectory for harness API keys, base URLs, Anthropic/OpenAI SDK use and extra CLI harness sessions foun
Searches of the trajectory for harness API keys, base URLs, Anthropic/OpenAI SDK use and extra CLI harness sessions found no use of harness credentials in the participant's own code. The R2/S3 credentials used in steps 91 and 257 are Spin transfer-session credentials, not harness credentials. Hugging Face was queried only through public API searches.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed3h 25m into the runtook 4m 16s
startup 5.3 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 5.3 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 16.5 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.8671, "offset": 8.8671, "pitch": 36, "instrument": "piano"}, {"onset": 2.12, "offset": 10.12, "pitch": 36, "instrument": "piano"}, {"onset": 7.12, "offset": 15.12, "pitch": 36, "instrument": "piano"}, {"onset": 9.9394, "offset": 17.9394, ...[clipped]
- ✓healthy after requests
- 2passed4h 26m into the runtook 3m 39s
startup 3.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 13.1 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.866, "offset": 8.866, "pitch": 36, "instrument": "piano"}, {"onset": 2.12, "offset": 10.12, "pitch": 36, "instrument": "piano"}, {"onset": 7.12, "offset": 15.12, "pitch": 36, "instrument": "piano"}, {"onset": 25.2401, "offset": 25.3601, ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Selfcheck PASSED: startup 3.8s, 618.85x real-time (need 50x), all gates green. The submission is final. Submit: