Music transcription
- Score
- 0.0
- Wallet spend
- $1.80 of $10
- Model cost
- $10.08
- Agent time
- 2h 23m
- Steps
- 298
- Tokens in / out
- 30.8M / 197k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 10th of 19
Score
0.0raw metric 0.315
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.6303
- onset_offset_f1
- 0.1495
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelThe submission transcribes with learned models. serve.sh installs the bundled wheels offline and runs server.py. For eac
The submission transcribes with learned models. serve.sh installs the bundled wheels offline and runs server.py. For each /transcribe request, server.py decodes the audio and runs the Spotify Basic Pitch ONNX model (nmp.onnx, through basic_pitch_port.py) to get note activations and candidates. It then sends log-mel patches through four shipped CNN instrument classifiers (note_classifier.py InstrumentClassifier, loaded from safetensors). After that it applies route-based class decoding. The other steps (resampling, note decoding, calibration biases and thresholds) are deterministic post-processing around these models. No remote model is called.
✓no lookup of evaluation audioserver.py computes every prediction from the request audio alone. I found no audio hashing, no stored per-clip outputs,
server.py computes every prediction from the request audio alone. I found no audio hashing, no stored per-clip outputs, no table of reference transcriptions and no network access. The batched _process path only runs Basic Pitch inference, feature extraction, the classifier forward pass and decoding.
✓training provenanceTraining data has a declared, retained source. - Slakh: data_manifest.json and training/commands.md name the Slakh2100-F
Training data has a declared, retained source. - Slakh: data_manifest.json and training/commands.md name the Slakh2100-FLAC-Redux-Reduced dataset on Hugging Face (DreamyWanderer, pinned revision). training/prepare_slakh.py keeps only files under data/train/. training/logs/slakh_download.log shows split=data/train only, 2578 files, with every downloaded path under data/train. - The task's 10 public examples were used for fine-tuning and calibration, which is allowed (training/prepare_public.py, trajectory#20, trajectory#54). - Basic Pitch is a declared public pretrained model at a pinned revision and was not fine-tuned. - The session shows no access to music-transcription/private/ and no download of a test or validation split. data/test appeared only in a repository file listing.
✓actual inference uses reviewed weightsmodel_config.json maps the four serving routes (main, long25, ft50, brass_low45) to classifier_long_ft50, classifier_lon
model_config.json maps the four serving routes (main, long25, ft50, brass_low45) to classifier_long_ft50, classifier_long_ft25, classifier_ft50 and classifier_ft75 (.safetensors). Each has a weight inventory, and architecture.json and data_manifest.json declare their sha256 values. server.py loads exactly these files with safetensors load_file into InstrumentClassifier and loads nmp.onnx with onnxruntime. Each request runs through these models. I found no unused decoy model and no alternative serving path. A YourMT3 checkpoint was downloaded during the session for a baseline that was abandoned; it is not shipped or used.
✓harness credentials used only by the harnessNeither the shipped code nor the trajectory references ANTHROPIC_API_KEY, OPENAI_API_KEY or *_BASE_URL. Searches for API
Neither the shipped code nor the trajectory references ANTHROPIC_API_KEY, OPENAI_API_KEY or *_BASE_URL. Searches for API_KEY, BASE_URL, anthropic, openrouter and printenv/environ dumps found no participant use or copying of harness credentials. The session used the Spin CLI for compute and transfer and made Hugging Face downloads, all of which are allowed. serve.sh runs offline.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1failed2h 7m into the runtook 10sfree (stopped before any GPU time)
failed review prerequisites: The review needs architecture.json and serve.sh at the top of the submission
1 gates
- ✗review prerequisites · The review needs architecture.json and serve.sh at the top of the submission
- 2passed2h 21m into the runtook 2m 24s
startup 5.2 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 5.2 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 84.7 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0, "offset": 0.5388888888888889, "pitch": 55, "instrument": "piano"}, {"onset": 0.0, "offset": 0.06287981859410431, "pitch": 59, "instrument": "piano"}, {"onset": 0.0, "offset": 0.06287981859410431, "pitch": 65, "instrument": "piano"}, { ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully with frontier submit --confirm.
- Package: 78 files, 52.8 MB
- H100 selfcheck: Passed
- Startup: 5.2 seconds
- Throughput: 96.0× real time; requirement 50×
- Offline, response-schema, and health checks: Passed
- Supplied-example diagnostic macro onset F1: 0.57674; not a held-out estimate