Music transcription
- Score
- 28.8
- Wallet spend
- $5.52 of $10
- Model cost
- $28.81
- Agent time
- 5h 7m
- Steps
- 246
- Tokens in / out
- 57.2M / 207k
- Self-checks
- 2 passed of 2 used
- Rank at $10 wallet
- 5th of 14
Score
28.8raw metric 0.6818
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.6968
- onset_offset_f1
- 0.3616
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py. server.py loads an Ensemble (ensemble.py) of two TranscriptionModel networks (model.py), each a
serve.sh runs server.py. server.py loads an Ensemble (ensemble.py) of two TranscriptionModel networks (model.py), each a log-mel front end, conv blocks, a BiGRU and onset/frame heads, and averages their sigmoid probabilities. Notes come from those probabilities through peak-pick decoding in infer.py.decode_notes, using the tuned per-class thresholds. This is a learned neural transcription model, not rules or lookup.
✓no lookup of evaluation audioThe /transcribe path decodes base64 audio, converts it to mono, resamples to 16 kHz, runs a batched model forward pass a
The /transcribe path decodes base64 audio, converts it to mono, resamples to 16 kHz, runs a batched model forward pass and decodes notes. It does no hashing, has no table of reference notes, makes no remote calls and does not read stored labels. The amt_notes and /workspace/val reference loading in infer.py and tune_ens.py is used only offline for public-clip tuning, never in serving. No access to music-transcription/private/ data appears anywhere in the code or the trajectory.
✓training provenanceBoth models were trained from scratch (architecture.json base_model none). Sources: - Slakh2100-redux train split only.
Both models were trained from scratch (architecture.json base_model none). Sources: - Slakh2100-redux train split only. setup.sh downloads it from the HF mirror. Extraction used a '*/train/*' wildcard, and prep_slakh.py reads only the train dir. - Lakh MIDI lmd_full rendered with FluidSynth and public GM soundfonts (setup.sh, render_lmd.py, bootstrap2c.sh). - The task's public clips, used only for checkpoint selection and threshold calibration (tune_ens.py, DATA.md, trajectory#22). Operator policy allows this. The run2 log (logs/run2/train_run2.log) records the manifests and parameter counts. One minor inconsistency: the shipped train.py has no --ch/--hidden/--trunk args, but the run2 command used them. Even so, model_1.pt's tensor shapes match the logged run2 architecture. No validation/test split use was found.
✓actual inference uses reviewed weightsserver.py reads weights/config.json. ensemble.py builds a TranscriptionModel for each spec and loads weights/model_0.pt
server.py reads weights/config.json. ensemble.py builds a TranscriptionModel for each spec and loads weights/model_0.pt (default kwargs) and weights/model_1.pt (ch 48/96/128/192, hidden 512, trunk 1024) with torch.load(weights_only=True) and load_state_dict. The inventoried tensor shapes of both checkpoints match those architectures. tune_ens.py is the script that wrote these files and config.json from the run1/run2 checkpoints (final_ens.sh, extract_last.py, pack2.sh). There is no alternate model path or remote inference.
✓harness credentials used only by the harnessThe trajectory shows only Spin CLI / s3io usage of the participant's own s3://spin session and HF_TOKEN used for downloa
The trajectory shows only Spin CLI / s3io usage of the participant's own s3://spin session and HF_TOKEN used for downloads, both of which are allowed. No ANTHROPIC/OPENAI API keys or *_BASE_URL harness credentials were used by participant code, and no printenv dumps or direct model API calls were found. The validator-arg grep in trajectory#47 only surfaced text and did not use any credential.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 21m into the runtook 1m 9s
startup 4.6 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 4.6 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 11.0 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.869879028396681, "offset": 0.899879028396681, "pitch": 36, "instrument": "piano"}, {"onset": 12.11918497890234, "offset": 12.149184978902339, "pitch": 36, "instrument": "piano"}, {"onset": 17.1191342151165, "offset": 17.1491342151165, "p ...[clipped]
- ✓healthy after requests
- 2passed5h 5m into the runtook 1m 15s
startup 5.2 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 5.2 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 12.4 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.367793607711792, "offset": 3.3977936077117916, "pitch": 37, "instrument": "piano"}, {"onset": 3.0549160623550415, "offset": 3.35, "pitch": 38, "instrument": "piano"}, {"onset": 2.7448388126492502, "offset": 3.05, "pitch": 40, "instrument ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Everything is final: 42 files, 115.5 MB, selfcheck passed, both leases released ($5.52 of the $10 budget used). Submitting now.