Music transcription
- Score
- 35.1
- Wallet spend
- $13.96 of $50
- Model cost
- $7.63
- Agent time
- 3h 8m
- Steps
- 232
- Tokens in / out
- 24.3M / 299k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 5th of 14
Score
35.1raw metric 0.7097
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.7610
- onset_offset_f1
- 0.5393
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh installs local wheels and runs server.py. server.py builds a Transcriber that loads FrameModel checkpoints (fra
serve.sh installs local wheels and runs server.py. server.py builds a Transcriber that loads FrameModel checkpoints (frame_model.py): a vendored YourMT3 Perceiver-TF MoE encoder plus a conv onset/frame head. inference.py runs windowed forward passes and combines the ensemble's sigmoid outputs, fast_moe.py packs the dense MoE weights, and postprocess.py turns onset and frame probabilities into notes by peak-picking. The transcription comes from a learned neural model.
✓no lookup of evaluation audioserver.py decodes the base64 FLAC request, checks the sample rate and length, and passes the audio straight into model i
server.py decodes the base64 FLAC request, checks the sample rate and length, and passes the audio straight into model inference and postprocessing. The serving path contains no hashing, fingerprinting, stored-label table, transcript cache or remote call. inference.py and postprocess.py are purely numerical. No access to music-transcription/private was found in the code or the trajectory.
✓training provenanceInitialization is the published YourMT3 mc13 checkpoint at pinned revision e45ebd70; its URL and sha256 are recorded in
Initialization is the published YourMT3 mc13 checkpoint at pinned revision e45ebd70; its URL and sha256 are recorded in evidence/pretrained_downloads.json, and training/train_frame.py keeps only the encoder-side weights. The training data comes from three sources. (1) Procedural FluidSynth renders using the public FluidR3_GM, TimGM6mb and GeneralUser soundfonts (training/synthetic_data_v3.py, evidence/data_manifest.json). (2) The 10 supplied public examples, which the 2026-09-29 operator decision allows for training, calibration and selection. (3) The Slakh2100-redux train split from HF J1mmymm/MIMuT_Data_v2. training/prepare_slakh_train.py restricts every download to data/Slakh2100_redux/train/ and asserts it, synthetic_data_v4.py asserts split=='train', and the preparation logs list only train records. In the trajectory, Slakh validation/test paths appear only in a directory listing, and the participant notes state that no validation/test split was used. Calibration and selection use only the supplied examples (training/final_calibration.sh).
✓actual inference uses reviewed weightsinference_config.json lists model_early.pt, model_phase1.pt, model_aug.pt, model_robust.pt and model_natural.pt, each wi
inference_config.json lists model_early.pt, model_phase1.pt, model_aug.pt, model_robust.pt and model_natural.pt, each with nonzero class weights. server.py loads every listed path with FrameModel.load (torch.load of the config and state_dict, frame_model.py) and serves predictions from those modules. Weight inventories exist for all five shipped files. training/finalize_artifacts.py removes unlisted checkpoints, and the unused muscriptor-small download is never loaded at serve time.
✓harness credentials used only by the harnessTrajectory searches found no use, copying or echoing of harness credentials (API_KEY, OPENAI_API_KEY, BASE_URL, setup.js
Trajectory searches found no use, copying or echoing of harness credentials (API_KEY, OPENAI_API_KEY, BASE_URL, setup.json) and no launch of any extra harness CLI such as codex exec. 'openrouter' appears only in spin help text. HF_TOKEN is used only for Hugging Face downloads in training/prepare_slakh_train.py, which the policy allows.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed30m 12s into the runtook 2m 49s
startup 14.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 14.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 23.4 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.09375, "offset": 0.39375, "pitch": 77, "instrument": "piano"}, {"onset": 0.39375, "offset": 0.5625, "pitch": 77, "instrument": "piano"}, {"onset": 0.5625, "offset": 0.9, "pitch": 77, "instrument": "piano"}, {"onset": 4.78125, "offset": 4 ...[clipped]
- ✓healthy after requests
- 2passed3h 5m into the runtook 3m 18s
startup 18.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 18.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 80.0 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.09375, "offset": 0.39375, "pitch": 77, "instrument": "piano"}, {"onset": 0.39375, "offset": 0.73125, "pitch": 77, "instrument": "piano"}, {"onset": 4.78125, "offset": 4.95, "pitch": 77, "instrument": "piano"}, {"onset": 5.08125, "offset" ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with trained weights and serve.sh.
- Supplied-example macro onset F1: 97.28%
- Final offline self-check: passed
- Startup: 18.8 seconds
- Transcription speed: 101.7× real time
- Total submission size: 258.3 MB
Training, calibration, and data-provenance evidence are included.