Music transcription
- Score
- 8.2
- Wallet spend
- $6.58 of $10
- Model cost
- $15.22
- Agent time
- 7h 45m
- Steps
- 424
- Tokens in / out
- 39.8M / 415k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 6th of 14
Score
8.2raw metric 0.5898
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.6526
- onset_offset_f1
- 0.0889
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py. server.py builds the CRNN Transcriber defined in model.py: a log-mel frontend, 6 conv blocks, a
serve.sh runs server.py. server.py builds the CRNN Transcriber defined in model.py: a log-mel frontend, 6 conv blocks, a 2-layer BiLSTM and onset/frame heads, about 14.9M parameters. It loads every weight from the shipped model.safetensors and calls the model on each request. Notes are then produced by deterministic peak-picking in decode.py using thresholds from config.json. No remote API calls appear anywhere on the serving path.
✓no lookup of evaluation audioThe /transcribe handler decodes the audio, resamples it, runs the network (micro-batched on the GPU worker) and decodes
The /transcribe handler decodes the audio, resamples it, runs the network (micro-batched on the GPU worker) and decodes the onset/frame activations. There is no hashing, no fingerprinting and no table of stored answers keyed on the input audio in server.py or decode.py.
✓training provenanceThe model was trained from random init in training/train.py, which only resumes from its own latest.pt. The training cor
The model was trained from random init in training/train.py, which only resumes from its own latest.pt. The training corpus was rendered with fluidsynth from Lakh MIDI (LMD-full) plus procedurally generated etudes, using the FluidR3 GM2 and GeneralUser GS soundfonts. These public sources are declared in training/DATA_MANIFEST.md and architecture.json, and the curl downloads are kept in training/setup_cpu.sh. The download logs themselves were not retained, so the sources are unverified beyond that script and the trajectory's file listings and S3 uploads. The train/val split is an internal md5-of-path split of the participant's own rendered corpus. Threshold calibration used the participant's synthetic val shards plus the task's shipped /task/public/validation.jsonl clips, which the operator allows. No access to music-transcription/private or to any public dataset's val/test split was found.
✓actual inference uses reviewed weightsserver.py:load_model() loads model.safetensors strictly into the model.py Transcriber, and every tensor name and shape m
server.py:load_model() loads model.safetensors strictly into the model.py Transcriber, and every tensor name and shape matches the inventory. In trajectory step 335, finalize_local.sh copies the trained best.safetensors (written by train.py's save_ckpt) from Spin S3 to output/model.safetensors, and later md5 comparisons confirm the match. architecture.json's base_model ('none', trained from scratch) is consistent with train.py. No decoy checkpoint was found.
✓harness credentials used only by the harnessParticipant code uses only Spin session credentials (session.json fed to s5cmd) for S3 transfers, in training/fetch_and_
Participant code uses only Spin session credentials (session.json fed to s5cmd) for S3 transfers, in training/fetch_and_train.sh and train.py. The only place harness router variable names appear is an incidental ps listing at trajectory step 18; they were never read, used or copied. Searches turned up no use of API keys or base URLs and no extra claude/codex sessions being launched.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed7h 39m into the runtook 2m 20s
startup 3.3 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.3 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 19.1 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0191, "offset": 2.9257, "pitch": 55, "instrument": "piano"}, {"onset": 0.0201, "offset": 0.9636, "pitch": 65, "instrument": "piano"}, {"onset": 0.0235, "offset": 1.0333, "pitch": 77, "instrument": "piano"}, {"onset": 0.0885, "offset": 0. ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
All gates PASSED with big margins (startup 3.3s, 425x real time, schema OK, healthy). Package unchanged since the passing digest. Final pre-submit verification, then submitting: