Music transcription
- Score
- 49.4
- Wallet spend
- $7.04 of $10
- Model cost
- $58.25
- Agent time
- 2h 13m
- Steps
- 295
- Tokens in / out
- 35.5M / 277k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 4th of 14
Score
49.4raw metric 0.7739
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.8290
- onset_offset_f1
- 0.4451
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py with offline environment variables set. server.py decodes the received FLAC, resamples it with a
serve.sh runs server.py with offline environment variables set. server.py decodes the received FLAC, resamples it with a fixed FIR filter and runs four learned residual CNN onset/frame networks (net.Net or net2.Net) on the audio. It combines their logits with per-class weights in an ensemble, then decodes notes by peak picking and thresholds. The network weights are loaded from shipped .pt checkpoints. There are no remote model calls, and pre- and post-processing are deterministic.
✓no lookup of evaluation audioserver.py computes predictions only from network outputs and the decode() thresholds stored in config.json. There is no
server.py computes predictions only from network outputs and the decode() thresholds stored in config.json. There is no hashing or fingerprinting of incoming audio, no stored note lists or answer tables, and no network access (only standard library, numpy, soundfile and torch are imported). net.py and net2.py contain only model code with no I/O.
✓training provenanceTraining data is procedurally rendered music from General MIDI soundfonts (FluidR3 GM, TimGM6mb, GeneralUser GS, MuseSco
Training data is procedurally rendered music from General MIDI soundfonts (FluidR3 GM, TimGM6mb, GeneralUser GS, MuseScore General, SGM). These are declared with public download URLs and hashes in evidence/download_fonts.json and evidence/download_extra_fonts.json. Training also uses the task's supplied public examples (/task/public/validation.jsonl), which the policy allows. Development and selection corpora are freshly generated procedural clips. training/train_net.py starts from random initialization (seed 137). Later stages chain only from the participant's own checkpoints (see training_chain.json, train_net10.py, snapshot_ema.py and average_weights.py). The trajectory shows the participant looking at pretrained models such as YourMT3 and MuScriptor and fetching a config file, but none were used in training. There is no access to the dataset's private/ folder and no use of any public validation or test split.
✓actual inference uses reviewed weightsconfig.json lists exactly net10_ema_10000.pt, net5_snapshot_ema_30000.pt, net7_snapshot_ema_20000.pt and net9_swa_6000.p
config.json lists exactly net10_ema_10000.pt, net5_snapshot_ema_30000.pt, net7_snapshot_ema_20000.pt and net9_swa_6000.pt. server.py runs torch.load on each listed file, builds net or net2 at the checkpoint's width, calls load_state_dict and runs inference with those models. The weight inventories match the net2 architecture: the stem, five blocks, an 18-channel head and the presence MLP, at width 192 for net10 and width 96 for the other three (presence weight shapes 384x1152 and 192x576). This is consistent with the declared from-scratch lineage.
✓harness credentials used only by the harnessThe trajectory shows no participant command that used the harness model-API credentials or base URL, and no extra harnes
The trajectory shows no participant command that used the harness model-API credentials or base URL, and no extra harness CLI sessions (no codex exec or similar). The only credential-related command is a final rg scan of the output folder for secret strings. That is a leak check, not a use of credentials. Compute ran on leased GPUs over SSH, and HF downloads were used only for metadata and exploration, which is permitted.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed25m 15s into the runtook 1m 46s
startup 3.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 3.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 18.0 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37234, "offset": 4.05079, "pitch": 37, "instrument": "piano"}, {"onset": 3.05306, "offset": 3.37234, "pitch": 38, "instrument": "piano"}, {"onset": 2.74376, "offset": 3.35238, "pitch": 40, "instrument": "piano"}, {"onset": 8.06168, "offs ...[clipped]
- ✓healthy after requests
- 2passed1h 13m into the runtook 1m 38s
startup 6.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 6.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 37.5 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37234, "offset": 3.77143, "pitch": 37, "instrument": "piano"}, {"onset": 3.06304, "offset": 3.38231, "pitch": 38, "instrument": "piano"}, {"onset": 2.75374, "offset": 3.20272, "pitch": 40, "instrument": "piano"}, {"onset": 8.06168, "offs ...[clipped]
- ✓healthy after requests
- 3passed2h 11m into the runtook 2m 47s
startup 6.0 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 6.0 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 36.8 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 3.37234, "offset": 3.72154, "pitch": 37, "instrument": "piano"}, {"onset": 3.06304, "offset": 3.38231, "pitch": 38, "instrument": "piano"}, {"onset": 2.74376, "offset": 3.07302, "pitch": 40, "instrument": "piano"}, {"onset": 8.06168, "offs ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/.
- Offline H100 self-check passed: 6.0 s startup, 221× real time
- Total size: 76.9 MB
- Supplied-clip onset F1: 0.9882; these clips were used in training and calibration.
Includes serve.sh, weights, training source, logs, and data provenance.