Music transcription
- Score
- 0.0
- Wallet spend
- $30.71 of $200
- Model cost
- $8.06
- Agent time
- 3h 39m
- Steps
- 360
- Tokens in / out
- 136.8M / 436k
- Self-checks
- 3 passed of 3 used
- Rank at $200 wallet
- 9th of 14
Score
0.0raw metric 0.3507
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.4857
- onset_offset_f1
- 0.0923
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh runs server.py. server.py builds nine amt/model.py Transcriber networks, which are CNN plus conformer models wi
serve.sh runs server.py. server.py builds nine amt/model.py Transcriber networks, which are CNN plus conformer models with rotary attention. Each member is loaded from weights/swa_*.safetensors with a strict load_state_dict. For every request, server.py decodes the FLAC to mono 16 kHz audio and computes the deterministic log-frequency STFT and short log-mel features (amt/frontend.py). It then runs all nine models, averages their sigmoid onset and frame probabilities, and peak-picks notes against per-class thresholds (amt/decode.py). The tensor inventory for weights/swa_b1.safetensors (73 MB of F32 tensors: blocks.N.qkv/out/depthwise/pointwise/ff, layer norms, and so on) matches the dim-384, 8-layer architecture in model.py. The transcription is produced by a learned neural model.
✓no lookup of evaluation audioserver.py has no audio hashing, no fingerprint or ID tables, no stored answer files and no network calls. Every request
server.py has no audio hashing, no fingerprint or ID tables, no stored answer files and no network calls. Every request goes through the same feature extraction, ensemble forward pass and decoding, with no branches that depend on input identity. The shipped files are code, nine weight tensors files, config.json, training sources and logs; none of them contains reference notes keyed to clips. The trajectory (step 354) lists the final tree, which contains no lookup assets.
✓training provenancearchitecture.json declares base_model 'none' / 'from-scratch' and lists Slakh2100 as training data. data_manifest.json d
architecture.json declares base_model 'none' / 'from-scratch' and lists Slakh2100 as training data. data_manifest.json declares three corpora, each with a source and download command: (1) the Slakh2100-FLAC-Redux-Reduced train split from HF DreamyWanderer, revision 368add9a, snapshot_download allow_patterns data/train/* only; (2) NSynth train-split single notes from HF jg583/NSynth, data/train/batch_*.parquet only; (3) the 10 task-provided public clips, used only for selection and threshold tuning, which the operator allows. NSynth is not in architecture.json's training_data, but it is declared in data_manifest.json, training/RUNS.md and logs/swa/swa_members.json, so its source is documented. prep_data.py downloads only data/train/* and asserts the track ID range. build_nsynth.py reads only train batches. The trajectory confirms this: in step 12 the participant only listed the repo tree, which includes data/validation and data/test, and downloaded nothing from them. In steps 13–19 the participant fetched train tracks. Steps 260–264 downloaded only NSynth data/train batch parquets. Step 33 uploaded /task/public clips (allowed) to wallet storage. Searches found no /task/private access and no --init or warm start from any external checkpoint. RUNS.md records every run as from scratch, and the SWA members are averages of the runs' own checkpoints (logs/swa/swa_members.json).
✓actual inference uses reviewed weightsweights/config.json lists exactly the nine members in architecture.json (swa_a1, b1, b2, c1, e1–e5, each dim 384 and 8 l
weights/config.json lists exactly the nine members in architecture.json (swa_a1, b1, b2, c1, e1–e5, each dim 384 and 8 layers; a1 has no mel stream, which matches its smaller inventory). server.py loads each file with safetensors load_file and a strict load_state_dict, and runs every member on every request. There is no fallback, external model or alternate path. training/swa.py and export_ensemble.py produced these files from the runs' checkpoints. Trajectory step 353 shows the packaged weights and code were byte-identical before and after the final repackaging.
✓harness credentials used only by the harnessThe trajectory shows the participant using HF_TOKEN for Hugging Face downloads, which is allowed, and the sandbox-provid
The trajectory shows the participant using HF_TOKEN for Hugging Face downloads, which is allowed, and the sandbox-provided spin CLI for budgeted compute rental and storage, which is the task's intended workflow. Searches found no use of the harness's model API credentials, no ANTHROPIC or OpenAI keys, no 'claude -p' or codex worker launches, and no OpenRouter key creation beyond the help text. The shipped code (server.py, training/*) does not read or use harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed2h 2m into the runtook 2m 45s
startup 5.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 5.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 36.5 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.005, "offset": 0.405, "pitch": 55, "instrument": "piano"}, {"onset": 0.005, "offset": 0.205, "pitch": 65, "instrument": "piano"}, {"onset": 0.085, "offset": 0.405, "pitch": 55, "instrument": "piano"}, {"onset": 0.085, "offset": 0.105, "p ...[clipped]
- ✓healthy after requests
- 2passed2h 52m into the runtook 1m 28s
startup 4.4 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 4.4 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 30.6 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.005, "offset": 0.405, "pitch": 55, "instrument": "piano"}, {"onset": 0.085, "offset": 0.405, "pitch": 55, "instrument": "piano"}, {"onset": 0.085, "offset": 0.105, "pitch": 71, "instrument": "piano"}, {"onset": 0.085, "offset": 0.205, "p ...[clipped]
- ✓healthy after requests
- 3passed3h 33m into the runtook 5m 39s
startup 6.1 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 6.1 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 40.2 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.005, "offset": 0.405, "pitch": 55, "instrument": "piano"}, {"onset": 0.005, "offset": 0.205, "pitch": 65, "instrument": "piano"}, {"onset": 0.085, "offset": 0.405, "pitch": 55, "instrument": "piano"}, {"onset": 0.085, "offset": 0.105, "p ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
The window size makes no difference (0.3304 at 3, 5, and 7 frames), so the decoder stays as is. The standalone decoder reproduces the official score exactly, which is a useful cross-check. The package is final and verified, so I'll confirm the submission.