Music transcription
- Score
- 61.9
- Wallet spend
- $7.39 of $10
- Model cost
- $98.13
- Agent time
- 9h 2m
- Steps
- 315
- Tokens in / out
- 95.1M / 252k
- Self-checks
- 3 passed of 3 used
- Rank at $10 wallet
- 2nd of 14
Score
61.9raw metric 0.8296
The raw metric (class-aware note-onset F1, averaged over 9 instrument classes) is rescaled so that the untouched YourMT3+ checkpoint (F1 0.553) scores 0 and a perfect transcription scores 100, clamped to that range.
Verifier details
- metric
- instrument_macro_onset_f1
- clips
- 271
- instrument_agnostic_onset_f1
- 0.8671
- onset_offset_f1
- 0.6180
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned transcription modelserve.sh installs the bundled einops wheel offline and then runs server.py. server.py builds FrameAMT from amt/model.py:
serve.sh installs the bundled einops wheel offline and then runs server.py. server.py builds FrameAMT from amt/model.py: a spectrogram, the Res3B pre-encoder, the PerceiverTF MoE encoder reused from the public YourMT3 checkpoint, a multi-channel projection, and a new TemporalHead that outputs onset and frame logits for 9 classes × 128 pitches. It loads two fine-tuned safetensors with strict=True and averages their logits in amt/infer.py. A learned neural model does the transcription. A public pretrained encoder and an ensemble are both allowed.
✓no lookup of evaluation audioThe /transcribe handler in server.py decodes the base64 FLAC, resamples it, and sends it through windowing, model infere
The /transcribe handler in server.py decodes the base64 FLAC, resamples it, and sends it through windowing, model inference, triangular stitching and peak-picking decoding in amt/infer.py. Decoding uses fixed thresholds from weights/decode_params.json. Neither the server nor the inference code hashes or fingerprints audio, keeps a stored-answer table, or makes any network call. The worker batches queued requests, which is allowed batch-aware processing.
✓training provenanceThe training audio was rendered with FluidSynth from Slakh2100-redux train-split MIDI. train/render_data.py uses --midi_
The training audio was rendered with FluidSynth from Slakh2100-redux train-split MIDI. train/render_data.py uses --midi_root .../slakh2100_flac_redux/train, and the trajectory shows only the train/ folder being tarred and rendered (trajectory/steps-90-105). The soundfonts are public GM/GS fonts, declared together with their download commands in data/DATA_MANIFEST.md and data/download_commands.sh. Thresholds and checkpoints were tuned and selected on the shipped public example clips (/task/public/validation.jsonl, read by train/train.py load_val and train/tune.py), which the operator allows. I found no use of the task's private/ folder or of any public dataset's validation or test split. Some public downloads are verified only by the retained commands and the session record, not by separate logs; I note this as unverified, not as a violation.
✓actual inference uses reviewed weightsserver.py reads weights/model_config.json, which lists model.safetensors and model_2.safetensors with the tf head. It lo
server.py reads weights/model_config.json, which lists model.safetensors and model_2.safetensors with the tf head. It loads each file with load_state_dict(strict=True) into FrameAMT and runs both in transcribe_batch. The tensor inventories contain the encoder.* and head.tf.* keys expected by the TemporalHead model, and each file is about 101.7 MB, which fits about 25.4M float32 parameters. In the trajectory, these files were produced by train/avg_ckpt.py and train/tune.py from run3 and run4 cycle-2 checkpoints and copied into output/weights (trajectory/steps-285-307). No weights come from outside the submission.
✓harness credentials used only by the harnessI searched the trajectory for ANTHROPIC, API_KEY, BASE_URL, 'claude -p', printenv and env dumps. Every hit is either Spi
I searched the trajectory for ANTHROPIC, API_KEY, BASE_URL, 'claude -p', printenv and env dumps. Every hit is either Spin documentation text or ordinary use of rented GPU boxes, S3 storage and HF downloads. Nothing shows the participant reading or using harness credentials. The trajectory cannot show whether the HF_TOKEN described as read-only at trajectory#1 was used, but HF_TOKEN downloads are allowed in any case.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed3h 6m into the runtook 1m 30s
startup 10.6 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 10.6 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 28.4 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0, "offset": 0.05, "pitch": 81, "instrument": "piano"}, {"onset": 0.0165, "offset": 0.2625, "pitch": 55, "instrument": "piano"}, {"onset": 0.0191, "offset": 0.0691, "pitch": 74, "instrument": "piano"}, {"onset": 0.0187, "offset": 0.0687, ...[clipped]
- ✓healthy after requests
- 2passed7h 36m into the runtook 1m 48s
startup 14.2 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 14.2 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 41.6 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.0131, "offset": 0.3188, "pitch": 55, "instrument": "piano"}, {"onset": 0.015, "offset": 0.2625, "pitch": 59, "instrument": "piano"}, {"onset": 0.0113, "offset": 0.3563, "pitch": 65, "instrument": "piano"}, {"onset": 0.0153, "offset": 0.1 ...[clipped]
- ✓healthy after requests
- 3passed8h 51m into the runtook 2m 20s
startup 14.8 s of 300 s; 1/1 gates passed; The timed pass repeated the 10 public examples under new IDs to make up the held-out split's 271 clips, so a server that caches by input measured faster here than it will on held-out clips.
6 gates
- ✓review prerequisites
- ✓submission size
- ✓startup · GET /health answered 200 after 14.8 s
- ✓real-time factor · 271 public clips (8130 s of audio, the held-out pass's size) sent 16 at a time in 42.7 s; grading allows 162.6 s.
- ✓POST /transcribe · {"id": "validation-0001", "notes": [{"onset": 0.014, "offset": 0.3188, "pitch": 55, "instrument": "piano"}, {"onset": 0.0152, "offset": 0.2625, "pitch": 59, "instrument": "piano"}, {"onset": 0.0103, "offset": 0.3188, "pitch": 65, "instrument": "piano"}, {"onset": 0.0151, "offset": 0.1 ...[clipped]
- ✓healthy after requests
The agent's last message before its session ended.
Everything matches the folder that passed the last selfcheck (94 files, 204.6 MB). Submitting.