Voice endpointing
- Score
- 62.9
- Wallet spend
- $13.96 of $50
- Model cost
- $20.62
- Agent time
- 37m
- Steps
- 108
- Tokens in / out
- 11.9M / 98k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 2nd of 14
Score
62.9raw metric 0.872
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.6090
- recall
- 0.8720
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 1328
- fn
- 195
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelserver.py loads a trained 9-member MLP ensemble head from models/head.json. It applies the head per speaker to the conca
server.py loads a trained 9-member MLP ensemble head from models/head.json. It applies the head per speaker to the concatenated features: Silero VAD history, VAP outputs and embeddings, and Smart Turn v2 prefix outputs. export_head.py, train_mil.py and train_embed.py fit the head with a window-supervised MIL loss and leave-speaker-pair-out cross-validation, then refit on the 8 provided examples. logs/final_fit.log records the member configurations and the CV metrics. The frozen backbones are declared pretrained checkpoints with pinned revisions in architecture.json.
✓causal frame probabilitiesEvery feature path uses only past or current audio. Resampling (resample_causal) is a one-sided firwin plus lfilter, and
Every feature path uses only past or current audio. Resampling (resample_causal) is a one-sided firwin plus lfilter, and the Silero ONNX state is carried forward over 32 ms chunks. history_features uses cumulative past statistics indexed at each frame's right edge. VAP runs in fixed blocks with left context only and the frame index floor((k+1)·0.064/0.02)−1. The CPC convolutions are causal, ChannelNorm normalizes across channels only, and the ALiBi attention mask is causal, including in the stereo cross-attention. Smart Turn clips end at the current frame boundary, are normalized on their own and padded to a fixed length; their outputs are forward-filled from past evaluations only. The head is a per-frame MLP, so the output at frame k depends only on audio up to frame k.
✓no lookup of evaluation audioserver.py and features.py compute predictions only from the incoming audio through the model pipeline. No code hashes, f
server.py and features.py compute predictions only from the incoming audio through the model pipeline. No code hashes, fingerprints or matches audio against stored labels or examples, and no per-id tables ship with the submission. The CV out-of-fold logs are diagnostic artifacts and are not loaded by the server.
✓training provenancedata_manifest.json lists only the 8 permitted ids (20, 21, 22, 94, 95, 122, 124, 125) with hashes, and external_corpus_d
data_manifest.json lists only the 8 permitted ids (20, 21, 22, 94, 95, 122, 124, 125) with hashes, and external_corpus_downloads is empty. download_data.sh only copies /task/public with hash verification, and prepare.py and train.py read only those records. The pretrained VAP, Smart Turn v2 and Silero checkpoints are declared with revisions. In the trajectory, model downloads were limited to public model repos (faster-whisper, livekit/turn-detector, parakeet, smart-turn) used for non-serving experiments, and the participant's /task/validate.py runs used only the provided samples. No forbidden turnbench dataset or public validation/test split was accessed. The public model-card sources are recorded but were not independently verified.
✓harness credentials used only by the harnessThe trajectory shows the GPU was leased through the permitted spin compute CLI and reached over SSH with the key spin re
The trajectory shows the GPU was leased through the permitted spin compute CLI and reached over SSH with the key spin returned (trajectory#17). No harness API keys or base URLs were read or copied, and no other harness CLI was launched: searches for API_KEY, BASE_URL, OPENAI, ANTHROPIC, codex exec and printenv found no matches. OpenRouter appears only in the spin --help output (trajectory#6).
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed14m 57s into the runtook 5m 24s
startup 72.6 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 72.6 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.064, "frames": 10689, "peak": 0.9823338389396667}, "speaker_2": {"hop_seconds": 0.064, "frames": 10689, "peak": 0.9849593639373779}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.064, "frames": 9638, "peak": 0.9868594408035278}, "speaker_2": {"hop_seconds": 0.064, "frames": 9638, "peak": 0.9894375205039978}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 487 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 17.9 s) plus the other 19771 s at conversation 124's pace (617 s in 14.6 s, 42.1x real time). The two conversations alone: 40.0x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
- 2passed34m 37s into the runtook 2m 54s
startup 54.8 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 54.8 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.064, "frames": 10689, "peak": 0.9902400374412537}, "speaker_2": {"hop_seconds": 0.064, "frames": 10689, "peak": 0.9900921583175659}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.064, "frames": 9638, "peak": 0.9910432696342468}, "speaker_2": {"hop_seconds": 0.064, "frames": 9638, "peak": 0.9917741417884827}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 458 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 16.7 s) plus the other 19771 s at conversation 124's pace (617 s in 13.8 s, 44.8x real time). The two conversations alone: 42.7x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/.
- Offline self-check passed: 54.8-second startup, 44.7× real time.
- All 8 HTTP validation requests and 14 causality checks passed.
- Speaker-pair cross-validation achieved 76.9% recall at 4.92% pause false positives.