Voice endpointing
- Score
- 60.4
- Wallet spend
- $10.47 of $200
- Model cost
- $24.68
- Agent time
- 50m
- Steps
- 119
- Tokens in / out
- 16.5M / 105k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 1st of 14
Score
60.4raw metric 0.8634
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.7526
- recall
- 0.8634
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 1315
- fn
- 208
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelserver.py runs one call chain: Silero VAD, then energy and activity features (features.py), then frozen VAP and Smart Tu
server.py runs one call chain: Silero VAD, then energy and activity features (features.py), then frozen VAP and Smart Turn v3.2 features (neural_features.py, vap/*, smart_model.py), then Fusion. Fusion loads the trained 25->8->1 tanh MLP from models/fusion.npz, three seeds averaged (fusion.py). The weight inventory confirms the npz holds learned tensors: w1 [3,8,25], b1, w2 [3,1,8], b2, plus normalization stats. train_final.py writes those tensors after training on features from the public conversations, and logs/final_training.json records the run. The frozen pretrained models load from vap.pt and smart.pt, whose inventories show real encoder and transformer tensors. The trained model is what serving actually calls.
✓causal frame probabilitiesEvery feature for frame i uses only audio up to the end of that frame. The resampler is a causal lfilter FIR. Silero win
Every feature for frame i uses only audio up to the end of that frame. The resampler is a causal lfilter FIR. Silero windows use only left context. Rolling, since and lag features look only at the past (features.py). The VAP CPC encoder's receptive field ends inside the frame. Its CConv1d is causal and its LSTM runs forward (vap/encoder_components.py). The transformer gets a causal tril mask (vap/modules.py). The VAP frame mapping ix = floor((m+1)*0.032/0.02)-1 never reads ahead, and the chunked runs keep only the trailing frames (neural_features.py). Smart Turn sees an 8s prefix ending at (i+1)*512, and its mel normalization is prefix-only (smart_model.py). Fusion output is a per-frame function of these features. Padding repeats the last computed frame, so no future data enters.
✓no lookup of evaluation audioserver.py computes predictions only from the incoming audio, using the models and features described above. Nothing in i
server.py computes predictions only from the incoming audio, using the models and features described above. Nothing in it hashes the input, looks up stored labels, or ships per-conversation answers. fusion.npz holds only the small MLP weights and normalization stats (weight inventory). features.py and fusion.py contain no fingerprinting or answer tables.
✓training provenanceTraining scripts (prepare_data.py, extract_features.py, train.py, training_mil.py, train_final.py) read only /task/publi
Training scripts (prepare_data.py, extract_features.py, train.py, training_mil.py, train_final.py) read only /task/public/validation.jsonl and the cache built from it. data_manifest.json lists only the 8 allowed public ids (20, 21, 22, 94, 95, 122, 124, 125) with sha256 hashes, and declares no public val/test splits. Searches of all evidence, trajectory included, found no reference to the forbidden mundo-ai or JSALT TurnBench dev datasets, and no held-out-id hits. Pretrained models have declared, pinned sources: ErikEkstedt/VoiceActivityProjection, pipecat smart-turn-v3@f766f81 and silero-vad. download_models.py retains the fetch commands, and trajectory steps 9 and 11 record the git clones and the HF metadata queries.
✓harness credentials used only by the harnessThe trajectory has no step where participant code reads, copies or exports harness API keys or base URLs, and none that
The trajectory has no step where participant code reads, copies or exports harness API keys or base URLs, and none that launches an extra harness CLI. Searches for API_KEY, BASE_URL and codex exec returned no hits. The only OpenRouter mention is spin --help output (trajectory#6). Package installs and repo clones (trajectory#9) used no harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed53m 27s into the runtook 3m 53s
startup 54.5 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 54.5 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.032, "frames": 21377, "peak": 0.9975037574768066}, "speaker_2": {"hop_seconds": 0.032, "frames": 21377, "peak": 0.9978918433189392}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.032, "frames": 19275, "peak": 0.9973302483558655}, "speaker_2": {"hop_seconds": 0.032, "frames": 19275, "peak": 0.9974351525306702}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 282 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 11.7 s) plus the other 19771 s at conversation 124's pace (617 s in 8.4 s, 73.1x real time). The two conversations alone: 64.8x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
Submitted /workspace/output/ with serve.sh, trained weights, and reproducible training code.
- Self-check passed on an offline H100.
- Startup: 54.5 seconds.
- Processing: 64.8× real time.
- Provided examples: 80.6% recall at 4.9% false positives.
- Causality: prefix predictions matched exactly.