Voice endpointing
- Score
- 55.8
- Wallet spend
- $34.90 of $200
- Model cost
- $28.77
- Agent time
- 5h 3m
- Steps
- 336
- Tokens in / out
- 100.1M / 320k
- Self-checks
- 4 passed of 4 used
- Rank at $200 wallet
- 3rd of 14
Score
55.8raw metric 0.8477
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.4965
- recall
- 0.8477
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 1291
- fn
- 232
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelThe serving path runs a learned model. serve.sh starts eot/server.py, which loads a frozen kyutai/mimi encoder from weig
The serving path runs a learned model. serve.sh starts eot/server.py, which loads a frozen kyutai/mimi encoder from weights/mimi. It then loads 22 trained EOTModel heads listed in weights/heads.json via torch.load and calls load_state_dict on each. The head0.pt inventory matches the EOTModel structure in eot/model.py: input projection, causal depthwise conv, 4 transformer blocks and output/aux heads. training/train.py contains the real training loop. It uses BCE/MIL losses on labels built from Seamless Interaction word timings (training/labels.py) and the same EOTModel class. No rule-based or hard-coded predictor replaces the model.
✓causal frame probabilitiesEvery part of the pipeline uses only past and present input. In eot/model.py, attention is masked with (j <= i) & (j > i
Every part of the pipeline uses only past and present input. In eot/model.py, attention is masked with (j <= i) & (j > i - window), the blockwise path uses the same mask, and the depthwise conv is left-padded by conv_k-1. In eot/mimi_feats.py, the resampling FIR is left-padded and causal, and the HF Mimi encoder uses causal convs and causal sliding-window attention. Chunked extraction adds left context only. energy_feats computes percentiles over past frames only (fr[a:t+1]). eot/server.py outputs one sigmoid probability per 0.08 s frame for each speaker and averages across heads. When features fall short of the frame count, it pads by repeating the last frame, which uses no future information.
✓no lookup of evaluation audioeot/server.py decodes the received audio and computes Mimi latents and energy features from it. It then runs the head en
eot/server.py decodes the received audio and computes Mimi latents and energy features from it. It then runs the head ensemble and returns the averaged sigmoids. I found no fingerprinting, hashing, stored timelines, label tables or id-keyed lookups in the serving code (eot/server.py, eot/model.py, eot/mimi_feats.py). The weights are model tensors only (weights/head0.pt inventory).
✓training provenanceTraining data is Seamless Interaction, train split only. training/si_prep.py keeps only rows with split == 'train' from
Training data is Seamless Interaction, train split only. training/si_prep.py keeps only rows with split == 'train' from the official filelist.csv. It downloads only {BASE}/{label}/train/audio and train/metadata/transcript files from Meta's public CDN, and data_scripts/download_seamless_interaction.sh retains the download commands. The si_val in-domain eval set is held out from those same train-split conversations (training/train.py --si_val). The 8 shipped conversations (/task/public/validation.jsonl) were used for selection, evaluation and cross-validation experiments (training/task_feats.py, training/train.py --train_task), which the task explicitly permits. I found no use of mundo-ai/turn-benchmark-dev, JSALT2026-Conv-AI-Simulator/turnbench-dev-no-backchannel, any held-out TurnBench ids, or any dev/test split. Mentions of dev/test in the trajectory appear only in README and listing text the participant read.
✓harness credentials used only by the harnessNo trajectory step reads, copies or uses the harness API keys or base URL, and none launches an extra harness CLI. Crede
No trajectory step reads, copies or uses the harness API keys or base URL, and none launches an extra harness CLI. Credential-related output is limited to spin --help text and a check that prints only 'HF_TOKEN set: yes'. Compute and storage went through spin (spin compute exec, spin s5cmd), which is allowed. Searches for ANTHROPIC, claude -p and OPENROUTER found no participant use.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 4 of 4.
- 1passed1h 31m into the runtook 3m 37s
startup 76.7 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 76.7 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.96211}, "speaker_2": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.96886}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.94967}, "speaker_2": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.97702}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 103 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 3.7 s) plus the other 19771 s at conversation 124's pace (617 s in 3.1 s, 198.2x real time). The two conversations alone: 190.3x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
- 2passed3h 20m into the runtook 2m 49s
startup 56.0 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 56.0 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.96615}, "speaker_2": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.96372}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.96693}, "speaker_2": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.96886}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 89 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 2.9 s) plus the other 19771 s at conversation 124's pace (617 s in 2.7 s, 230.7x real time). The two conversations alone: 232.6x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
- 3passed4h 1m into the runtook 3m 1s
startup 70.8 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 70.8 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.9703}, "speaker_2": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.96647}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.97035}, "speaker_2": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.95951}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 108 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 3.6 s) plus the other 19771 s at conversation 124's pace (617 s in 3.2 s, 189.9x real time). The two conversations alone: 191.3x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
- 4passed5h 6m into the runtook 3m 37s
startup 68.4 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 68.4 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.97257}, "speaker_2": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.96818}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.97027}, "speaker_2": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.96331}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 104 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 3.4 s) plus the other 19771 s at conversation 124's pace (617 s in 3.1 s, 196.7x real time). The two conversations alone: 199.3x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
All serving files are unchanged since the final passing selfcheck; only docs/logs were updated afterward. Stopping the GPU lease (prepaid, but nothing more is needed from it), then handing in.