All tasks
Task · end-of-turn

Voice endpointing

Detect when a speaker has finished talking so voice assistants reply promptly without interrupting

Vision & AudioInspired byThinking Machines
Score
windowed detection score on held-out conversations
Best
74.7
Median over trials
32.8

Instruction

Train a model that listens to a two-person conversation and predicts when each speaker has finished their turn, as opposed to pausing or hesitating mid-turn. I have provided 8 labeled example conversations.

The model has to be causal: its output at any moment can only depend on audio up to that moment.

Write a script serve.sh which serves the model. It should have a POST /predict endpoint which takes both speakers' audio and returns, for each speaker, a per-frame probability that their turn just ended. It should run offline on one H100, be ready within 300 seconds, and process audio at least 3x faster than real time.

Optimize end-of-turn recall at a 5% false-positive rate on mid-turn pauses.

Do not train, fine-tune, calibrate or select on the validation or test split of any public dataset or benchmark; training splits and other public corpora are fine. However, you are free to use the validation data samples I have provided however you want.

Submit in /workspace/output/. Interface and runtime details: /task/CONTRACT.md. Supporting files: /task/public/. Evaluate with python /task/validate.py --server-url URL. Test the finished submission the way it will be run with frontier selfcheck. Hand it in with frontier submit when it is final. Use spin --help for compute and model APIs.

Trials

42 graded · click a score for the trajectory
Model$10 wallet$50 wallet$200 wallet
Claude Opus 5.5xhigh74.768.855.8
Claude Sonnet 5.5xhigh49.446.157.4
Claude Fable 5.1xhigh65.00.050.7
Claude Haiku 5.5max33.63.719.9
GPT-6 Astraxhigh50.562.960.4
GPT-6.1 Solxhigh45.80.042.3
GPT-6 Lunaxhigh29.032.443.7
Gemini 3.8 Flashhigh40.00.00.0
Grok 4.7xhigh43.30.00.0
Muse Spark 1.3max34.10.00.0
Muse Wondermax0.05.20.0
Kimi K3max0.00.00.0
Qwen3.8 Maxxhigh50.533.20.0
GLM 5.3max40.80.00.0

Scoring

The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.

MetricScores 0 atScores 100 at
recall_at_5pct_fpr0.6551