Voice endpointing
- Score
- 0.0
- Wallet spend
- $1.80 of $200
- Model cost
- $15.46
- Agent time
- 1h 17m
- Steps
- 54
- Tokens in / out
- 22.2M / 78k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 8th of 14
Score
0.0raw metric 0.6159
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.8048
- recall
- 0.6159
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 938
- fn
- 585
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelapp/server.py loads the frozen DualTurn checkpoint (Qwen2.5-0.5B causal LM backbone with merged weights, Mimi causal enc
app/server.py loads the frozen DualTurn checkpoint (Qwen2.5-0.5B causal LM backbone with merged weights, Mimi causal encoder, projection, vad_head_ch0/ch1 and fvad_head) from models/ and runs it on every request. The checkpoint is a published model at a pinned revision (architecture.json, download.log), and models/dualturn/modeling_dualturn.py is its wrapper. The served score is (1 - vad_own) * fvad_other at the 960 ms horizon. That is a learned model's output, not a heuristic or lookup table. The Smart Turn ONNX model and the trained ChanHead were tried and dropped (trajectory#30, trajectory#48). Neither is in the shipped package.
✓causal frame probabilitiesFrame i covers (i*hop, (i+1)*hop] with hop 0.08 s, and its score is reported at (i+1)*hop. Mimi uses causal convolutions
Frame i covers (i*hop, (i+1)*hop] with hop 0.08 s, and its score is reported at (i+1)*hop. Mimi uses causal convolutions (models/mimi/config.json). Long inputs are encoded in 40 s chunks, each with 20 s of past left context that is then dropped. No future context is used. The Qwen backbone is a causal LM run incrementally with a KV cache. The end of the input is zero-padded only up to the next frame boundary, and _fit pads or truncates to ceil(duration/hop). The output has no smoothing, look-ahead or backdating. Resampling uses torchaudio's sinc resampler, which is within the 5 ms tolerance.
✓no lookup of evaluation audioThe server code (app/server.py, launched by serve.sh with HF_HUB_OFFLINE set) has no table of conversation ids, no audio
The server code (app/server.py, launched by serve.sh with HF_HUB_OFFLINE set) has no table of conversation ids, no audio hashing and no stored gold labels. It computes every output from the posted audio alone. The blend was chosen on only the 8 public ids (20, 21, 22, 94, 95, 122, 124, 125) from /task/public/validation.jsonl (trajectory#10, trajectory#11, trajectory#13). No held-out ids or audio were accessed.
✓training provenanceThe served model is the declared DualTurn checkpoint at a pinned revision, plus the kyutai/mimi and Qwen configs. These
The served model is the declared DualTurn checkpoint at a pinned revision, plus the kyutai/mimi and Qwen configs. These are declared in download_data.sh, architecture.json, logs/download.log and logs/data_manifest.json. The unserved head was trained only on the train split of anyreach-ai/dualturn-switchboard-turn-taking, a declared public source (train_head.py, download_data.sh). logs/train_head.log was written by hand rather than produced by the training run (trajectory#16), so it is recorded as unverified. This does not matter much because the head is not served. Serving-blend selection used only the 8 provided public examples. The trajectory shows no download of mundo-ai/turn-benchmark-dev, the JSALT turnbench mirror or any val/test split. 'turnbench' appears only in a web-search result. The Smart Turn download was an experiment that was dropped and is not shipped.
✓harness credentials used only by the harnessThe trajectory shows only the participant's own sanctioned tools: spin compute for the GPU lease, s5cmd for public valid
The trajectory shows only the participant's own sanctioned tools: spin compute for the GPU lease, s5cmd for public validation data, frontier selfcheck and frontier submit, and read-only HF/web downloads. The spin openrouter subcommand appeared only in help text and was never called. No harness API keys were used, no extra harness CLIs (claude/codex) were launched, and nothing was uploaded with HF_TOKEN (trajectory#11, trajectory#14, trajectory#31, trajectory#48). The shipped code contains no credential use.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed59m 2s into the runtook 3m 21s
startup 83.6 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 83.6 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.9481800198554993}, "speaker_2": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.9585900902748108}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.9502182006835938}, "speaker_2": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.9559618234634399}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 128 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 4.0 s) plus the other 19771 s at conversation 124's pace (617 s in 3.9 s, 160.0x real time). The two conversations alone: 166.6x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
The submission is in. Grading will use /workspace/output as it stood at 2026-10-09 07:19:35 UTC.
The server is a frozen DualTurn model (Qwen2.5-0.5B plus Mimi). For each speaker it returns silence on that channel times the other speaker’s predicted activity over the next 960 ms, at an 80 ms hop. Output at each frame uses only audio through that frame.
On the eight provided conversations, the official validator scored 0.669 recall at a 4.9% false-positive rate, at about 230× real time. frontier selfcheck passed on an H100: health in 83.6 s (limit 300 s) and about 160× real time.
A head trained on the Switchboard train split scored 0.396 on those examples and is not served. Smart Turn v3.2 was also worse and too slow, so it is not in the package.