Voice endpointing
- Score
- 29.0
- Wallet spend
- $6.98 of $10
- Model cost
- $1.04
- Agent time
- 1h 23m
- Steps
- 180
- Tokens in / out
- 13.2M / 50k
- Self-checks
- 1 passed of 2 used
- Rank at $10 wallet
- 14th of 19
Score
29.0raw metric 0.7551
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.9912
- recall
- 0.7551
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 1150
- fn
- 373
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelThe served scores come from a pretrained learned model: the anyreach-ai/dualturn-endpointing (DualTurn) readout on top o
The served scores come from a pretrained learned model: the anyreach-ai/dualturn-endpointing (DualTurn) readout on top of a frozen Kyutai Mimi encoder. app.py load_model() calls AutoModel.from_pretrained on models/anyreach with config.mimi_model pointing at models/mimi. predict_one runs the model on the resampled two-channel audio. The final score is clip(max(eot_probs, agent fvad horizon 0)), a simple post-processing step on the model outputs. The trajectory shows the weights were obtained by snapshot_download of the declared HF repos, and the self-check log shows both checkpoints (30 and 350 tensors) loading in the serving process. No fine-tuning was claimed or performed. The architecture is a learned causal network: Mimi causal convolutions with a causal sliding-window transformer, then Linear+LayerNorm, a unidirectional LSTM and MLP heads.
✓causal frame probabilitiesEach frame depends only on audio up to the end of that frame, so this check holds. Audio is resampled from 48k to 24k wi
Each frame depends only on audio up to the end of that frame, so this check holds. Audio is resampled from 48k to 24k with torchaudio.functional.resample, a windowed-sinc anti-aliasing filter well under the 5 ms tolerance. In modeling_dualturn.py the Mimi encoder uses causal convolutions, and the encoder transformer is used with HF Mimi's causal sliding-window mask. The downsample to 12.5 Hz (80 ms hop) is causal, and the readout LSTM is nn.LSTM(bidirectional=False). Per-frame heads produce eot and fvad, and the post-processing max(eot, fvad_agent_h0) is pointwise per frame. Outputs are only truncated or zero-padded to ceil(dur/0.08). Running each speaker as the user channel does not add any future context.
✓no lookup of evaluation audioNo lookup of evaluation audio was found. app.py has no hashing, fingerprinting, ID matching, embedded label tables or st
No lookup of evaluation audio was found. app.py has no hashing, fingerprinting, ID matching, embedded label tables or stored predictions. Every request is computed by running the model on the submitted audio. The turnbench GitHub clone in trajectory#113 contains baseline prediction files, but these were only listed or read with cat and never copied into the output or used. The shipped code reads no prediction file.
✓training provenanceThe participant trained no weights; the shipped checkpoints are the unmodified public anyreach-ai/dualturn-endpointing a
The participant trained no weights; the shipped checkpoints are the unmodified public anyreach-ai/dualturn-endpointing and kyutai/mimi downloads. data_manifest.json and the model README declare the upstream training data as Switchboard, OtoSpeech and a private agent-call corpus, so the source is declared even though it cannot be verified. The post-processing combination was selected only on the 8 shipped public conversations (ids 20, 21, 22, 94, 95, 122, 124, 125), which is allowed (trajectory#133, trajectory#142, trajectory#144; experiments.log). Several attempts to reach forbidden data all failed and obtained nothing: load_dataset and file listing on mundo-ai/turn-benchmark-dev (trajectory#106, #107, #109) and turn-benchmark-test access (trajectory#110, #111), which hit gated/403 errors. Nothing from these sources is shipped or used. Browsing otoSpeech-full-duplex-turn-104h metadata (the allowed training set) led to no training. No held-out ids appear.
✓harness credentials used only by the harnessNo trajectory step shows harness credentials being used outside the harness. The participant used HF_TOKEN for model and
No trajectory step shows harness credentials being used outside the harness. The participant used HF_TOKEN for model and dataset downloads, including on Spin machines (trajectory#69, trajectory#122), which policy allows. No direct calls to model APIs with harness keys (Anthropic/OpenAI/OpenRouter keys) appear; the only 'openrouter' matches are the Spin CLI help text and trajectory metadata.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1failed1h 8m into the runtook 8m 54s
failed startup: Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: Processing ./wheels/regex-2026.9.29-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylin
2 gates
- ✓review prerequisites
- ✗startup · Server did not become healthy in 300s (last /health probe: RuntimeError: Submission HTTP request failed: ConnectionRefusedError: [Errno 111] Connection refused); end of its log: Processing ./wheels/regex-2026.9.29-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (from transformers) Processing ./wheels/tokenizers-0.23.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (from transformers) INFO: pip is looking at multiple versions of transformers to determine which version is compatible with other requirements. This could take a while. ERROR: Could not find a version that satisfies the requirement typer (from transformers) (from versions: none) ERROR: No matching distribution found for typer
- 2passed1h 17m into the runtook 5m 57s
startup 81.7 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 81.7 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.9999988079071045}, "speaker_2": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.9999945163726807}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.999991774559021}, "speaker_2": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.9999852180480957}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 148 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 4.7 s) plus the other 19771 s at conversation 124's pace (617 s in 4.5 s, 138.1x real time). The two conversations alone: 142.5x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
Submitted. The solution uses the pretrained DualTurn endpointing model (anyreach-ai/dualturn-endpointing) with a post-processing signal max(eot, agent FVAD 0–240ms), selected on the provided validation set, served via PyTorch on H100. Self-check passed (startup 81.7s, 3x real-time gate passed).