Voice endpointing
- Score
- 68.8
- Wallet spend
- $6.73 of $50
- Model cost
- $52.64
- Agent time
- 4h 59m
- Steps
- 250
- Tokens in / out
- 70.5M / 282k
- Self-checks
- 3 passed of 3 used
- Rank at $50 wallet
- 1st of 14
Score
68.8raw metric 0.8923
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.7424
- recall
- 0.8923
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 1359
- fn
- 164
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelserve.py loads models/eot_*.pt, which are 14 EOTModel checkpoints (two-stream causal transformer with is_causal SDPA, se
serve.py loads models/eot_*.pt, which are 14 EOTModel checkpoints (two-stream causal transformer with is_causal SDPA, see eotlib/model.py), and averages their stream_probs outputs. The weight inventories (for example models/eot_00.pt: in_self/in_other projections, blocks.*.qkv/ff, mu/sd) match the EOTModel layout. train_src/train.py builds these checkpoints with AdamW, BCE end-of-turn loss and an auxiliary VAD loss on precomputed frozen features. models/MEMBERS.txt maps each member to a training run and checkpoint step. The frozen front-ends (Mimi, FastConformer, Qwen2.5-0.5B) are pretrained feature extractors with shipped weights.
✓causal frame probabilitiesEvery feature path is causal. Mimi chunking and the FIR/energy features use left padding only (eotlib/features.py). The
Every feature path is causal. Mimi chunking and the FIR/energy features use left padding only (eotlib/features.py). The FastConformer uses chunked_limited attention with context [70,0] and causal convolutions, and its grid mapping reports encoder frame n no earlier than the audio it depends on (eotlib/fastconformer.py). Lexical features release CTC words only after the next word starts or a blank gap, and take the causal LM state at or before each frame (eotlib/lexical.py). The predictor uses causal attention over past-only windows (eotlib/model.py). logs/causality_and_robustness_tests.txt reports a max diff of 0 for frames up to T when the audio after T is replaced with noise.
✓no lookup of evaluation audioserve.py decodes the posted audio, computes features, runs the ensemble and pads/truncates the output to ceil(duration/0
serve.py decodes the posted audio, computes features, runs the ensemble and pads/truncates the output to ceil(duration/0.04). It does no hashing, filename or id matching, and contains no stored per-conversation outputs. The fallback returns zeros only on error. No held-out TurnBench ids or audio appear in the serving path.
✓training provenanceTraining data is otoearth/otoSpeech-full-duplex-turn-104h at a pinned revision. It is declared in architecture.json and
Training data is otoearth/otoSpeech-full-duplex-turn-104h at a pinned revision. It is declared in architecture.json and DATA_MANIFEST.md, and data_scripts/download_oto.sh retains the download command. train_src/train.py reads oto SRT labels from /data/oto and uses only the shipped 8 from /task/public/validation.jsonl (copied to /data/val) for selection and optional fitting, which is allowed. The trajectory shows the oto download and metadata checks, plus actor-disjointness (0 overlap) and negligible 4-gram overlap with the shipped 8. Oto folder numbers are that corpus's own ids, not TurnBench ids. There is no evidence that the TurnBench dev/test release or any mirror was downloaded.
✓harness credentials used only by the harnessThe session searches show no use of ANTHROPIC/OPENAI API keys or base URLs by participant code, and no extra claude/CLI
The session searches show no use of ANTHROPIC/OPENAI API keys or base URLs by participant code, and no extra claude/CLI harness sessions. BASE_URL/API_KEY strings appear only in spin help text and validate.py's --judge-key-env help output. The participant used spin compute, upload, s5cmd and HF_TOKEN-based downloads, all of which are permitted.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.
- 1passed1h 35m into the runtook 4m 57s
startup 28.3 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 28.3 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.997368335723877}, "speaker_2": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.997285008430481}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.9966233968734741}, "speaker_2": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.9969245791435242}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 79 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 2.8 s) plus the other 19771 s at conversation 124's pace (617 s in 2.4 s, 258.9x real time). The two conversations alone: 252.2x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
- 2passed3h 54m into the runtook 2m 15s
startup 22.0 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 22.0 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.997691810131073}, "speaker_2": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.9975526332855225}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.9973351359367371}, "speaker_2": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.9965507388114929}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 78 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 2.7 s) plus the other 19771 s at conversation 124's pace (617 s in 2.3 s, 263.0x real time). The two conversations alone: 258.2x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
- 3passed4h 58m into the runtook 1m 58s
startup 26.1 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 26.1 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.9978576898574829}, "speaker_2": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.9976327419281006}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.9975460171699524}, "speaker_2": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.9967638254165649}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 86 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 3.0 s) plus the other 19771 s at conversation 124's pace (617 s in 2.6 s, 238.1x real time). The two conversations alone: 234.7x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
Everything checks out. The submission is final, so handing it in: