Voice endpointing
- Score
- 0.0
- Wallet spend
- $3.60 of $50
- Model cost
- $10.19
- Agent time
- 2h 59m
- Steps
- 224
- Tokens in / out
- 51.6M / 226k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 8th of 14
Score
0.0raw metric 0.4025
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.9111
- recall
- 0.4025
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 613
- fn
- 910
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelserver.py defines its own CausalEOT model: a 6-layer, 384-d, 6-head Transformer with a triu causal mask and about 10.7M
server.py defines its own CausalEOT model: a 6-layer, 384-d, 6-head Transformer with a triu causal mask and about 10.7M parameters. It loads model.pt. The pytorch inventory of model.pt (model.in_proj.weight [384,160], encoder.layers.0-5) matches that config. train/train.py starts from random init and trains with AdamW and frame BCE. train/train.log reports 420 conversations, 10.7M parameters, 35 epochs and best=0.4338. The trajectory shows the shipped model.pt was copied from best_full420.pt, the output of that run (trajectory#206, trajectory#207). No pretrained weights are used.
✓causal frame probabilitiesserver.py builds a causal log-mel (n_fft=win=hop=1920 at 48kHz, center=False, 0.04 s hop). It runs the Transformer with
server.py builds a causal log-mel (n_fft=win=hop=1920 at 48kHz, center=False, 0.04 s hop). It runs the Transformer with a causal mask in chunks of 750 frames with a 625-frame step, keeping only newly produced frames. Each frame's output is a sigmoid probability at a 0.04 s hop. There is no right-context lookahead and no backdating of decisions.
✓no lookup of evaluation audioThe serving path (serve.sh, which installs offline wheels and then runs exec python server.py) only computes features an
The serving path (serve.sh, which installs offline wheels and then runs exec python server.py) only computes features and runs the network. It contains no audio fingerprinting, no hash tables, no stored per-conversation outputs and no label lookup.
✓training provenanceTraining data is otoSpeech (otoearth/otoSpeech-full-duplex-turn-104h, pinned revision 46f52029..., declared in data_mani
Training data is otoSpeech (otoearth/otoSpeech-full-duplex-turn-104h, pinned revision 46f52029..., declared in data_manifest.json and architecture.json and fetched with hf_hub_download in train/precompute_oto.py and train/precompute.py) plus the 8 permitted shipped dev ids (20, 21, 22, 94, 95, 122, 124, 125). No download command or log for the corpus itself was retained beyond these scripts and train/precompute.log, so the otoSpeech source is unverified. The otoSpeech labels use a similar taxonomy to TurnBench, but nothing establishes that its conversations are TurnBench conversations. The numeric ids that coincide with held-out ids are otoSpeech ids carrying an oto_ prefix, not TurnBench identifiers. The single probe of mundo-ai/turn-benchmark-dev failed because huggingface_hub was not installed and was never retried (trajectory#20, trajectory#21). Split lists and leaderboard text were only viewed in session output and never used (trajectory#14, trajectory#18). Selection used a 40-conversation random split of otoSpeech, not the val or test split of a public benchmark. Only the 8 /task/public flacs were uploaded (trajectory#77).
✓harness credentials used only by the harnessHF_TOKEN was used only to download otoSpeech (train/precompute_oto.py, trajectory#55), and nothing was uploaded to HF wi
HF_TOKEN was used only to download otoSpeech (train/precompute_oto.py, trajectory#55), and nothing was uploaded to HF with it. Spin transfer session credentials were used only for spin storage transfers of the dev flacs, models and logs (trajectory#76, trajectory#77, trajectory#206). Searches found no model-API calls with harness keys or base URL, no echoing of harness credentials and no launch of other harness CLIs.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 47m into the runtook 3m 26s
startup 57.3 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 57.3 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.95947265625}, "speaker_2": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.95068359375}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.9609375}, "speaker_2": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.94482421875}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 54 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 2.2 s) plus the other 19771 s at conversation 124's pace (617 s in 1.6 s, 378.1x real time). The two conversations alone: 338.5x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
- 2passed2h 46m into the runtook 3m 13s
startup 71.5 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 71.5 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.9755859375}, "speaker_2": {"hop_seconds": 0.04, "frames": 17102, "peak": 0.9306640625}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.97509765625}, "speaker_2": {"hop_seconds": 0.04, "frames": 15420, "peak": 0.919921875}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 68 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 2.5 s) plus the other 19771 s at conversation 124's pace (617 s in 2.0 s, 302.0x real time). The two conversations alone: 287.1x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
Submitted.
Dual-channel causal Transformer for end-of-turn detection, trained from scratch on otoSpeech 104h (380 train / 40 val) plus the 8 provided examples (5x). Input 80-dim causal log-mel at 25 Hz per speaker, 6-layer 384-dim Transformer with causal masking, per-speaker sigmoid heads. Hop 0.04 s.
Local validation on 8 public conversations: recall 0.69 at 5% FPR, 82x real time on CPU. Selfcheck on H100 passed twice: startup 57-71 s, 300-375x real time, healthy.
Files in /workspace/output/: serve.sh, server.py, model.pt (41 MB), wheels/ (3.7 GB CUDA), architecture.json, data_manifest.json, train/ source and logs.