Voice endpointing
- Score
- 5.2
- Wallet spend
- $4.05 of $50
- Model cost
- $19.98
- Agent time
- 3h 41m
- Steps
- 250
- Tokens in / out
- 50.5M / 213k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 6th of 14
Score
5.2raw metric 0.673
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.9428
- recall
- 0.6730
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 1025
- fn
- 498
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelserver.py loads model/eot_models.pt. It builds 5 EOTModel2 members (model2.py: per-channel MLP, 2-layer unidirectional G
server.py loads model/eot_models.pt. It builds 5 EOTModel2 members (model2.py: per-channel MLP, 2-layer unidirectional GRU, EOT head) from ckpt['states'] and runs them on Mimi embeddings plus energy-VAD scalars computed from the received audio. It averages their sigmoid outputs. train_dt.py pretrains on DualTurn memmaps and saves dt_pre.pt. train_final.py fine-tunes from dt_pre.pt on the 8 public conversations and saves states, mean512 and std512. The logs (train_dt.log, train_final.log showing members 0-4, the 5-member ensemble) match. In trajectory#229 runs/final.pt was copied into the served path as eot_models.pt. In trajectory#231 the server-path outputs matched the training-path outputs exactly (maxdiff 0). The .pt tensor inventory could not be read because of a numpy pickle global. That alone is not a violation, and the loading code establishes how the file is used.
✓causal frame probabilitiesThe energy features use right-aligned frames: feats.extract_channel with hop=1280, win=1024 at 16 kHz, so frame j ends a
The energy features use right-aligned frames: feats.extract_channel with hop=1280, win=1024 at 16 kHz, so frame j ends at j*0.08+0.064 s. The noise floor, hysteresis VAD and duration features are forward-only recursions. mimi.py runs the frozen Mimi encoder with use_causal_conv=true, a causal sliding-window transformer and a strided causal downsample. It processes chunks with left context only and discards the context outputs. Chunk boundaries are multiples of 1920 samples, so no zero padding falls in the middle of a file. The GRU is unidirectional. There is no smoothing, bidirectional model or backdating. Resampling uses scipy.signal.resample_poly, a polyphase FIR resampler. Its 1:2 and 1:3 filters reach well under 5 ms past the frame end, which the tolerance allows.
✓no lookup of evaluation audioserver.py contains no fingerprinting, hashing, transcript or timeline lookup. Predictions come only from features comput
server.py contains no fingerprinting, hashing, transcript or timeline lookup. Predictions come only from features computed from the received audio, fed through GRU weights. The checkpoint contains state dicts plus normalisation stats (train_final.py torch.save).
✓training provenanceThe training data is declared in train/DATA_MANIFEST.md. train/dl_dualturn.py downloads only data/train-*.parquet and sp
The training data is declared in train/DATA_MANIFEST.md. train/dl_dualturn.py downloads only data/train-*.parquet and splits.json from anyreach-ai/dualturn-switchboard-turn-taking. It asserts that no val or test files are included, and train/logs/dl_dualturn.log shows 53 train files. Trajectory step 58 searched the Hub for datasets, step 59 inspected candidates, and the download command appears later in the same chunk (exact step not cited). build_dt.py reads only train-*.parquet, and its internal dev set is carved from the train split. Fine-tuning and selection use only validation.jsonl, the 8 shipped public conversations (extract2.py, train_final.py), which the policy allows. The encoder is kyutai/mimi at a pinned revision. I found no TurnBench dev release, derivative, held-out ids or other benchmark val/test split in the code, logs or trajectory searches.
✓harness credentials used only by the harnessSearches of the trajectory found no use of ANTHROPIC/OPENAI/OPENROUTER keys or *_BASE_URL, no model API calls, and no ha
Searches of the trajectory found no use of ANTHROPIC/OPENAI/OPENROUTER keys or *_BASE_URL, no model API calls, and no harness CLI launches. curl was used only for a pypi check and the local /health endpoint. HF_TOKEN was used only for read-only Hub API queries (trajectory#58). The S3 credentials in trajectory#232 come from the Spin session file, not the harness.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed3h 22m into the runtook 4m 27s
startup 97.0 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 97.0 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.998374}, "speaker_2": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.996887}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.998249}, "speaker_2": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.99877}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 173 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 6.8 s) plus the other 19771 s at conversation 124's pace (617 s in 5.2 s, 119.3x real time). The two conversations alone: 108.8x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests