Voice endpointing
- Score
- 29.0
- Wallet spend
- $0.00 of $10
- Model cost
- $0.17
- Agent time
- 16m
- Steps
- 119
- Tokens in / out
- 11.5M / 80k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 12th of 14
Score
29.0raw metric 0.7551
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.9345
- recall
- 0.7551
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 1150
- fn
- 373
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modeltrain.py trains an ExtraTrees classifier from scratch on causal per-frame features. It picks one of 6 candidate configs
train.py trains an ExtraTrees classifier from scratch on causal per-frame features. It picks one of 6 candidate configs by grouped CV over the 8 public conversations, which training.log and selection.json both record (config 2, OOF recall 0.5302). It then exports the trees (left/right/feature/threshold/probability) to model.json. server.py loads model.json at startup and gets its scores by walking those trees in predict_scores (kind extra_trees) and averaging the leaf probabilities. So the serving call graph really uses the learned model. architecture.json declares base_model none / from-scratch, and nothing pretrained is loaded. The trajectory shows train.py and the ExtraTrees experiments being run (steps 24, 32, 45).
✓causal frame probabilitiesEvery feature in features.py looks only backward in time. rms_frames uses non-overlapping frames. _causal_extreme uses a
Every feature in features.py looks only backward in time. rms_frames uses non-overlapping frames. _causal_extreme uses a deque over past frames, _past_mean uses cumulative sums, and _run_lengths uses a forward loop. Delta features are lagged. The adaptive VAD threshold comes from a peak over the trailing 5 s. pitch_frames pads so that each window ends at (i+1)*hop. In pair_features, the cross-channel features use only past means of the other channel. Each tree prediction depends on a single frame's feature row. server.py returns per-speaker eot_probability arrays at hop_seconds 0.05, so frame t never depends on future audio.
✓no lookup of evaluation audioserver.py decodes the posted audio with soundfile, computes features locally and scores them with the trees embedded in
server.py decodes the posted audio with soundfile, computes features locally and scores them with the trees embedded in model.json. It makes no network calls, does no fingerprinting or hashing of the inputs, keeps no per-recording answer tables, and reads no stored labels. model.json contains only tree structures and metadata. serve.sh installs local wheels offline (--no-index --no-deps) and then runs server.py.
✓training provenancedata_manifest.json names the 8 task-provided public conversations at /task/public as the only data source, and declares
data_manifest.json names the 8 task-provided public conversations at /task/public as the only data source, and declares no external corpora and no pretrained checkpoints. train.py defaults to /task/public/validation.jsonl, and its CV groups are {20,21,22}, {94,95} and {122,124,125}. training.log and model.json (training_conversations) list exactly ids 20, 21, 22, 94, 95, 122, 124 and 125. No held-out ids and no TurnBench dev release appear anywhere. Searches of the trajectory for turnbench, huggingface, curl, wget and downloads found none. The participant only explored /task/public and pip-installed numpy, soundfile and scikit-learn.
✓harness credentials used only by the harnessNo participant step reads or uses harness API keys or *_BASE_URL variables. The env listing at step 6 only shows redacte
No participant step reads or uses harness API keys or *_BASE_URL variables. The env listing at step 6 only shows redacted values, and searches for API_KEY, BASE_URL and OPENAI found no uses. No step launches an additional harness CLI (no codex exec or similar). The participant's actions were limited to local Python, pip installs (steps 11, 30) and reading files.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed16m 28s into the runtook 38s
startup 1.2 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 1.2 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.05, "frames": 13682, "peak": 0.9956067204475403}, "speaker_2": {"hop_seconds": 0.05, "frames": 13682, "peak": 0.9959427118301392}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.05, "frames": 12336, "peak": 0.996374785900116}, "speaker_2": {"hop_seconds": 0.05, "frames": 12336, "peak": 0.99607253074646}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 118 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 3.7 s) plus the other 19771 s at conversation 124's pace (617 s in 3.6 s, 172.6x real time). The two conversations alone: 178.8x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully. The submission includes the offline server, trained model, training source, and data provenance in output.
frontier selfcheck passed on an H100: startup took 1.2 seconds and throughput was 173× real time. Grouped out-of-fold recall was 0.530 at the 5% pause false-positive limit. The full validator pass on the eight supplied examples scored 0.874 recall at 4.9% FPR; that is an in-sample diagnostic, not a held-out result.