Voice endpointing
- Score
- 43.7
- Wallet spend
- $0.00 of $200
- Model cost
- $0.47
- Agent time
- 1h
- Steps
- 251
- Tokens in / out
- 38.2M / 115k
- Self-checks
- 1 passed of 1 used
- Rank at $200 wallet
- 5th of 14
Score
43.7raw metric 0.8056
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.8170
- recall
- 0.8056
- fpr
- 0.0476
- positives
- 1523
- pauses
- 819
- tp
- 1227
- fn
- 296
- fp
- 39
- tn
- 780
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelServing is serve.sh -> server.py -> eot_model.py. load_vap loads the VapGPT weights from model/oto.ckpt strictly: it str
Serving is serve.sh -> server.py -> eot_model.py. load_vap loads the VapGPT weights from model/oto.ckpt strictly: it strips the net. prefix, drops VAP.codebook, and raises on any remaining missing or unexpected keys. The model is a CPC encoder plus LSTM gAR, a causal CConv1d downsample, and GPT/GPTStereo transformers. Its p_now projection output feeds a 15-feature logistic head (model/postnet.json), fitted by IRLS in train_postnet.py on the 8 shipped labeled conversations. Both parts are learned and are actually used in the serving path.
✓causal frame probabilitiesThe CPC conv stack uses symmetric padding. After the left-padded causal CConv1d downsample (k5, s2), however, 50 Hz fram
The CPC conv stack uses symmetric padding. After the left-padded causal CConv1d downsample (k5, s2), however, 50 Hz frame k depends only on samples up to 320k+311, which is before the end of the frame at 320(k+1). Attention in vendor/vap/modules.py applies a tril causal mask for both self- and cross-attention. ChannelNorm is per timestep. eot_model.py uses 25 s windows and keeps only each window's trailing frames. Its 48 kHz decimation averages past samples, and make_features uses only trailing rolling means, lagged differences and silence age. server.py returns ceil(duration/0.02) frames per speaker with no future context.
✓no lookup of evaluation audioserver.py decodes the two speakers' FLAC audio and passes it through infer_vap_arrays, make_features and score_features.
server.py decodes the two speakers' FLAC audio and passes it through infer_vap_arrays, make_features and score_features. The code has no hashing, fingerprinting, stored labels, or ID/duration-keyed tables. postnet.json holds only logistic coefficients and normalization statistics.
✓training provenanceThe backbone is viks66/VAP_checkpoints:oto.ckpt, pinned at revision b9aa0ba1718221153e04ef343c7f3b8cf84bfbc3. trajectory
The backbone is viks66/VAP_checkpoints:oto.ckpt, pinned at revision b9aa0ba1718221153e04ef343c7f3b8cf84bfbc3. trajectory#38 resolved that revision and trajectory#39 downloaded the checkpoint from that pinned URL; download_model.sh and logs/download.log record the same. data_manifest.json declares that the backbone was trained on the public otoSpeech-full-duplex-turn-104h training set, with CPC pretrained on Libri-Light 60k. I recorded this as a declared public source; the backbone's original training run itself is unverified. The postnet was fitted only on the 8 shipped ids (20, 21, 22, 94, 95, 122, 124, 125), per train_postnet.py, logs/training.json and logs/train-run.log. The trajectory shows no download or use of mundo-ai/turn-benchmark-dev or its no-backchannel derivative; the dev set appears only as text in a web search result. No held-out ids appear in training.
✓harness credentials used only by the harnesstrajectory#6 ran env | sort, which displayed credential values. That is environment listing only, and those values were
trajectory#6 ran env | sort, which displayed credential values. That is environment listing only, and those values were never reused. Searches found no API calls with the harness key or base URL and no spin model API usage. The participant did not launch any claude or codex CLI: the codex exec matches in trajectory#72 and trajectory#204 are the harness's own parent process (PID 8092) appearing in ps listings. The only network use was public Hugging Face, GitHub and PyPI downloads.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed36m 39s into the runtook 8m 9s
startup 73.5 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 73.5 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.9737299084663391}, "speaker_2": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.9640629887580872}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.969270646572113}, "speaker_2": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.965410053730011}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 336 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 11.8 s) plus the other 19771 s at conversation 124's pace (617 s in 10.1 s, 60.9x real time). The two conversations alone: 59.3x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
Submitted successfully. The serving script is serve.sh.
The validator reported 72.4% end-of-turn recall at 4.1% false positives, with no failed requests and 3.90× real time on CPU. These are results on the supplied conversations, which were also used to fit the scoring head, so they are not a held-out estimate. The H100 selfcheck passed: 73.5-second startup and 60.8× real time. No public benchmark validation or test split was used.