Voice endpointing
- Score
- 51.3
- Wallet spend
- $13.96 of $50
- Model cost
- $7.61
- Agent time
- 2h 10m
- Steps
- 227
- Tokens in / out
- 22.8M / 147k
- Self-checks
- 2 passed of 2 used
- Rank at $50 wallet
- 4th of 19
Score
51.3raw metric 0.8319
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.7138
- recall
- 0.8319
- fpr
- 0.0476
- positives
- 1523
- pauses
- 819
- tp
- 1267
- fn
- 256
- fp
- 39
- tn
- 780
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelserver.py serves a learned model. Its Engine strictly loads two frozen pretrained VapGPT bases (weights/vap_oto.pt and w
server.py serves a learned model. Its Engine strictly loads two frozen pretrained VapGPT bases (weights/vap_oto.pt and weights/vap_swbd.pt, whose inventories show full CPC/GRU/transformer/VAP-head tensors) plus weights/eot_head.pt. The eot_head.pt inventory shows LayerNorm(1038), Linear(1038→256) and Linear(256→1) tensors. These match the ResidualHead in eot_model.py and training/train_eot_head.py. train_eot_head.py trains that head with AdamW, using window-max BCE plus a ranking loss on features from the frozen bases computed over otoSpeech conversations. evidence/training_full_v2.log records the run, and step 700 at residual scale 0.65 was selected. The output is a learned model, not a hand-coded rule.
✓causal frame probabilitiesThe model components are causal. The CPC encoder convs have a receptive field that ends before each 20 ms frame's end. T
The model components are causal. The CPC encoder convs have a receptive field that ends before each 20 ms frame's end. The downsampling CConv1d uses left-only padding. CPCAR is a forward-only GRU, and MultiHeadAttentionAlibi applies a lower-triangular causal mask. ChannelNorm and LayerNorm are per-frame. In extract_base (eot_model.py), the first window keeps frames 0–1250 of the first 25 s. Each later window starts at k·5 s and keeps only its final 5 s of frames, so each frame sees only past and current audio. The final window is right-padded, and padding never precedes the frames it affects. Resampling in server.py stays within tolerance. evidence/causal_prefix_invariance.log is consistent with this.
✓no lookup of evaluation audioserver.py decodes the submitted FLAC, resamples it, runs predict_features (eot_model.py) through the neural models and r
server.py decodes the submitted FLAC, resamples it, runs predict_features (eot_model.py) through the neural models and returns per-frame probabilities at a 0.02 s hop. There is no hashing, fingerprinting, stored timeline, label table or per-id lookup in the serving path. The only fallback is all zeros on exception.
✓training provenancedata_manifest.json declares the training corpus as otoearth/otoSpeech-full-duplex-turn-104h at a pinned revision. That c
data_manifest.json declares the training corpus as otoearth/otoSpeech-full-duplex-turn-104h at a pinned revision. That corpus is the permitted public speaker-disjoint training split. training/download_oto.py streams only that dataset. In training/train_eot_head.py, load_records reads only otoSpeech metadata and audio, with an actor-disjoint internal validation split recorded in evidence/training_split.json. Selection used only the 8 shipped supplied examples (/workspace/inputs/public/validation.jsonl), which is allowed. The pretrained VAP and CPC checkpoints are declared public sources. The trajectory shows only a metadata listing of mundo-ai/turn-benchmark-dev at trajectory#8 (filenames only). There was no download, and its IDs or labels were never used. evidence/no_conversation_overlap.log shows the otoSpeech audio does not duplicate the supplied examples.
✓harness credentials used only by the harnessTrajectory searches found no participant use of harness credentials: no ANTHROPIC/API_KEY/BASE_URL use and no `claude -p
Trajectory searches found no participant use of harness credentials: no ANTHROPIC/API_KEY/BASE_URL use and no `claude -p` invocations. The only credential use was HF_TOKEN for read-only downloads of public otoSpeech and the checkpoints (training/download_oto.py), which is allowed. No uploads were made with it.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 2 of 4.
- 1passed1h 14m into the runtook 2m 57s
startup 69.3 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 69.3 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.9991775155067444}, "speaker_2": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.9986282587051392}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.998921275138855}, "speaker_2": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.9981491565704346}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 137 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 4.5 s) plus the other 19771 s at conversation 124's pace (617 s in 4.1 s, 148.8x real time). The two conversations alone: 151.2x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
- 2error1h 51m into the runtook 1m 47sfree (stopped before any GPU time)
The self-check could not finish (ServiceError: jwks fetch failed: error sending request for url (https://oidc.modal.com/.well-known/jwks.json)).
1 gates
- ✓review prerequisites
- 3passed2h 6m into the runtook 3m 28s
startup 73.6 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 73.6 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.9951613545417786}, "speaker_2": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.9970707893371582}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.9952821135520935}, "speaker_2": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.9945172667503357}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 137 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 4.8 s) plus the other 19771 s at conversation 124's pace (617 s in 4.1 s, 150.1x real time). The two conversations alone: 145.7x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
The final selfcheck passed on the exact submission digest: 73.6-second startup and 149.8× projected real-time throughput on one offline H100. All response and health checks passed. I’m submitting that unchanged artifact now.