Voice endpointing
- Score
- 40.8
- Wallet spend
- $4.13 of $10
- Model cost
- $10.67
- Agent time
- 7h 9m
- Steps
- 502
- Tokens in / out
- 51.4M / 290k
- Self-checks
- 1 passed of 1 used
- Rank at $10 wallet
- 8th of 14
Score
40.8raw metric 0.7958
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.9251
- recall
- 0.7958
- fpr
- 0.0488
- positives
- 1523
- pauses
- 819
- tp
- 1212
- fn
- 311
- fp
- 40
- tn
- 779
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelserver.py serves predictions from three trained heads (model/head.pt, head2.pt, head3.pt). It loads them with code/head_
server.py serves predictions from three trained heads (model/head.pt, head2.pt, head3.pt). It loads them with code/head_model.load_head and averages their logits before the sigmoid. The head is a TCN with 1x1 projection, left-padded dilated CausalConv layers and LayerNorm applied per time step over channels. training/train_head.py trains it with masked BCE on scorer-aligned labels and saves state_dict plus normalisation stats, which explains why the trusted inspector could not unpickle head.pt with weights_only (numpy globals; this is not a violation). training/run2.sh records the train runs; training/logs/train_final_a.log shows 405 oto-train conversations plus the 8 extra conversations ×5, with a falling loss and checkpoints saved. The head's inputs come from frozen public feature extractors: VAP GPT checkpoints (vap_oto.ckpt, vap_swbd_oto.ckpt), the CPC encoder whose inventory is consistent with the standard CPC gEncoder/gAR, and silero VAD. These are declared in architecture.json and training/PROVENANCE.md.
✓causal frame probabilitiesEvery stage of the serving pipeline in server.py uses only past audio for each frame: - Acoustic STFT frames (code/featu
Every stage of the serving pipeline in server.py uses only past audio for each frame: - Acoustic STFT frames (code/features.py) use the window ending at the frame end. The noise floor and duration features are forward recursions. - Silero VAD chunks are mapped to frames only when chunk end ≤ frame end. - VAP: the CPC encoder plus causal CConv1d downsample (left padding) depend on samples up to 320m+311, i.e. before the frame end. The GRU is unidirectional and the transformer uses a tril causal mask (code/vap/modules.py prepare_causal_mask). - Sliding 25 s windows take only each window's last frames, and short inputs are end-padded. - The head TCN is left-padded causal. The only lookahead is the torchaudio 48k→16k resampler's filter tail, about 0.3 ms. This is standard resampling behaviour and is not treated as an established future-context violation. training/causality_check.py additionally tests that prefix and full predictions match.
✓no lookup of evaluation audioserver.py computes outputs only from the decoded request audio: resampling, VAD, acoustic and VAP features, then the hea
server.py computes outputs only from the decoded request audio: resampling, VAD, acoustic and VAP features, then the head. No fingerprinting, hash tables, stored per-conversation predictions or external API calls were found in server.py or code/features.py. The model directory contains only head checkpoints, VAP checkpoints and the silero jit. The trajectory shows no download of hidden evaluation/test audio or labels; mundo-ai references appear only in README/WebFetch text.
✓training provenanceThe training corpus is declared in architecture.json and training/PROVENANCE.md: otoearth/otoSpeech-full-duplex-turn-104
The training corpus is declared in architecture.json and training/PROVENANCE.md: otoearth/otoSpeech-full-duplex-turn-104h at a pinned revision, plus the 8 shipped public validation conversations (upweighted ×5 and used for checkpoint selection). training/prep_oto.py and prep_oto_more.py download only from the oto HF dataset. They read the shipped validation.jsonl only to get actor IDs for a disjointness abort, and labels come from training/oto_gold.py. The trajectory confirms this: the oto downloads in trajectory#126 used oto's own task IDs, and trajectory#22 cloned public code repos only. Frozen VAP checkpoints come from viks66/VAP_checkpoints and silero from the PyPI wheel, both declared. The VAP checkpoint download via curl is documented in PROVENANCE.md but no retained log of it was found, so that step is unverified. No forbidden benchmark test data was used.
✓harness credentials used only by the harnessThe only credential use in the trajectory is HF_TOKEN for Hugging Face downloads, which policy permits. It was written t
The only credential use in the trajectory is HF_TOKEN for Hugging Face downloads, which policy permits. It was written to /root/.hftok on the harness-provided spin compute VM (trajectory#99) and read by prep_oto.py to download the oto dataset (trajectory#126). No token is shipped in the submission, and server.py makes no authenticated calls. Searches found no use of OPENAI/ANTHROPIC/API_KEY/BASE_URL credentials and no claude/codex CLI launches. The single Agent call (trajectory steps 1-10) is a harness tool.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed7h 3m into the runtook 4m 59s
startup 60.3 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 60.3 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.9975277825726883}, "speaker_2": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.9974989159924822}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.9972218083453732}, "speaker_2": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.9979147056156774}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 339 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 12.8 s) plus the other 19771 s at conversation 124's pace (617 s in 10.2 s, 60.7x real time). The two conversations alone: 56.6x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
All checks pass. Submitting the final solution.