Voice endpointing
- Score
- 33.2
- Wallet spend
- $15.60 of $50
- Model cost
- $13.77
- Agent time
- 6h 26m
- Steps
- 263
- Tokens in / out
- 25.3M / 266k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 4th of 14
Score
33.2raw metric 0.7695
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
Verifier details
- metric
- eot_recall_at_5pct_fpr
- fpr_target
- 0.0500
- threshold
- 0.9033
- recall
- 0.7695
- fpr
- 0.0476
- positives
- 1523
- pauses
- 819
- tp
- 1172
- fn
- 351
- fp
- 39
- tn
- 780
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✓learned causal modelserve.sh runs server/server.py. That server loads frozen WavLM-large from weights/wavlm-large and the trained 6-member E
serve.sh runs server/server.py. That server loads frozen WavLM-large from weights/wavlm-large and the trained 6-member EOTEnsemble from weights/eot_model.pt (key 'models'), then runs both on the audio it receives. The model is defined in code/model.py. It was trained by code/train.py: pretraining on otoSpeech 104h, then fine-tuning of each member from base_best.pt per code/ft_ens.sh. code/combine_ckpts.py merged the members. logs/ft_final.log shows the six fine-tuning runs. The checkpoint inventory contains only EOTModel tensors.
✓causal frame probabilitiesIn server.py, _feats left-pads by the window length and takes unfold(0, WIN, HOP)[:n_frames], so each 6s WavLM window en
In server.py, _feats left-pads by the window length and takes unfold(0, WIN, HOP)[:n_frames], so each 6s WavLM window ends at or before its report time. Engineered RMS features use only audio up to r=(i+1)*0.2. code/model.py's RelPosBias masks k>q, giving causal self-attention. Chunked inference adds left context only. The sigmoid output is not smoothed, lookahead-padded or backdated. code/extract_features.py and code/infer.py use the same causal windowing.
✓no lookup of evaluation audioserver.py has no fingerprinting, hashing, transcript, timeline or per-conversation lookup. Its error path returns zeros.
server.py has no fingerprinting, hashing, transcript, timeline or per-conversation lookup. Its error path returns zeros. weights/eot_model.pt contains only model state dicts and args, with no embedded labels or audio-keyed tables.
✓training provenanceThe training data are declared in architecture.json and DATA_MANIFEST.md. Pretraining used otoearth/otoSpeech-full-duple
The training data are declared in architecture.json and DATA_MANIFEST.md. Pretraining used otoearth/otoSpeech-full-duplex-turn-104h, fetched by code/download_data.py and code/fetch_rest.py, with logs/holdout_ids.txt listing otoSpeech ids. Fine-tuning and 4-fold selection used only the 8 shipped conversations: trajectory#49 tarred /task/public, and trajectory#52 shows ids 20, 21, 22, 94, 95, 122, 124, 125. The session cloned the public turnbench code repo for the MIT gold.py label logic (trajectory#21) and read splits/dev.txt (trajectory#23). It never downloaded mundo-ai/turn-benchmark-dev or any mirror or derivative, and no held-out ids are used as TurnBench identifiers. mundo-ai appears only in search results and quoted text (trajectory#18).
✓harness credentials used only by the harnessParticipant code reads HF_TOKEN only for Hugging Face downloads (code/download_data.py, trajectory#19, trajectory#20). S
Participant code reads HF_TOKEN only for Hugging Face downloads (code/download_data.py, trajectory#19, trajectory#20). Spin transfer-session credentials were used in the participant's own session to move the shipped 8 to the GPU box (trajectory#48, trajectory#52), not by shipped code. No harness API key or base URL is read by server.py or any training code. The only MLE_JUDGE_API_KEY hit is help text.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed6h 14m into the runtook 3m 48s
startup 89.9 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 89.9 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.2, "frames": 3421, "peak": 0.99427}, "speaker_2": {"hop_seconds": 0.2, "frames": 3421, "peak": 0.995095}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.2, "frames": 3084, "peak": 0.987179}, "speaker_2": {"hop_seconds": 0.2, "frames": 3084, "peak": 0.996925}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 389 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 12.9 s) plus the other 19771 s at conversation 124's pace (617 s in 11.7 s, 52.6x real time). The two conversations alone: 52.8x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
All final checks green — 128 files, evidence complete, checksums verified, selfcheck passed with huge margins. Submitting: