Voice endpointing
- Score
- 0.0
- Wallet spend
- $1.03 of $50
- Model cost
- $12.22
- Agent time
- 7h 37m
- Steps
- 606
- Tokens in / out
- 59.5M / 344k
- Self-checks
- 1 passed of 1 used
- Rank at $50 wallet
- 8th of 14
The reviewer found a rule violation; see the Review tab.
Score
0.0
The raw metric (end-of-turn recall at 5% false positives) is rescaled so that the shipped silence heuristic (recall 0.655 at 5% false positives) scores 0 and perfect recall scores 100, clamped to that range.
After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.
✗causal frame probabilities
Serving processes each whole conversation offline, and the VAD post-processing looks into the future. In servecore.py, silero_vad_50hz calls get_speech_timestamps on the full waveform with min_speech_duration_ms=100 and min_silence_duration_ms=50, and leaves speech_pad_ms at its default of 30 ms (signature at trajectory#87). Segment starts are therefore moved 30 ms earlier, and whether a segment is kept depends on audio that comes after it. clean_vad(min_speech_ms=150, min_silence_ms=150) then deletes any speech run shorter than 150 ms and fills any silence run shorter than 150 ms, including trailing runs. Each frame's VAD value therefore depends on how long its run lasts after that frame. These two-sided VAD streams feed core_feats.timing_features and then EOTNet. As a result, the probability for frame k can depend on audio after (k+1)*0.08 s, by up to about 150 ms. That is backdating, which the contract forbids (trajectory#1). The participant's look-ahead test (trajectory#145-146) does not refute this. It used a different dev clean_vad (trajectory#88) that only fills gaps bounded by speech, and it only cut audio at segment offsets, never inside short gaps or speech blips.
✓learned causal modelThe served predictor is EOTNet, a learned dilated CNN with about 2.68M parameters. It was trained from scratch in train/
The served predictor is EOTNet, a learned dilated CNN with about 2.68M parameters. It was trained from scratch in train/train.py. Each CBlock pads only on the left (2*dil, 0), so the network itself is causal. The ckpt/eotnet.pt tensor inventory matches the EOTNet modules (proj_s/proj_o [192,512], fp_s/fp_o [32,19], blocks.*, head). server.py loads this checkpoint with weights_only and runs it through servecore.predict_pair. The frozen Mimi encoder is configured causal: use_causal_conv is True, and a truncation test differed from the full run by about 1.6e-7 (trajectory#86-87). The learned model is causal. The look-ahead in the VAD feature pipeline is judged under causal_frame_probabilities.
✓no lookup of evaluation audioserver.py and servecore.predict_pair compute Mimi and VAD features from the incoming audio and return sigmoid outputs of
server.py and servecore.predict_pair compute Mimi and VAD features from the incoming audio and return sigmoid outputs of EOTNet. There is no fingerprint matching, hash table or stored per-conversation output anywhere in the serving path. train/prep_gold.py builds targets only for the 8 shipped validation ids, which the policy allows. logs/fingerprint.txt and trajectory#368 show the shipped conversations were checked against the training split, not looked up at serving time.
✓training provenancePROVENANCE.md, architecture.json and train/download_data.sh declare the public otoSpeech train split as the pretraining
PROVENANCE.md, architecture.json and train/download_data.sh declare the public otoSpeech train split as the pretraining corpus, labelled automatically from VAD. logs/download_train.log and logs/prep_sessions.log show 896 sessions from train shards only. Fine-tuning and hyperparameter selection used only the 8 shipped conversations, with leave-one-actor-pair-out validation (logs/lopo_summary.txt). One public val shard was downloaded and inspected during exploration (trajectory#70, trajectory#80). There is no evidence it was used for training, calibration or selection: no script references it (trajectory#363), and it was deleted at trajectory#362.
✓harness credentials used only by the harnessNo participant code reads, copies or uses the harness API keys or base URL, and none launches harness CLIs. ANTHROPIC/OP
No participant code reads, copies or uses the harness API keys or base URL, and none launches harness CLIs. ANTHROPIC/OPENROUTER strings appear only in a ps aux process listing (trajectory#560), which the policy allows. Model downloads used HF_TOKEN, which is permitted. serve.sh sets HF offline mode and contains no harness credentials.
LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.
frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 1 of 4.
- 1passed7h 31m into the runtook 5m 47s
startup 89.2 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the
6 gates
- ✓review prerequisites
- ✓startup · GET /health answered 200 after 89.2 s
- ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.9976769089698792}, "speaker_2": {"hop_seconds": 0.08, "frames": 8551, "peak": 0.9974491000175476}}
- ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.9976769089698792}, "speaker_2": {"hop_seconds": 0.08, "frames": 7710, "peak": 0.9982993006706238}}
- ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 280 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 9.0 s) plus the other 19771 s at conversation 124's pace (617 s in 8.4 s, 73.0x real time). The two conversations alone: 74.4x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
- ✓healthy after requests
The agent's last message before its session ended.
73x real time on the H100 (required 3x), GPU path confirmed working, all gates passed. Submitting: