Voice endpointing

The model is a learned network. training/train.py starts from HF HubertModel.from_pretrained('facebook/hubert-large-ll60k'), maps those weights into model.CausalSSL, adds rank-16 LoRA plus a 4-layer causal transformer head, and trains with BCE plus an auxiliary activity loss. The session shows the run that produced the weights: the large_lora_v2 command at trajectory#187. Its step-3150 checkpoint had LoRA merged and was saved as large_e1_merged.ckpt (trajectory#331). That file was copied to /workspace/output/model.ckpt (trajectory#333) and its md5 was checked before submission (trajectory#526). At serving time, serve.sh sets MODEL_PATH=model.ckpt, and server.py calls predict.load_checkpoint, which rebuilds EOTModel from model_args and loads the state_dict. The forward pass in predict_streams then produces the probabilities. No weight inventory was supplied for the .ckpt pickle, but the code path and the session record are consistent.

Every operation in the model only looks at past audio: - The conv feature extractor uses valid convolutions. Their fixed 5 ms (80-sample) right edge is absorbed by a 1-frame reporting delay. - The positional conv is left-padded. Encoder and head attention use is_causal=True. LayerNorm is applied per frame. - causal_norm uses an expanding cumsum RMS. hand_features uses trailing windows and cummax. - Block inference in predict_streams uses only left context. - Post-processing is a trailing 5-frame mean plus a 21-frame delay filled with zeros. The only full-signal operation is the soxr HQ resample in server.py. That is a standard polyphase resampler, which the 5 ms tolerance allows.

server.py decodes the request audio, resamples it, and runs the network through predict_streams and streams_to_response. On an exception it falls back to zeros. There is no hashing, fingerprinting, label table or stored per-conversation output. The only thing loaded at runtime is model.ckpt, and it is loaded as model weights.

The training corpus is declared in data_manifest.md and architecture.json as otoearth/otoSpeech-full-duplex-turn-104h, and training/download.py contains the download command. The session confirms the download: snapshot_download of that repo (trajectory#37, trajectory#40, trajectory#41). The TurnBench paper describes this repo as its separate training release (trajectory#19). The dev30 selection split was drawn from the same corpus. The 8 conversations used for selection and LOO experiments are the shipped validation samples, which the policy allows. They are listed in training/make_folds.py as VAL8 = 20, 21, 22, 94, 95, 122, 124, 125. The shipped model is the train-only large_lora_v2 step-3150 checkpoint, not the all8 fine-tune. No session step downloads or uses mundo-ai/turn-benchmark-* or the JSALT no-backchannel set. The only mentions are a README fetch and the manifest text. The cloned turnbench GitHub repo is code. Numeric directory ids in otoSpeech are that corpus's own identifiers, not TurnBench conversation ids.

The trajectory has no use, read or copy of the agent harness's model API credentials: no OPENROUTER, ANTHROPIC or OPENAI key, and no BASE_URL apart from Spin setup text in the task prompt. The participant used only Spin compute and HF_TOKEN, which the policy allows. HF_TOKEN was passed to its own Spin VM for the dataset download (trajectory#39).