Voice endpointing

After grading, a reviewer model reads the submission's files and the agent's trajectory and checks each of the task's rules. A failed check makes the score 0. Step numbers link into the trajectory.

LLM adversarial review is not proof of arbitrary program behavior. Host storage and task-specific deterministic checks remain required.

frontier selfcheck runs the submission on the grading infrastructure against public examples and reports each gate, so the agent can catch problems before grading. Each run gets four. This agent used 3 of 4.

1. 1
   failed4h 58m into the runtook 6m 22s

   failed startup: Server did not become healthy in 300s (last /health probe: HTTP 503 b'{"status": "loading", "error": "ModuleNotFoundError(\\"No module named \'torchaudio\'\\")"}'); end of its log: installing vendored wheels (log: /tmp/eot_install.log) [09:56:19] starting; HOST=0.0.0.0 PORT=8

   2 gates

   • ✓review prerequisites
   • ✗startup · Server did not become healthy in 300s (last /health probe: HTTP 503 b'{"status": "loading", "error": "ModuleNotFoundError(\\"No module named \'torchaudio\'\\")"}'); end of its log: installing vendored wheels (log: /tmp/eot_install.log) [09:56:19] starting; HOST=0.0.0.0 PORT=8000 MODEL_NAME=submission Traceback (most recent call last): File "/submission/server.py", line 155, in main load_model() File "/submission/server.py", line 36, in load_model from eot_model import EotNet File "/submission/src/eot_model.py", line 13, in <module> from vap.model import VapGPT, VapConfig # noqa: E402 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/submission/vap/model.py", line 9, in <module> from vap.encoder import EncoderCPC File "/submission/vap/encoder.py", line 4, in <module> from vap.encoder_components import load_CPC, get_cnn_layer File "/submission/vap/encoder_components.py", line 10, in <module> from vap.utils import repo_root File "/submission/vap/utils.py", line 7, in <module> from vap.audio import time_to_frames, load_waveform File "/submission/vap/audio.py", line 2, in <module> import torchaudio ModuleNotFoundError: No module named 'torchaudio' [09:56:20] model load FAILED; serving 503 on /health [09:56:20] serving on 0.0.0.0:8000
2. 2
   failed5h 16m into the runtook 6m 20s

   failed startup: Server did not become healthy in 300s (last /health probe: HTTP 503 b'{"status": "loading", "error": "AssertionError(\'audio VAP ENCODER: torch.Size([1, 8, 400000]) != (B, 2, n_samples)\')"}'); end of its log: installing vendored wheels (log: /tmp/eot_install.log) [10:14:08]

   2 gates

   • ✓review prerequisites
   • ✗startup · Server did not become healthy in 300s (last /health probe: HTTP 503 b'{"status": "loading", "error": "AssertionError(\'audio VAP ENCODER: torch.Size([1, 8, 400000]) != (B, 2, n_samples)\')"}'); end of its log: installing vendored wheels (log: /tmp/eot_install.log) [10:14:08] starting; HOST=0.0.0.0 PORT=8000 MODEL_NAME=submission Froze EncoderCPC! [10:14:11] model loaded on cuda in 0.7s; serve combo: {'mode': 'ari', 'alpha': 0.15} Traceback (most recent call last): File "/submission/server.py", line 155, in main load_model() File "/submission/server.py", line 51, in load_model eot_probs(model, w8, device, batch=8, combo=combo) File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/submission/src/infer.py", line 68, in eot_probs p = fwd(w)[0] ^^^^^^ File "/submission/src/infer.py", line 61, in fwd ph, pn = model.forward_both(w) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/submission/src/eot_model.py", line 44, in forward_both x1, x2 = v.encode_audio(wav) ^^^^^^^^^^^^^^^^^^^ File "/submission/vap/model.py", line 171, in encode_audio audio.shape[1] == 2 AssertionError: audio VAP ENCODER: torch.Size([1, 8, 400000]) != (B, 2, n_samples) [10:14:11] model load FAILED; serving 503 on /health [10:14:11] serving on 0.0.0.0:8000
3. 3
   passed5h 33m into the runtook 2m 27s

   startup 63.4 s of 300 s; 1/1 gates passed; The 3x real-time gate is projected from 2 of 30 requests at concurrency 1: public conversations 122 (the cold first request) and 124 (its pace). Each gets half its audio's duration to answer, the first 300 s more for warm-up; grading bounds only the

   6 gates

   • ✓review prerequisites
   • ✓startup · GET /health answered 200 after 63.4 s
   • ✓POST /predict (conversation 122) · {"conversation": "122", "speaker_1": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.988755}, "speaker_2": {"hop_seconds": 0.02, "frames": 34203, "peak": 0.980753}}
   • ✓POST /predict (conversation 124) · {"conversation": "124", "speaker_1": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.980372}, "speaker_2": {"hop_seconds": 0.02, "frames": 30840, "peak": 0.985327}}
   • ✓real-time factor · Held-out pass projected from 2 of 30 requests at concurrency 1 at 100 s of server time for 20455 s of audio, 6818 s allowed: the cold first request (conversation 122, 684 s of audio in 4.2 s) plus the other 19771 s at conversation 124's pace (617 s in 3.0 s, 207.0x real time). The two conversations alone: 182.2x real time. Clock: trusted HTTP client inside the serving sandbox; provider transfer/exec latency excluded.
   • ✓healthy after requests

The agent's last message before its session ended.