Blog.
Results, explainers and technical notes from p99lab.
Results
dddd
d8888888b
dd8888888888b
d8888888888888b
dd8888888888888888b
d88888888888888888888b
d88888888888888888888888b
dd88888888888888888888888888b ddd
d8888888888PPP d8888888888888b d8888
Y8888PP d8888888888888b d8888888
P d8888888888888b dd888888888
d8888888888888b dd888888888888
d8888888888888b dd8888888888bbb d888888888888888dddddddddddddb
d8888888888888b dd88888888888888888888b d8888888888888888888888888888888P
d8888888888888b d888888888PPPYYY88888888b 88888888888888888888888888888888P
d8888888888888b d8888888P Y888888 Y888888888888888888888888888888
d8888888888888b 88888888P Y88888 Y888888888888888YYYYYYYYYYYYP
d8888888888888b d88888888b 88888 Y8888888888888
d8888888888888b 8888888888 8888 d888888888888
d8888888888888b 88888888888bb 888 d888888888888
d8888888888888b 888888888888888bb YY d888888888888
d8888888888888b 8888888888888888888bb d888888888888
d8888888888888b Y8888888888888888888888bb d888888888888
d8888888888888b 8888888888888888888888888b d888888888888
d8888888888888b Y888888888888888888888888b d888888888888
d8888888888888b YY888888888888888888888888 d888888888888
d8888888888888b Y8888888888888888888888b d888888888888
d8888888888888b YYY888888888888888888 d888888888888
d8888888888888b YY88888888888888 d888888888888
d8888888888888b Y88b Y888888888888 d888888888888
d8888888888888b Y888b Y8888888888 Y888888888888b
d8888888888888b Y8888b 8888888888 Y8888888888888b d
d8888888888888b 88888 888888888P Y888888888888888bbbbdddd8888b
d8888888888888b 888888 d88888888P 888888888888888888888888888
ddd8888888888888888888bbb Y888888 d888888888 88888888888888888888888P
dd8888888888888888888888888888888888bb Y888888888888888888888888P Y888888888888888888PP
Y8888888888888888888888888888888888888 Y88888888888888888888P Y8888888888888PP
YYYYYYYYPPPPPP YYYYYPPPP
turn-1-mini on Sesame's TurnBench
Ranked first on end-of-turn and second on interruption on the held-out test set. What the benchmark measures, and the full numbers with latency spread.
Kollaikal RupeshRead →
Explainer
dd8888 dd8888888bbb ddd8888888888bb ddddddddddb ddddddddddb ddd8888888888bbb dd88888888888bb
ddd888888888 d88888888888888888b dd88888888888888888888b d888888888888888888P 888888888888888888P dd88888888888888888888b dd8888888888888888888b
dd88888888888888 d8888888888888888888888b d88888888888888888888888888 Y888888888888888888 888888888888888888 d888888888PPPYYY88888888b d888888888888888888888888b
88888888888888888888888888888888888888888888 d8888888PPPPYYYY88888888888888b Y888888888888888 YY88888888888888 d8888888P Y888888 888888888P Y8888888888b
Y88888888888888888888888888888888888888888888 d888888888 Y8888888888888 8888888888888 8888888888888 88888888P Y88888 d888888888P 88888888888
Y88888888888888PP Y8888888888888888 d888888888P Y888888888888b 8888888888888 8888888888888 d88888888b Y8888 d888888888P 88888888888
8888888888888b Y888888888888888 d888888888 Y888888888888b 8888888888888 8888888888888 8888888888 8888b d8888888888 88888888888b
d888888888888b Y88888888888888 Y88888888 d888888888888b 8888888888888 8888888888888 888888888888b 888b 88888888888 Y88888888888
d888888888888b 88888888888888b YPP d888888888888b 8888888888888 8888888888888 888888888888888bb YY d88888888888dddd88888888888888888888
d888888888888b Y88888888888888 d888888888888b 8888888888888 8888888888888 8888888888888888888bb 888888888888888888888888888888888888b
d888888888888b 88888888888888 dd888888888888888b 8888888888888 8888888888888 Y8888888888888888888888bb d888888888888888888888888888888888888P
d888888888888b 88888888888888 dd88888888888888888888b 8888888888888 8888888888888 Y888888888888888888888888b d88888888888b
d888888888888b 88888888888888 dd8888888888PY888888888888b 8888888888888 8888888888888 Y888888888888888888888888b d88888888888b
d888888888888b 88888888888888 d8888888888P d888888888888b 8888888888888 8888888888888 Y888888888888888888888888 Y888888888888
d888888888888b 8888888888888P d8888888888P d888888888888b 8888888888888 8888888888888 Y8888888888888888888888b Y888888888888b
d888888888888b 8888888888888 d88888888888 d888888888888b 8888888888888 8888888888888 YYY888888888888888888 Y8888888888888
d888888888888b 8888888888888 d88888888888P d888888888888b 8888888888888 8888888888888 YY88888888888888b Y8888888888888b
d888888888888b 888888888888P 888888888888 d888888888888b 8888888888888 8888888888888 Y88b Y88888888888P 88888888888888b
d888888888888b d88888888888P 888888888888b d888888888888b 8888888888888b 8888888888888 Y888b Y8888888888P Y888888888888888b d88b
d888888888888b 88888888888P 8888888888888 d8888888888888b Y88888888888888b dd8888888888888 Y8888b 8888888888 Y88888888888888888bb dd88888P
d8888888888888 d8888888888P 88888888888888b dd888888888888888 Y8888888888888888888888888888888888888 88888 888888888P Y88888888888888888888888888888888P
d88888888888888 d8888888888P Y8888888888888888888888888888888888bb dddb 888888888888888888888888888888888888888b 888888b d88888888P Y88888888888888888888888888888P
d888888888888888bb dd8888888888P 88888888888888888888 Y888888888888888888P Y8888888888888888888888PY88888888888888888b Y888888 d888888888 Y88888888888888888888888888P
d88888888888888888888888888888888P 88888888888888888P Y8888888888888888P 8888888888888888888P Y88888888888888888P 888888888888888888888888P Y8888888888888888888888P
d88888888888888888888888888888P Y8888888888888P Y888888888888P Y88888888888888P 8888888888PPP Y88888888888888888888P Y88888888888888888P
d888888888888PYYY888888PPPP YYYYPPPP YYYYPPPP YYYYYPPPPP YPPPP YYYYYYYYPPPPPP YYYYYYPPPPP
d888888888888b
d888888888888b
d888888888888b
d888888888888b
d888888888888b
d888888888888b
d888888888888b
d888888888888b
d888888888888b
dd888888888888888bb
d88888888888888888888888b
YYYYYYYYYYYYYYYYYYYYYYYYP
What a turn detector decides at every pause
Finished, or just pausing? The one decision our models make, the two ways to get it wrong, and where it sits in a voice agent.
Kollaikal RupeshRead →
Technical note
dddd
dd888888b
d8888888888b
d8888888888888b
dd8888888888888888b
d88888888888888888888b
d88888888888888888888888b
dd dd88888888888888888888888888b
d8888b d8888888888PPP d8888888888888b
dd888888b Y8888PPP d8888888888888b
d888888888b P d8888888888888b
d888888888888b d8888888888888b
d888888888888888bddddddddddddb ddddddddddb dddddddddb dd888b dd8888b dd888b dd88888888bb d8888888888888b
d8888888888888888888888888888888 8888888888888888888 d888888888888888888 ddd888888888 d8888888888b dd88888888b d8888888888888888b d8888888888888b
Y88888888888888888888888888888888 888888888888888888P Y888888888888888888 ddd88888888888888 d8888888888888 dd88888888888888 d8888888888888888888b d8888888888888b
Y888888888888888888888888888888P YY88888888888888P Y88888888888888P d888888888888888888 d888888888888888 888888888888888888 d8888888888888888888888b d8888888888888b
Y8888888888888888YYYYYYYYYYYYP Y888888888888b d888888888888b Y88888888888888888d88888888888888888 Y88888888888888888d8888888888888888888888888b d8888888888888b
Y888888888888b d888888888888b d888888888888b Y88888888888888888888888888888888P Y88888888888888888888888888888888888888888 d8888888888888b
d888888888888b d888888888888b d888888888888b Y88888888888888888888888888888P Y8888888888888P Y88888888888888 d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 YYYYPP d888888888888b Y8888888888888 d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b 8888888888888 d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b 8888888888888 d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b d888888888888 dddddddddddddddddddddb d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b d888888888888 d88888888888888888888b d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b d888888888888 d88888888888888888888b d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b d888888888888 d88888888888888888888b d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b d888888888888 Y88888888888888888888P d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b d888888888888 d8888888888888b
d888888888888b d888888888888b d888888888888b d888888888888 d888888888888b d888888888888 d8888888888888b
d888888888888b Y888888888888b d888888888888b d888888888888 d888888888888b d888888888888 d8888888888888b
Y8888888888888 Y8888888888888 d888888888888b d888888888888 d888888888888b d888888888888 d8888888888888b
Y8888888888888b Y88888888888888b dd8888888888888b d888888888888 d888888888888b d888888888888 d8888888888888b
8888888888888888bbbddddd888b 8888888888888888888888888888888888888b d888888888888 d888888888888b d888888888888 d8888888888888b
Y88888888888888888888888888P Y88888888888888888888888888888888888888b d888888888888 d888888888888b d888888888888 d8888888888888b
888888888888888888888888P 8888888888888888888888P 88888888888888888bb d8888888888888bb d8888888888888 8888888888888b ddd8888888888888888888bbb
Y8888888888888888888P Y888888888888888888P 888888888888888888P dd88888888888888888888bb dd88888888888888888888b dd88888888888888888888b dd8888888888888888888888888888888888bb
Y88888888888888P Y88888888888888PP 88888888888PPP 888888888888888888888888 Y8888888888888888888888P Y8888888888888888888888P Y8888888888888888888888888888888888888
YYYYYPPPP YYYYYPPPPP YPPPPP
turn-1 and turn-1-mini
Our two end-of-turn models: sizes, speed, how they are built, and their licences.
Kollaikal RupeshRead →