Speed is only useful when it is dependable enough to disappear into the experience.
Why the old path had to change
The first version of our inference path was designed for excellent outputs, not continuous conversation. Every new request rebuilt too much context, and the time between a person speaking and hearing a reply was becoming visible.
We replaced it with a streaming graph that keeps useful work in flight. The model does less waiting, and the interaction feels less like a sequence of requests.
The bugs we found by listening
The most revealing failures were not in dashboards. They were in conversation: a reply that began too early, a pause that cut through a sentence, an old context frame that surfaced one turn late.
We built playback traces next to our system traces so engineers could hear the consequence of a timing decision. That changed how quickly we could find the work that mattered.
What shipped
The new path is faster, but more importantly it has a clearer contract. Each stage knows when it may continue, when it must wait, and how to hand off gracefully.
The result is a system that feels calmer under load—a quality that is hard to benchmark and easy to notice.
Keep a good signal close.
New research and product notes from Sunesis, sent occasionally and thoughtfully.
Subscribe to the journal