Concurrency & queue
Only one model generation runs at a time across all of your chats. Everything else waits in a FIFO queue — and you can always jump the line when you need to.
One generation at a time
Local models are heavy on hardware: most of the time only one prompt can stream at once without thrashing VRAM or pinning the CPU. Loach enforces this at the application level, regardless of which chats or providers are involved. The benefit is predictable performance — no two chats fighting for the same GPU — and a clear mental model of what is happening.
Inside a single chat the rule is even simpler: only one in-flight request per chat. Pressing send while a reply is already streaming in the same chat is a no-op.
The queue
Send a prompt while another chat is busy and your request is parked. The chat’s row in the sidebar shows a spinner so you can see at a glance which chats are waiting. The queue is strictly first-in, first-out — earlier requests run before later ones.
You can keep typing in any chat while you wait, switch around the sidebar, browse Spaces or Snippets, or just close the window — the queue survives until the runner finishes.
Respond now
If a chat is waiting and you do not want to wait any more, the chat header offers a Respond now action. It cancels whatever generation is currently running (preserving its partial output, see below) and immediately starts yours.
This is the right tool when the in-flight reply has gone off the rails, or when you are switching contexts and the queued chat is the one you actually care about.
Cancelling and errors
Whether you stop a stream manually (the send button becomes a stop button while streaming) or the runner is pre-empted by Respond now, the partial output is kept in the transcript along with a short error tail so the message bubble is never silently empty. The same applies if the provider drops the connection — you can see how far it got, decide what to do, and retry without losing context.