← All features

Concurrency & queue

Only one model generation runs at a time across all of your chats. Everything else waits in a FIFO queue - and you can always jump the line when you need to.

One generation at a time

Local models are heavy on hardware: most of the time only one prompt can stream at once without thrashing VRAM or pinning the CPU. Loach enforces this at the application level, regardless of which chats or providers are involved. The benefit is predictable performance - no two chats fighting for the same GPU - and a clear mental model of what is happening.

Inside a single chat the rule is even simpler: one request per chat. While a reply streams, the send button becomes a stop button and the message box reads Replying - press the Stop button to cancel and ask again…. While the chat waits in the queue, the box reads This chat is waiting in the queue… and sending stays disabled until its prompt runs or you cancel it. Sending again while a message is still being prepared does nothing and keeps your draft.

The queue

Send a prompt while another chat is busy and your request is parked. The chat’s row in the sidebar shows a spinner so you can see at a glance which chats are waiting, and the chat itself shows a Waiting for other chats to finish… card under your message. The queue is strictly first-in, first-out - earlier requests run before later ones.

You can keep typing in other chats while you wait, switch around the sidebar, or browse Spaces and Snippets - the queue keeps running in the background, also while the window is hidden with Hide to tray from the tray icon’s menu. On Windows and Linux closing the window quits Loach, so anything still waiting in the queue doesn’t run; on macOS closing only hides the window, and the queue keeps going.

Respond now

The waiting card in a queued chat has two buttons:

• Respond now - stops whatever generation is currently running (keeping its partial output, see below), moves this chat to the front of the queue and starts it immediately.
• Cancel - takes this prompt out of the queue. Your message stays in the transcript, and no reply is generated for it. The /stop slash command does the same.

Respond now is the right tool when the in-flight reply has gone off the rails, or when you are switching contexts and the queued chat is the one you actually care about.

Replies waiting for your approval

When a reply pauses on a tool approval card - an MCP tool call, or a change to a file in the chat’s folder - it keeps its turn, so other chats queue behind it until you answer - or until the card times out unanswered after 10 minutes. If the card is in a chat you aren’t looking at, Loach shows a notice with an Open button, and a queued chat’s card reads Waiting for your approval of a tool call in “…” with a Go to it button. Respond now still works there: it cancels the reply that was waiting for approval. See Tools, MCP & folders troubleshooting if a reply seems stuck.

While Private Chat is open

Opening Private Chat stops any regular reply that is streaming and holds the queue: nothing waiting in your regular chats starts until you close the overlay (or Loach locks, which ends the Private Chat session), and then the queue picks up where it left off.

Cancelling and errors

Whether you stop a stream yourself (the send button becomes a stop button while streaming, or type /stop) or Respond now pre-empts it, whatever was written so far stays in the transcript. If the provider fails or drops the connection mid-reply, the partial output is kept too, and an italic ⚠ error line is added at the end so a failed reply is never silently empty - you can see how far it got, decide what to do, and retry without losing context. Stopped and failed replies don’t get a metrics chip.

Archiving or deleting a chat also stops its reply, or takes it out of the queue if it was still waiting. The same holds for a message still being prepared - while Loach fetches its links or reads its sources, before it streams or joins the queue: /stop, archiving, deleting or /clear then keeps its reply from starting at all.

If Loach can’t create the reply at all, a Couldn’t start the reply notice names the chat and the reason, and a chat you aren’t looking at gets its unread dot, so a message is never left silently unanswered.