← All features

Provider selection

Loach is a chat client, not a model. Out of the box it speaks two protocols: Ollama for local models running on your machine, and any OpenAI-compatible endpoint for hosted or self-hosted services. You can keep both wired up at once and switch between them at will.

Local: Ollama

The default backend. Loach talks to an ollama serve process over HTTP and probes it as soon as the app starts, so the models you’ve pulled are in the model picker without a refresh. The picker checks again whenever you change a base URL or key, and has its own refresh button. If Ollama is not running, the picker lists it as Not running and offers to start it for you (see below); the Models tab shows the connection error.

When the chosen model supports them, streaming, multimodal images and thinking-mode reasoning are passed through to the daemon without extra configuration. By default Loach looks for Ollama at http://localhost:11434 - change the Ollama base URL in Settings → Providers if it lives on another machine or port.

Starting Ollama from Loach

You do not have to run ollama serve by hand:

• Start Ollama - when the daemon is not answering, the model picker in the chat header shows a Start Ollama button. It finds the ollama program (on your PATH or in Ollama’s default install location), starts it on the host and port of your base URL (port 80 or 443 when the URL names none, matching where Loach connects), waits up to 20 seconds for it to answer, then swaps in the model list. Errors - Ollama not installed, a base URL on another machine, a server that exits straight away - show in the menu itself.
• Auto-launch Ollama - an opt-in switch in Settings → Providers does the same thing every time Loach opens, if nothing is already running. It is off by default (starting a background service is your call) and limited to a base URL that points at this computer - a remote Ollama is not Loach’s to start. The button above has the same limit.
• Left running - an Ollama that Loach started keeps running after you quit Loach, since other apps may be using it.
• Keep model loaded - Settings → Features sets how long Ollama keeps a model resident after a reply: 5 min, 30 min, 1 hour or Always. Until you click one, Loach sends nothing and Ollama’s own default applies (OLLAMA_KEEP_ALIVE, or 5 minutes), even though 5 min shows as selected. Once you pick one, it’s sent with every request and overrides the daemon’s default. OpenAI-compatible providers ignore it.

Cloud or self-hosted: OpenAI-compatible

Any endpoint that implements the OpenAI /chat/completions API works. That includes the real OpenAI API, plus vLLM, LM Studio, LiteLLM, OpenRouter, Groq and other proxies. Set the API base URL and OpenAI API key in Settings → Providers; the key is optional for local servers such as llama.cpp, LM Studio and vLLM.

A presets dropdown at the end of the base URL field one-clicks the URL to the documented endpoint for OpenAI, llama.cpp (llama-server), LM Studio, vLLM, or LiteLLM - handy when you switch between local proxies and hosted services.

The catalog is fetched with the saved key - or without one when the base URL points somewhere other than the OpenAI default, since local servers accept keyless requests. With the default URL and no key, Loach skips the catalog request instead of collecting 401s, and clearing the key empties the API model list so the pickers never offer a model that can only fail at send time.

API keys are saved into the OS credential store (Windows Credential Manager, macOS Keychain or the Linux Secret Service), not written into the SQLite database, and never handed to the app’s web view - the Settings field only shows whether a key is stored. The key is sent only over https://, or over http:// to this computer; an http:// server elsewhere on your network gets requests without it. Windows Credential Manager can’t hold a key longer than 1,280 characters, so Loach refuses one up front (“OpenAI API key is unusually long … - refusing to store”). On Linux, a Secret Service that wasn’t running yet when Loach first needed it is connected on the next save or read, without a restart; until then a local server’s catalog is still listed, as if no key were stored.

Some servers need special handling, which Loach does on its own:

• Reasoning models - OpenAI’s o-series and GPT-5 refuse max_tokens and any non-default temperature. When a server turns a request down with a 400 that names a field Loach set itself, Loach resends it with max_completion_tokens in place of max_tokens, or without the refused field (temperature, top-p, the penalties or the seed). Each field is retried once per reply and remembered only for that reply, so every new message goes through the refusal again before the answer starts.
• Reasoning text - reasoning a server streams as reasoning_content, or as reasoning (Ollama’s /v1, OpenRouter, newer vLLM), shows as the reply’s thinking block, even though the Thinking toggle stays off on these chats.
• Context window - these servers don’t report their window and ignore num_ctx, so Loach assumes 32K (32,768 tokens) unless the chat sets a Context Length. The figure only sizes Library sources and the context bar; it’s never sent.

Test connection

Each provider section exposes a one-click probe so you can verify a base URL (and API key) before opening a chat:

• Ollama - queries /api/tags and reports Connected with the number of models available, or Connection failed with the error.
• OpenAI-compatible - calls the endpoint’s /models listing with the stored key and reports the model count or the error. Disabled while there is an unsaved key still in the input (“Save the key first, then test.”).

A failed test adds a hint under the raw error when it recognises the cause: an unreachable server, a URL that doesn’t parse, an unauthorized key, a /models path that wasn’t found, or a blocked link-local address. The Ollama test sends no credentials, so there a 401 or 403 points at a proxy in front of Ollama that asks for a login (aim Loach at Ollama directly), and a 404 means /api/tags wasn’t found - use the server’s address without a path, such as http://localhost:11434 rather than …/v1 or …/api.

The model picker

Every chat has a model dropdown in its header, labelled with the current model and provider. It lists both providers’ catalogs grouped by provider: Ollama, with a green tick when it’s running or an amber warning when it isn’t, and API, which lists every model the endpoint returns (or Not connected when there is no key). A long list scrolls, and typing a model’s first letters jumps to it. With no Ollama models and no API key, a Configure providers… entry opens Settings → Providers. Picking a model:

• Persists onto the chat - reloading the app remembers which model that chat last used.
• Updates the “most recent” pair - used by Default model → Use most recent when starting fresh chats.
• Unloads the old model - for Ollama, the previous model is released from VRAM and the new one’s parameter cache is warmed for the parameters sidebar.

You can also switch from the composer: /model <name> takes an exact name or a unique part of one - write ollama/<name> or openai/<name> when both providers serve the same id - a bare /model opens this picker, and /list models shows what’s available. /unload frees the chat’s Ollama model from memory; it loads again with your next message. See Slash commands.

When a request fails

A failed reply ends with a one-line warning that names the provider and base URL, then says what went wrong in plain words - for example “Ollama (http://localhost:11434) - could not reach the endpoint. Is it running and the URL correct?” The common cases:

• Out of memory - “ran out of memory loading or running the model. Lower the context length or pick a smaller model.” See Performance & VRAM.
• Key withheld - “API key withheld because the base URL is http:// to another machine. Switch to https:// to authenticate.”
• Authentication - “API key invalid or expired” (401) or “API key not authorized for this endpoint or model” (403).
• Reachability - the endpoint could not be reached, the host name didn’t resolve, the request timed out, or the HTTPS certificate is broken.
• Server answers - endpoint or model not found (404), rate limited (429), an upstream server error (5xx), or a conversation too long for the model’s context window.
• Cut off mid-reply - when the server fails partway through (an Ollama runner that crashes or runs out of memory, an error from vLLM or llama-server, OpenRouter ending the reply with an error), the text so far stays and the reply ends with the warning - for example “Ollama stopped mid-reply: …” or “OpenAI stream stopped mid-reply: …” - instead of looking finished.

Anything else is shown as the provider’s own error text.