Providers

Loach doesn’t ship with built-in models. It connects to a provider you choose - local or remote. Both kinds are available side-by-side; switch from the chat header.

Ollama (local)

Ollama is the easiest way to run open-weight models on your machine. Loach probes the Ollama base URL (http://localhost:11434 by default) as soon as it starts; any models you’ve pulled appear in the chat-header model picker without a refresh.

Setup

1. Install Ollama from ollama.com/download.
2. Start the server: ollama serve (or just open the Ollama app - it runs in the background). Loach can also start it for you - see below.
3. Pull at least one model: ollama pull gemma4:e4b. If you skip this, the onboarding wizard recommends one sized to your machine.
4. Open Loach. The model appears in the chat-header dropdown.

The Models library

The Models tab inside Loach is a full UI on top of the Ollama HTTP API. From there you can:

• Pull new tags by name.
• Duplicate a model under a new tag.
• Customize an existing model - system prompt, prompt template, parameters - and save the result as a new model.
• Delete models you no longer need.

See Local model management for the details.

Starting Ollama from Loach

You don’t have to run ollama serve yourself. When nothing is answering, the model picker in the chat header lists Ollama as Not running and offers a Start Ollama button that launches the server and swaps in the model list once it responds. Settings → Providers → Auto-launch Ollama does the same thing every time Loach opens - off by default, and only for an Ollama whose base URL points at this computer. Loach looks for the ollama program on your PATH and in Ollama’s default install location, and a server it started keeps running after you quit Loach.

Settings → Features → Keep model loaded controls how long Ollama keeps a model resident after a reply (5 min, 30 min, 1 hour or Always), so a reply after a pause skips the cold reload.

Pointing at a remote Ollama

If Ollama is running on another machine on your network, change the Ollama base URL in Settings → Providers. Make sure the remote server is reachable from your machine and listening on a routable interface (OLLAMA_HOST=0.0.0.0 on the server). Test connection under the field queries the server and reports how many models it lists, or a readable error, so you can confirm a custom base URL works without opening a chat. A 404 usually means the URL has a path on it - use the server’s address alone, such as http://localhost:11434, not …/v1 or …/api. A 401 or 403 comes from something in front of Ollama asking for a login: Loach sends Ollama no credentials, so point it at Ollama directly. Start Ollama and Auto-launch Ollama only work for this computer - a remote server is yours to start.

For any provider, Loach refuses a base URL on a link-local address (169.254.x.x, fe80::), the range where cloud-metadata services live.

Requests to both providers go through a proxy when your environment or system settings name one. Web fetch, Library links and HTTP MCP servers are different: they always connect directly.

OpenAI-compatible endpoints

Anything that speaks the OpenAI Chat Completions protocol works - the real OpenAI API, llama.cpp’s llama-server, LM Studio, vLLM, LiteLLM, OpenRouter, Groq and other proxies. Configure it in Settings → Providers:

1. Set the API base URL, or pick one from the presets dropdown at the end of the field. It’s saved as you type. Loach appends /chat/completions and /models to it, so include the /v1 (or other prefix) the server’s docs give.
2. Paste the OpenAI API key and click Save. It’s stored in your OS credential manager (Windows Credential Manager, macOS Keychain or the Linux Secret Service) - never in plain text on disk - and the field stays blank afterwards, showing •••••••• (stored). Clear removes it. Local servers such as llama.cpp, LM Studio and vLLM usually need no key at all.
3. Use Test connection to verify the base URL and key before opening a chat - it lists the endpoint’s models and reports the count, or a readable error if something is off. Save a typed key first; the button waits for it.
4. Pick a model from the API group in the chat-header model picker. It lists every model the endpoint returns; the menu scrolls, and typing a model’s first letters jumps to it.

On Windows, the Credential Manager can’t hold a key longer than 1,280 characters, so Loach refuses one that long when you save it. On Linux, a Secret Service that starts after Loach is picked up on the next save or read - no restart needed - and while none can be read, a server without a key still lists its models.

With the default OpenAI base URL, Loach lists models only once a key is saved - so clearing the key empties the API list in the model pickers instead of leaving models that could only fail at send time. A base URL of your own is listed with or without a key.

Common base URLs

The presets dropdown at the end of the base-URL field fills in the documented endpoint for OpenAI, llama.cpp (llama-server), LM Studio, vLLM and LiteLLM in one click. The table above is the fuller reference for the providers that aren’t in that dropdown.

Reasoning models

OpenAI’s reasoning models (the o-series and GPT-5) turn down some of the fields Loach sends, such as max_tokens or temperature. When a server answers 400 naming one of those fields, Loach adjusts the request and sends it again - max_tokens becomes max_completion_tokens, and the other fields are dropped - so these models work with no setup. The adjustment lasts for that reply, so each new message makes the first, refused request again.

Servers that stream a model’s reasoning - as reasoning_content, or as reasoning like Ollama’s /v1, OpenRouter and newer vLLM - show it in a Thinking block above the reply.

Context window

An OpenAI-compatible server decides the context window itself, so Loach never sends one. To size the context bar and how much of your Library sources fits, it assumes 32k (32,768 tokens) unless the chat sets a Context Length in its parameters panel. Set it to your model’s real window for an accurate bar.

Embedding models for the Library

The Library searches big sources by keyword out of the box. To search them by meaning as well, set an embedding model in Settings → Library. It runs on either provider: on Ollama (Loach suggests models such as embeddinggemma and offers to download one), or on your OpenAI-compatible endpoint (text-embedding-3-small, for example). With the OpenAI-compatible endpoint, your sources’ text and every message that searches them are sent there - in chats with a local model too. Changing the provider, the model or that provider’s base URL re-embeds every source in the background. See Library → Search.

Default model

Settings → General → Default model picks the model new chats open with: whichever you used most recently, the last model you used on a given provider, or one specific model. You can still change the model per chat from the header. See Default model selector.

Preloading and Low VRAM

Two settings worth knowing about for local providers:

• Model preloading - Preload on startup, under the default-model picker, warms your default Ollama model into VRAM at launch so the first message streams faster.
• Low VRAM mode - a global or per-chat toggle that sends Ollama’s low_vram flag with every request. Useful on lower-spec devices.