← All troubleshooting topics

Performance & VRAM

When the GPU runs out of room, the UI feels heavy, or generation is slower than the hardware should manage, there are a handful of dials that almost always fix it.

Ollama crashes with “out of memory” or the model fails to load

A pull works, but the model fails to load or the reply fails with “ran out of memory loading or running the model. Lower the context length or pick a smaller model.”

1. Turn on Low VRAM in the parameters panel (or globally with Settings → Features → Low VRAM mode). This forces smaller batches and a leaner KV cache.
2. Lower Context Length in the same panel - a smaller context window uses dramatically less VRAM.
3. Set GPU Layers (Advanced view; blank uses the model’s default - or Ollama’s auto-detect when it has none - and 0 means CPU only) below the model’s layer count to push more of it onto CPU/RAM at the cost of speed.
4. Pick a smaller quantization (for example q4_K_M instead of q8_0) or a smaller parameter size.

Onboarding sized my machine wrong

The model recommendation calls a model too big (or too small) for what your hardware actually manages, or names a constraint you don’t recognise.

The card names the number it used - “Based on 8.0 GB VRAM · NVIDIA GeForce RTX 4060 · 120 GB free on disk”, or a RAM figure when there’s no usable GPU reading. Which one you get:

• Windows reads dedicated video memory directly, for any vendor.
• Linux reads nvidia-smi, then the amdgpu sysfs node. Intel Arc isn’t covered and falls back to system RAM, which is usually more than the card holds - so the pick can overflow it. If Ollama runs on your Arc, choose a variant that fits its VRAM by hand.
• macOS deliberately uses system RAM: Apple Silicon shares it with the GPU, so a separate figure would double-count.
• Adapters under 1 GB - and any adapter that reports unified memory on Windows, or that amdgpu reports as an APU (an AMD processor’s built-in graphics) on Linux - are treated as integrated graphics and ignored, since they carve out system RAM the RAM path already counts. With several GPUs, the one with the most VRAM is used.

Any probe that fails falls back to RAM. Nothing here restricts you - every catalog entry stays pullable, and the custom-tag field takes any tag at all. The badges are advice, except Not enough disk space, which does disable that row’s Pull button. If a model marked Runs slowly feels fine in practice, that’s the expected direction of error for a rough estimate - Ollama publishes no runtime memory figures to size against.

The UI feels sluggish

Scrolling, typing, or window resizing is choppy.

• In Settings → Appearance, switch the theme from Aurora to Solid. Aurora’s heavily blurred gradient and translucent panels are heavy on weak GPUs.
• Long chats mount only their latest 50 messages; Show earlier messages, Search in chat or jumping to an old pinned response mounts more. Switch to another chat and back to collapse them again.
• A code block over 50,000 characters in a reply is shown without syntax highlighting, which would otherwise hold the UI for seconds.

The first reply takes forever, even on a fast model

Sending the first prompt of the day takes 10+ seconds before streaming starts; subsequent replies are instant.

Ollama loads the model into VRAM on first use. To preload at launch, turn on Preload on startup under Settings → General → Default model (available when the default model is an Ollama model). Only that model is warmed, at the cost of pinning VRAM as soon as Loach opens - or as soon as you unlock it, if you use an app lock. It then stays loaded only as long as Settings → Features → Keep model loaded allows (5 minutes by default); raise it to keep the model resident between prompts. See Model preloading.

Making Ollama generate faster

Generation is slower than you’d expect, or you want to squeeze more tokens/sec out of your hardware.

These are environment variables on the Ollama server (not Loach settings) - set them where ollama serve runs, then restart it. As of Ollama 0.5+:

• OLLAMA_FLASH_ATTENTION=1 - enables flash attention, which lowers memory bandwidth and usually speeds up generation, especially at longer contexts. Newer builds enable it by default for some models on the new engine; setting it explicitly is harmless.
• OLLAMA_KV_CACHE_TYPE=q8_0 - quantizes the KV cache to 8-bit, roughly halving its memory so you can run a bigger context (or a bigger model) in the same VRAM. Only takes effect with flash attention enabled. q4_0 is smaller but lossier; f16 (the default) is highest quality.
• OLLAMA_KEEP_ALIVE=30m (or -1 for “until unloaded”) - how long Ollama keeps a model resident after a request, so a reply after a pause skips the cold reload. Loach also exposes this as Settings → Features → Keep model loaded (5 min, 30 min, 1 hour, Always). Until you pick an option there, Loach sends nothing and this env default applies - even though the control already shows 5 min highlighted. Once you click any option, it’s sent with every request and overrides the env default, and there’s no way back to “unset”, so pick the one that matches your env var.
• OLLAMA_NUM_PARALLEL=1 - caps concurrent requests per model. The default splits VRAM across parallel slots; pinning it to 1 gives a single chat the whole budget (and the largest usable context).
• OLLAMA_MAX_LOADED_MODELS=1 - keeps only one model resident at a time. Useful on a single-GPU machine where a second model would evict the first or spill to system RAM.

Setting environment variables for Ollama: on Windows, use System Properties → Environment Variables (or setx OLLAMA_FLASH_ATTENTION 1) and restart the Ollama tray app. On Linux with systemd, systemctl edit ollama (add a [Service] block of Environment="OLLAMA_…=…" lines) then systemctl restart ollama. On macOS, launchctl setenv OLLAMA_FLASH_ATTENTION 1 and relaunch the Ollama app.