braindead-local-intelligence Turns a Radeon and Podman 5.0 Into a Local AI Stack
A podman-based starter kit that runs an OpenAI-compatible LLM, image generation and web search locally, swapping models on demand and freeing VRAM when idle.
braindead-local-intelligence is worth one evening if the machine has an AMD card and podman 5.0 already on it. It starts four containers — an OpenAI-compatible LLM server, an image generator, SearXNG for web search, Open WebUI as the chat window — behind one address, with a token for outside access. The name is a pun about not thinking, and starting it is the thinking-free part.
The part I would steal is the memory handling. One model sits in memory at a time: a request loads the right one, a few idle minutes unload it, and the VRAM goes back to whatever else wants it. Swapping models is a copied config block and a restart; the clients never change. I expected a fixed model per container and got something better.
The model list covers small text, image understanding, image generation, and two that modify an existing picture. Requirements: Linux, rootless podman, membership in render and video. Defaults land in ~/.local/share/ia-locale/models, about 8 GB. AMD only — NVIDIA was never tested, because the author has no card. Editor’s note: the 12 GB ceiling and 4 GB floor come from one person’s hardware, not a sweep.
Thousands of model files is the real friction in local AI, and a short curated list behind one endpoint removes it. On a 2021 build with 32 GB of RAM and a Radeon RX 6700 XT, Qwen3.6 35B-A3B with web search gave good results, at a few minutes per answer. Picture that wait, then run ia info before pulling the 8 GB of weights.