Yes, llama-swap and I use it for home assistant text-gen notifications, basic coding tasks, etc

If anyone here self-hosts definitely check out llama-swap as it has some nifty features for hotswapping LLMs, image generation models and voice models.

Bruno

Extremely conservative/risk averse and slow to implement basically anything new, unless it’s super flashy like AI lol

This is really cool in theory, I doubt my corp would let me use it though :(

Oh definitely, it was overwhelming at first for me too, but that’s how one learns imo

Now that I’ve been using it for a year and a half it’s much better but those first few months were quite the learning experience lol

Surprised no one has mentioned proxmox (at least not the top 10 threads I saw first)

Basically debian with a webui specifically for spinning up VMs and LXCs and managing storage and such, then install whatever distro in an LXC and run docker in that.