It already is, this beats the NSA and OpenAI et. al.

Just get a domain and point it at a provider. Now you’re not locked in and can switch at will upon enshittification. Get one of the offline mail archive services like OpenArchiver. Job done.

If it gets you started with local models, by all means go ahead, their onboarding is the easiest and it works. Also a lot of 3rd party stuff uses it as a first class citizen allowing you to try out other things (e.g. Open WebUI) easily as you explore what’s possible. Currently try the Qwen 3.6 and Gemma4 models as best bang for buck, somewhere there’s a does it fit in my machine website that can help (search for it).

That said, basically all roads in local LLM lead to llama.cpp, which gets the innovations first and then others copy their homework. Ollama (looks like they’re angling to go commercial) for a long time used it internally without attribution, now they use a bodged up engine of their own that is less performant and almost certainly a copy (possibly vibe coded) of llama.cpp. They heavily encourage using their own models / quantizations and don’t let you play with a lot of parameters without a lot of friction (possibly because they’re not implemented yet, but who knows, low transparency). You get the picture, wannabe techbros. That’s off the top of my head, search for more authoritative sources.

After you’ve gotten the hang of things, have a look at llama-swap which just wraps llama.cpp, lemonade if you’re on AMD, vLLM for nvidia, LM Studio for mac.

K, I’ll circle back in a week or so…

Hope so, bet it doesn’t without some tweaking though, OpenAI-compatible seldom is, and ollama is bad for that. Still, worth checking out, I’ll have a go at it sometime soonish and perhaps you’ll see a PR (or some doco in the best case scenario).

Ollama is enshittifying at a rate of knots, have you got a way to use llama-server (or preferably llama-swap) instead ?

Not to mention road-runners (humans) and ACME (OpenAi, Anthropic etc.) extending the metaphor in a different direction… Wile.E. (coyote, suupergenius) might be another name option.

Ahh, ninja’d, I’ll leave it as another vote.