For my #FIAF presentation & workshop in Bologna end of the week, I've put together a clean "#PlugAndPlayAI" setup:
#USB #LiveDistro: #MXLinux #GNU #Debian with #Ollama, #LLava and #OpenWebUI in #Docker and #LLM files on another USB3 #nvme
Fast like a local install. Given enough RAM and CPU/GPU.
Scary how easy it is to do that.
Bonus: Creates self-replicating #ISO boot images of itself (in current configuration) in less than 5 minutes. 
.
Wow.
From OpenWebUI:
> 
Screen Reader Support for Response Messages
All chat responses are now fully compatible with screen readers, making the platform more inclusive and accessible for everyone.
Well um, I mean, I don't use a hearing aid... But um, that's... Good... I'll see if it works.
@Gina If I had to choose, I'd probably go with #Ollama (which has been mentioned several times already). It's licensed under the MIT license and the models are about as close to open source as you can get. When I play with LLMs, it's what I use. Locally run and with an API that could be used to integrate with other stuff. I also have #OpenWebUI to make things prettier. Both can run locally, though OpenWebUI can integrate with cloud LLMs too. Of course, tomorrow everything could change.
#nixos roadblock getting a m.2 #google Coral TPU up 
no more steam left this Sunday morning.
Swapping for #Ubuntu and Docker so I can start testing with #OpenWebUI.
I have not been defeated yet though! Once my PCIe to m.2 e-key adapter shows up I will attempt to get the device working with NixOS once again. I want the device on the main app server.
I basically have a DIY Perplexity setup running in OpenWebUI (which is running politely alongside Plex). I'm using Mistral-Large with web search via SearXNG and the system prompt that Perplexity uses for their Sonar Pro searches.
And since OpenWebUI has an OpenAI-compatible API, I can connect to it from this GPTMobile app on my phone and query my custom search assistant model.
I set up #OpenWebUI on one of my more powerful servers, and it is fantastic. I'm running a couple smaller local Llama models, and hooked up my Anthropic and OpenRouter API keys to get access to Claude and a bunch of other models including Mistral, DeepSeek, and others. I also linked up my Kagi search API key to give web search capabilities to the models that don't have a web index. I will probably lower my Kagi Ultimate subscription to Professional since I no longer have a need for their Assistant.
