Introduction to Ollama Frontends
Hi, I'm Matt. One of the unique aspects of Ollama is its dual nature as both an end-user tool and a developer tool with an API endpoint. This allows developers to program against it. In fact, the GitHub repo homepage lists numerous Community Integrations, including desktop and web tools, over 30 or 40 or even 50 entries.
With so many options, it can be overwhelming to choose the right one for your needs. To help with this decision-making process, I started creating a chart and document on my blog at technevangelist.com that highlights the differences between these various tools. However, as I began exploring the first few Integrations, I discovered that many of them had significant issues. It felt like I was just highlighting the bad stuff rather than trying to be helpful.
And so I decided to create a video series that looks at each of these tools. My intention is to provide helpful insights into each app, highlighting what makes them compelling and point out what might be missing from their offerings from my perspective. Perhaps this will help you find the perfect tool for your needs or even inspire you to build your own application that combines the best features of multiple tools. Let's take a closer look at some of these desktop and web apps and see what they have to offer.
First Look at Open Web UI
It's important to remember that while I was a founding member of the Ollama team, I am no longer working for Ollama, so these are simply my opinions and not reflective of anything that anyone in the Ollama team believes. Okay, so let's look at one of these tools. With so many, it's hard to know where to start, so I decided to choose the application that I think is among the favorites for most people that are working with Ollama. It's the application I see most often mentioned in the Ollama Discord, which by the way you can find at discord.gg/ollama, and it's called Open Web UI.
It used to go by something like Ollama Web UI, but I think the author wants to be less tied to Ollama specifically. There are a lot of things to really like about this tool. It's visually very attractive to look at. It has a few themes so you can tweak it to be the way you prefer, whether it's light mode or dark mode, and I wish dark mode would die because it hurts my eyes. It gives me good suggestions about some things I could try, although I wish those suggestions would change occasionally, and also has a very opinionated way of working with prompts, which is interesting. I initially thought this was a little distracting, but I think I came around to to love it.
UI Walkthrough & Initial Test
Well, let's take a look. So when you first bring up the tool, you're asked to log in. I hate it when tools do this. If something's running locally, don't make me sign in. If you want to provide this layer as an option for those who want to host it, great, but make it an option.
In this UI, we have a few different sections. Front and center is the chat window. At the bottom, we can enter in a message and press enter and then see the results in the main window. Right up at the top, we can choose a model. Now, this is a pretty critical step, so I wish it wasn't as visually hidden. On the left, we see a big button for a new chat and then, curiously, sections for model files and prompts. Okay, we'll come back to those later. And then there's documents, and then below that is a list of chats you've had, along with a search box for searching those chats.
Let's ask a question: Why is the sky blue? This is my go-to question, and my model is set to Phi-3, which is a tiny model that is performing beautifully for so many use cases. Pretty quickly, we get a pretty amazing response. But I'm already getting ahead of myself a little bit.
Finding and Installing Open Web UI
Let's go back to the webpage and try to figure out how to get this thing installed. We can go to ollama.com and then click on the GitHub link. Now, this is the repo readme. Scroll all the way down to the end, well, almost all the way to the end, and we get to the top of the Community Integrations list. And look at the web and desktop tools, and here is Open Web UI at the top. This takes us to the GitHub repo for Open Web UI.
Looking at the readme, it looks like an incredible tool. I see local RAG integration, intuitive interface, code syntax highlighting, theme customization, Markdown and LaTeX support, web browsing capability, and so much more. Looks really, really cool, and this list of features just goes on and on and on. Well, let's scroll up closer to the top and click on the releases link. And this is interesting because we can see that it's not just one release. There was a release just in the last couple of days, and we can scroll through here and we can see that there are releases happening on a pretty regular basis. And they're not small releases, they're not just documentation fixes, there's significant new features that are being added, which is incredible.
Detailed Docker Installation Guide
So, let's look at the installation process. Now near the top is how to install, and right below that is a quick start for Docker. Now, not everyone loves Docker. Some people don't like it for, I don't know, whatever reason. They might have misunderstood that licensing fuffle that happened a few months and another one a few years ago, thinking that they need to pay for Docker when most individual users don't need to pay anything.
But if you're working with Docker and Ollama is running locally, then you can use this command. -d says run this as a daemon process with the UI running in the background. -p publishes the port 8080 in the container to port 3000 on your host. If you'd prefer to access on a different port, replace 3000 with anything else. The add-host parameter says to use Ollama on the local machine. -v creates a volume located at open-webui and mounts that to /app/backend/data in the container. name sets the name of the container, and restart always means that if it stops for any reason, it's coming back up. And finally, we're specifying that the container registry and image we should use is on ghcr.io.
If you don't want to use Docker, there are instructions you can follow over at this link. But once you have it running, you should be able to visit localhost and the port, which for me is 3000.
Exploring the Settings Panel
And here we have that sign-in page. The first time we get here, we need to create a new account, so click Sign Up and enter your name, an email, and a password, and click Create. Now you're dropped into the UI. The first thing you probably want to do is set the model to use. Come up here to this dropdown and type in the name of the model you want. I have a lot of models available to me because I've downloaded and created so many. Here I've chosen Llama 3, and now you can add a question.
Let's go to settings, which you can find up top here or down in the bottom corner at your name. Under General, we can set a theme and language, but then there's a system prompt. I'm not really sure why anyone would want to set what I assume to be a global system prompt. Does this get overridden on most models? Because the system prompt is set in the model, and there's no way to find out without digging around some docs. And then it looks like we can set some more default parameters, which you really want to be different for each model.
Next is Connections. We can set an OpenAI API URL as well as an API key or update the URL to use for Ollama. And I can also add additional hosts. Not sure what happens when I have multiple hosts because I can't choose the host for a model when I set the model to use, so that's a little strange.
In the Models section, we can set the host to use to manage models. It's not clear if this is different than the connection screen, and even though I set a second host, I only see the one default host here. To the right is a button to update all models. Now, unless you have a small number of models, don't click this button. It updates models in the most inefficient way possible, pulling every model in your list. It hung on my machine. Then you can pull a new model if you know the name of it, though this is a much more awkward way than the CLI. You can also delete models from here. Now, it's interesting that they offer a way to upload what they call models as the source GGUF files, but those are just the model weights and not the full Ollama model, so they're a bit useless until you make that model from that. And then you can do some management of models that you access through LiteLLM.
In Interface, we have some functionality options and some default prompt suggestions. I'm not sure what the value of this is. I could imagine some default system prompts would be a lot better served here. In Audio, we have the ability to set a speech-to-text engine, either using Whisper locally or using OpenAI, but we have no control over which Whisper models to use or which language you speak. And then there's text-to-speech, and again, there's local or OpenAI. The local one is all but useless. For Images, we can use this to access AUTOMATIC1111, which is a bit strange, but the author seems to want to support more than just Ollama, so that seems like it could be useful to some. And that's all that's really interesting here.
Core Feature: Comparing Multiple Models
So far, everything feels pretty good about Open Web UI. Things work roughly the way you would expect, and it generally seems to perform well. But here's where we start getting a bit more interesting. Let's come up to the place where we choose a model. There's this dropdown, but there's also this plus sign. So if I click this, I can add a second model, and a third, and a fourth, and so on.
So what happens now? Well, ask a question and you shall see. Why is the sky blue? We wait for it to respond, and we can read the answer. Take a look at the top. I set it to use Phi-3 and Llama 3, and so now at the bottom I see it showing two of two. Click the arrow, it's now showing me the output of the other model. So now you have a way of configuring multiple models, and it's going to ask the same question to each. There's no magic here. It's going to take longer to ask more models, but it becomes a bit easier to find the right model for you.
There's also a thumbs up and thumbs down button here. The readme seems to refer to some ability to use this to make something better, but there's no info on where that data is going. I couldn't find any way to get the info out. I looked at the SQLite DB that the app creates and nothing seems to have been added. You can also edit the answer. Again, maybe this is to help come up with a series of good questions and answers to help fine-tune later, but the feature doesn't seem to be complete.
Powerful Prompt Management with Slash Commands
Okay, let's go up to Prompts on the top left corner and click the plus sign to add a new prompt. Here you can add a phrase you want to use as the beginning of your prompt. Let's try 'summarize'. Enter 'summarize' for title, and it will default to using the same thing for command, and then enter the prompt: "Write a short summary of the following text." Click save and update.
Now go to a new chat and type the slash key. Press enter to select 'summarize', go grab some text from somewhere and paste it in and press enter, and we have a summary. But we had to go through a few extra steps there. So go back to that prompt and add the word 'clipboard' in double curly braces. Save and update, and let's go back to the chat. The text is still in our clipboard, so type /summarize and enter. It has automatically taken what's in my clipboard and added that to the end of the prompt. Well, that's awesome! But maybe you want to summarize it differently depending on the age of the person you're going to share this with. Add "for someone who is" and then 'age' in square brackets "years old." Go to the chat and bring that up again. The text gets pasted in and 'age' is highlighted, waiting for you to enter an age. You can add as many variables in square brackets as you like and as many prompts as you like.
Now, on that prompts page, there's also a link to discover a prompt, but it just takes you to the Open Web UI page. You can scroll down to find a list of prompts folks have shared, but it really should link you right to the right place. Even so, I didn't find that all that useful. I found the prompts tend to be super generic, and it's better to come up with your own. That said, the whole slash command to bring up prompts is amazing, and it may be my favorite feature of the app.
RAG with Web Pages and Documents
Now let's go back to the chat. Now press the pound key, or you may call it the hash key. Type https://technevangelist.com. Now you can ask any question about the content on that page. It doesn't seem to follow any links though, it's just what's on that page. So I can ask about my Patreon at patreon.com/technevangelist, but asking about my employment status with Ollama results in a very generic answer.
Back up on the top left is a section called Documents. Click on it and then the plus sign to add a document. You can add one document or a bunch. Unfortunately, you won't get any feedback until the process is done. But once the doc shows up, you can add tags to the document. Now you can come back to the chat and type the hash key again, and you'll be prompted to choose a document or a collection, which is the tags associated with your docs. It seems to do a pretty good job here. I wish there was some feedback, but maybe that'll come soon.
There are some document settings, but they don't seem to make any impact. I change them, and the next time I go back to this page, they're changed back to the default. Maybe this is something else that needs to get fixed. We saw how often releases are coming out, so who knows, maybe there's a new fix tomorrow.
The Confusing "Model Files" Section
The last thing we haven't looked at is the Model Files section. I have no idea why this exists. They should just make a better experience for working with models on the ollama.com site because it feels like this is just trying to do the same thing, just more awkwardly. I wish this would make it easier to take any model, then apply new parameters and system prompt, give the model a new name, and save it back to the Ollama library. The CLI already does that, but some folks prefer a UI, so it would be a nice addition.
Final Verdict and Key Takeaways
All covered everything, so what do you think? Overall, it's a nice tool, but some of the weird aspects are hard to get past. The whole user thing is a pain, though I'm sure they worked really hard on that. I wish you could just disable that whole piece entirely. What they're doing with the model file section doesn't make any sense to me, but having a better workflow with models would be great. And that button to update models is way too easy to press. So either make it harder to press or update it to compare the hashes first and then do the pull.
The slash command to pull up prompts is pretty genius. I love it. But that's pretty much the only differentiating feature from the CLI. Is that enough to use it more regularly? I don't know. I know there are a lot of folks who are scared of the command line and will be okay with it being a one-to-one version of the CLI tool, but that's not me. So I don't know.
I thought this would be a short video when I started putting it together, and I didn't expect to like Open Web UI as much as I did. I hope this will be the experience with all the other tools on the Community Integrations list. But what do you think? Do you use Open Web UI? Is there anything I missed? Do you think there's a better tool out there? Share that with me in the comments below. I love what folks share in the comments.
Also, be sure to subscribe to my newsletter. You can find it at technevangelist.com/newsletter. Thanks so much for being here. Goodbye.
Bonus Scene: Hydrate Spark Bottle
This is a cool bottle from Hydrate, and so when I haven't drunk any water for too long, it glows and reminds me to drink water.
Outro
drink water
