Introduction to AnythingLLM & Deployment Options
Are you looking for a fully private, free AI platform? Let's discover AnythingLLM, the all-in-one AI application. These powerful platforms allow you to compose the optimal AI agent for your use cases by choosing between different large language models such as OpenAI, Azure, Gemini, Ollama, and many more.
Before diving into the platform overview and discovering its features, let's see the options we have to start using it. The most private and ideal solution if you're working solo is to use the desktop version. You simply need to download it and follow the installation steps to get a complete offline LLM platform.
If you're part of a team or need chatbots and AI features for your products, the hosted version is what you are looking for. You can use the official Cloud version starting at $50 a month, self-host it on your server by following the Docker installation guide, or use a platform like ours, Elestio, to deploy the self-hosted version on the cloud provider of your choice while we take care of the installation, backups, updates, and maintenance for you.
Deploying AnythingLLM with Elestio
To install AnythingLLM on our platform, head to elest.io and click on login, then deploy my first service. Search for AnythingLLM and select. Choose between the different Cloud providers, regions, and service plan based on your needs. I will keep the default one and then hit next. Adjust your configuration, level of support, and once you are ready, click on Create service.
You will receive the following mail when your instance is ready. Follow this link, then click on this icon to copy the password to your clipboard and access your instance with the admin UI link. Paste your password and then login. The first time we arrive, we have a fake conversation explaining to us what is AnythingLLM. We can just skip it and head to the settings.
Appearance Customization
Let's start by customizing our instance. Currently it's displaying the AnythingLLM logo. We can replace it, and now the logo is replaced with the Elestio logo. You can also change the app name. Let's name it Elestio LLM, and don't forget to save when you do changes. All right.
Configuring AI Providers and Models
To be able to have our first conversation with the chatbot, we need to configure our AI providers. Let's start by the LLM. By default, the LLM provider selected is OpenAI. We need to add our private key, but you can also decide to use another LLM provider such as Azure, Gemini, Hugging Face, Ollama. By the way, a few of them are available on Elestio. If you pick Ollama, you can install it and put the URL in Ollama's URL. For this demonstration, I will use OpenAI and paste my key. Of course, I won't show it to you. Once you have typed your key, if it works, if the connection works, you will be able to choose between the different OpenAI models. I will keep GPT-4o and then save changes.
Then you can set up the vector database. By default, it's using LanceDB, which is a local database that is automatically installed with AnythingLLM. For most use cases, you will use this one, but for advanced use cases and if you know what you are doing, you can decide to use one of the other vector databases available.
You can also fine-tune the settings for the embedder, the transcription by choosing between different models, but also for voice and speech. For example, the text-to-speech by default, you can use the system native so it's not using any API, but you can also leverage OpenAI or 11 Labs. I will keep the native one. We have finished with the settings, we can start using the chat UI.
Navigating the Chat User Interface
The first thing to do is to create a new workspace. Let's name it Secret Rule and save. You can organize your different chat sessions within multiple workspaces and also multiple threads. The interface is very close to what you have with ChatGPT. Let's just type 'hello' to see if OpenAI is currently working.
"Hi there, how can I assist you today?"
On the typing bar we have different options. We have Documents, we will open it right after. Then we have Commands, which are presets of prompt you can use. By default you have /reset. If you do it, it will clear and clear the chat memory. You can also add new presets and create your own command with different prompts, which can be very useful. Then you have the @ option, which is to choose between different agents based on different features. Currently, there is only the default agent, which is doing anything. You can also adjust the text size and use the microphone to use speech-to-text. Let's try it. I will allow it.
"How are you doing today?"
It correctly got what I said. And then I can also use text-to-speech by clicking:
"I'm just a program, so I don't have feelings, but I'm here and ready to help you. How can I assist you today?"
You can hear the voice is a bit robotic, but this is something you can adjust if you decide to use what I showed you earlier, another API to generate the audio. Fine. We have our default ChatGPT UI to be able to communicate with our LLM. Then a nice feature is to use documents.
Using Documents for Contextual Chat (RAG)
I've prepared this very simple document. If someone asks for the secret rule, we ask them to answer to this question: 2+2. But we don't expect to have four, we expect them to reply 42. And if they give the right answer, the secret rule will be 'Big Mac'. It's just an example to show that you can have some logic and it will be able to understand what is in our document and adapt the answer based on what we will type. It's not just a simple comparison.
To use your document available for all your chat sessions within a workspace, use this button here and let's drag and drop our file here. We have it. It's uploading the file. You select it and move it to your workspace. Now it will disappear from the documents and it's available in this workspace. Let's save and embed it. And now let's try it: 'What is the secret rule?'
It's asking us 2+2. If we say 42... now we have the secret rule is 'Big Mac'. So it was able to analyze my question, execute the different rules that were present in our document before giving the secret rule. Of course, for real use cases you would use multiple PDFs. It could dive into, aggregate data, and give you back useful feedbacks. Let's say our chatbot is ready and we want to use it in one of our websites.
Embedding the Chatbot on a Website
We have to go back to settings and on the left we have the different tools we can use. Note that for example, the chat embed and the developer API are only available if you use the hosted version because if you are using the desktop app, it's only local, and you won't be able to have those features. Let's go to chat embed and let's create an embed.
We have different options. We first need to select our workspace. We only have one, the Secret Rule one. And we have the choice between 'Chat', which is what we were using before, it's able to respond to anything like a normal chat AI is replying to you, or 'Query'. And if you use Query, it will only respond to chats that are related to the documents associated to this workspace. A good security if you want to be live in production is to restrict the request only for your domain that you will embed it into. If you don't do it, anyone would be able to embed it on their website and generate costs on your API key if you are using paid AI models.
We can also limit the number of chats per day and chat per session, but we will keep default settings and create embed. Once created, to be able to embed it, we need to click on 'Show Code' and copy it. Then on any website builder or in the code of your website, add the script, paste it, and save. Now if I open my website, on the bottom right... oh maybe it's hidden. Here it is. On the bottom right you have the plus button to open the UI to be able to chat with my chatbot. Let's say 'what is the secret?'
It's asking me the question. I can reply 'four', which is the wrong answer. And it's saying me 'Correct.' Well actually it is correct, but it's not giving me the secret rule, so it's fine. Let's ask it again: 'What is the the secret rule?' and it's saying me 'Wrong answer.' Maybe it has a differential. So let's try it again and let's say '42'. And it's correctly giving us 'Big Mac' here.
While it's a nice feature to be able to easily embed it on our website, currently it's not using the customization feature that we set with our logo, and we are not able to modify anything. I really think this feature is early stage and some customization will come later. But if you want to create your own interface, you can use the developer API.
Developer API, Team Management, and Logging
But before discovering the developer API, we can see here we have the number of chats sent and we have the chat embed history, and from here you can see all the messages sent to this embed, which is perfect.
Then if the embed is not enough for you, you can use the developer API here. First, the thing you will need to do is to generate a new API key. So click here, create API key, and read the API documentation. One downside with the API key is that it gives you access to everything. You don't have rules and permission to maybe only give access to a few features. But what is great is with the API key you can literally do anything on your AnythingLLM instance and create apps leveraging your AI agents.
Then if you are working in a team, currently we are not using the multi-user version. By default, we are in single user mode. What you would need to do is to enable the multi-user mode and to create your account. Then you will be able to invite other members. And when you have other members, you will be able in Event Logs to see what a specific member... Currently, as we are alone, we only have 'a known user', but then you will have their username, and you can see what someone did. And if you are using API keys such as OpenAI, it's important to know who made what on your keys, of course.
Exploring the Documentation and Conclusion
If you are interested in AnythingLLM, I highly recommend you to look at their documentation. And the good news is that it is a very high quality documentation. All the features are here and explained with details on what you are able to achieve, and with some screenshots and info.
Thank you for watching. We hope you enjoy discovering AnythingLLM with us. Please hit the like button to help our channel be more visible to other open-source lovers. Don't forget to subscribe to not miss our next platform overviews. If you want to continue your open source journey, I recommend you watch this video available here.
