Introduction to AnythingLLM v2
Hey there. My name is Timothy Kimbat, founder of Mintplex Labs and creator of AnythingLLM, which is a tool I'm going to show you today that we recently just added a whole bunch of stuff into. So this is technically a revision of the other video that is also on this channel. This new version of AnythingLLM is far more powerful, way more beautiful, and has a ton more functionality that you can use to chat with anything.
Over time, we're going to be improving this tool a ton because we just believe that you should have access to an open-source ChatGPT that just kind of does it all and also supports an enterprise environment or a private chatbot environment where you can pick and choose what tools you want to use as opposed to hosting or using some other cloud-based service.
Now as I said, this is an open-source tool. You can find it on GitHub or you can go to useanything.com, and you can either trial a hosted version that will run for you, which gets automatically updated, and you can also find the GitHub link which will allow you to run this on Docker locally. Maybe you want to self-host it on your own servers that you run for on Google or Microsoft or AWS. It'll do all of that and more and you get the full functionality.
The Onboarding Flow: LLM & Vector Database Setup
Now I have already gone and started an instance on our Cloud hosting that we offer for all customers. You get a free three-day trial right now just so you can kick the tires, and we're going to start from the very beginning just so you know what I'm looking at. So you can see when I go to my special subdomain of useanything.com, this is where my instance lies. Now I haven't touched it yet. This is the first time I'm logging in and there's now an onboarding flow. This onboarding flow will leave no questions or no stone unturned, and you can always look at our docs for more information, but this will get your instance set up perfectly. So let's walk through each step.
The very first step is choosing what LLM do you want to use. We support three right now: OpenAI, Azure, and Anthropic's Claude v2, which has the 100k context token length, which is really awesome if you are trying to do some really big prompts. That being said, you have to get access to Anthropic to use it, so if you haven't applied, I suggest applying on their website. And obviously for OpenAI, we offer support for 3.5 Turbo and GPT-4. And actually in the future, we're going to have the ability for you to use your own model. So if you have a fine-tune, you can use it here as well. Now I've already gone and copied my OpenAI API key so I can chat with 3.5 Turbo, so I'm just going to paste it in there and click continue.
Configuring Your Vector Database
The next step is choosing your vector database. Now if you don't know what this is, just leave it as the default. LanceDB is a vector database that actually runs on the same instance as AnythingLLM, and it is a fully-fledged vector database so that basically your chatbot can be smart. However, if you are a bit more technical or understand how vector databases work, you may want to use a Pinecone, Chroma, Weaviate, or Qdrant vector database.
Choosing Your Database and Customizing Appearance
All of these have their own nuances and benefits and pros and cons. So we allow you to pick whatever one you want. We're going to add more over time. Milvus has been suggested, PGvector, or some others. We'll get to them, but right now these are the ones that we host. I'm going to stick with LanceDB, but if I needed to go to Pinecone, I would just enter in my key, the environment, and the index name and I would just click next. But we're going to use Lance because it's a zero-setup instance.
Now the next thing that's pretty cool is appearance. Now we try as much as we can to have AnythingLLM be as white-label as possible so that you can put your brand's identity into the product, even if it's just an internal tool. So one of the ways we do that is allowing you to customize the logo. You can upload one here; any dimension or size, we'll make it work, but you just upload it there and it'll appear all throughout the app.
Setting Up User Modes: Single vs. Multi-User
Now one place that AnythingLLM really separates itself from every other chatbot is we support multi-user instances. So if this is just for you, you're using this just to chat with GPT-4 because you don't have access to ChatGPT Pro, or maybe you're using this to talk with some special documents that you don't want to leave your machine, if that's the case you can use 'just me'. Or if you're using this in a more collaborative environment, you can do a 'team'.
If you choose team, we'll start assigning accounts and passwords and you would be the admin. You can make anybody else an admin and you can administrate over more users. Or if you just want to keep it simple, we'll actually go and use the 'just you' instance and you can just set a simple password. So when you need to log in, enter in the password and you're good to go. We're going to go with the multi-user approach so that I can show you everything that is to be offered here. So let's go to Team and I'll just do admin and password. Let's do, I think, super admin. Also, before we proceed, if you start in single-user mode, you can always go to multi-user mode, but you can't go from multi-user to single user. If not, just use your single account. But anyway, let's continue.
Creating and Using Your First Workspace
Now the first thing that we need to do is make a workspace. A workspace is basically a thread. It is a collection of documents that you are referring to during conversation where you're chatting with your LLM. So for me, I'm going to make a workspace called "AnythingLLM", where this workspace will know about AnythingLLM and we can talk to it, which is very meta.
So let's click finish. Now, because I am using LanceDB, any embedded data that I actually use here will never, ever leave this instance. But if I was using Chroma or Pinecone, the data would be stored there instead. So first thing is let's just click on our instance. Now you'll see that we're presented with an empty chat, and in this chat, we haven't embedded any information or given the LLM any special knowledge. So if we were to just send a chat right now, all we would get is a simple response as if we were talking to ChatGPT directly. There's no special data here, so there's nothing really to talk about. And so you'll see we get a default response which is just, 'Hello, how can I assist you today?' It's a very normal response from ChatGPT.
Now, let's actually go and specialize some data. The easiest way to do that is we can actually upload documents right now. And the best part is with AnythingLLM, if we upload one document here, but maybe we want to make a different workspace that also uses the same document, you only pay once. With AnythingLLM we allow you to smartly reuse data in other places without messing with the original data source. So let's just upload a single text doc I have called anythingllm.txt. You'll see the file was uploaded successfully. Now we can just click it and move it into the workspace. And you'll see we even give you a cost estimation because it does cost money to embed stuff. So you'll see this will cost less than a penny. So let's embed it and you'll see that that was done relatively quickly.
Advanced Workspace Settings & Customization
If you want to remove a document from your workspace, you can just click this little arrow and it'll move it back over. Documents that are already selected in your workspace will not appear here. Additionally, the settings tab allows you to really kind of mess with the bits and pieces of a specific workspace. So you can change its name, maybe change the temperature. This is kind of how random your LLM will respond. .7 is pretty normal. If you go low, it gets more concise and if you go high, it'll just go crazy. So play with this as you like. .7 is normal.
Chat history. Now we keep a running log of chat history so that your chat knows what you've been talking about. You can shrink or grow this number but try not to go too large because then you can wind up overflowing the context, which is basically the limit of how much text a single model can consume. And then of course you can change the prompt. Let's say that maybe this workspace is more for HR specialists, so you want to give more HR relative responses here. You can do that here. Maybe finance is more important, or maybe you're a programmer, you can have it respond with code here.
Chatting with Documents and Citing Sources
Now let's send a message asking our now smart LLM what AnythingLLM is. Now keep in mind the default ChatGPT or GPT-3 model, or even four, has no idea what AnythingLLM is, so we'll get to see what information is used here. So let's say, 'what is anything llm?' And you'll see that we actually get a really concise answer. What I gave it was a single text document that just had a short blurb of text about what AnythingLLM is. And it says this is what AnythingLLM is, but we also show you the citations and we give you the exact file that was used. And here you can see this is the exact text chunk that was used in formulation of the answer.
Managing Documents and Workspaces
This is important if you are getting odd answers and you want to know where it came from. We'll actually show you where in your vector database this came from if you want to go remove that data.
Now let's start another workspace that we can just call "duplicate" and we'll go and focus on that workspace and use this gear icon to open up the workspace management tool again. You'll see that anythingllm.txt is available. Now, because nothing's embedded in this workspace, we can reuse this document and get the same knowledge that the AnythingLLM workspace has. I just wanted to show you that that was possible.
And of course, if you are done using a piece of data or you have no more interest in using a workspace, you can always delete a workspace, which will then delete the vectors out of your vector database as well as just delete the workspace from AnythingLLM in general. Now, this does not delete the source document. If we go back to AnythingLLM, the document is still available. We didn't delete the document, so no data is lost in that process aside from what was specifically stored in your vector database.
System Administration and User Controls
And last but not least, let's look at some of the controls that you get as a user of AnythingLLM. All of this is free, there is no paid or premium AnythingLLM if you're using open source or you're using our hosted version, you get the same experience.
So the first thing is system preferences. If you're in multi-user mode, you'll see this. Allowing users to delete workspaces may not be something that you want them to do, so we can prevent that. Also, limiting messages. You'll notice when we signed up I used my OpenAI key. Well, maybe I want to prevent some of my customers from going crazy and running up my bill. You can limit them here. Also inviting people to your instance. You can create an invite link here and you can just email it to them, text it to them. This app works on mobile perfectly fine, so you could text it to them and they could sign up right there.
Users: this is where we can see users. You can see you have a username, a password, and modify their role. And as long as they are not an admin user, you can also suspend them to where they can no longer log in if they've been misbehaving, or you can delete them entirely if you no longer want them part of the system.
Workspaces is kind of like the hypervisor view of everything. You can see what the name is, a quick link to it, how many people are, or can interact with that workspace, deleting workspaces, and also adding users, which we don't have any extra ones so nothing shows. We can also see all of the chats sent through the system. Now the reason we do this is because as an admin, you should have this permission in case you are worried about how your instance is being used. You are paying for ChatGPT or using the API so you should know what data you're paying for.
Appearance, API Access, and Data Management
Appearance. Uh, here is where you can actually set the custom logo and also you may have noticed when we first used our instance, you'll notice that there's like this kind of tutorial setting. This is pre-written by us and it's the default message. You may want to have default messaging that is more specific to your use case of AnythingLLM. On appearance settings, you can actually customize these messages with system messages and, you know, fake user messages so that maybe you can walk through the new user through a tutorial of how they should best use your installation of AnythingLLM.
And of course, what would an open source project be without an API interface? Now you can actually see all of the API endpoints that are available by going to api-docs or by clicking this link here on your instance and you will see all of the endpoints that are made available to you. And in order to use them, you must have an API key. You can generate one by clicking this button. And here, of course, if you want to change the LLM you're using, let's say you want to migrate from OpenAI to Azure to Claude, you can do it here instantaneously. The same goes for vector database. Keep in mind if you change this, we won't automatically re-embed your stuff in your new database, so change this with caution.
Exporting and importing information. Let's say you just want to take a snapshot of your entire database. You can snapshot it here and preserve it in case for whatever reason you might want to roll back settings, users, chats, whatever. And of course, security. We're in multi-user mode, so we can't really do anything else here, but if we were in password mode or no protection mode at all, we could reset the password also turn it into multi-user mode.
Conclusion & Final Thoughts
And that is AnythingLLM. This new version I'm really excited about. As you may have noticed, it looks extremely clean and I mean, we have just really, really pulled no punches in redesigning this entire app. Again, it is open source or you can get a free Cloud hosted version through us for three days, or you can pay for us to automatically maintain and update your instance. And of course, we have a Discord. We're happy to have you join, ask questions. We want to hear your feedback. But that is the new latest and greatest AnythingLLM, and I hope to see you using it and I hope to hear from you soon. Thank you.
