What is AnythingLLM?
Hey there. My name is Timothy, founder of Mintplex Labs, and today I wanted to make a pretty short video on how to host your own private instance of AnythingLLM. Now, AnythingLLM is a private open-source chat with your documents software that contains everything that you're going to need to either use this yourself or with a team.
Now that being said, there's a lot of features in here, and I'm not going to go over each one, but this is pretty simple to run locally. But sometimes you're going to want to host stuff on the internet to where it's publicly accessible and to where people can also collaborate and you can upload docs and everyone can contribute. It's all just a beautiful, beautiful thing that AnythingLLM can be when you add it into your business or your consultancy. You add it into your whole stack, it pretty much gives you superpowers.
Now for those of you who are familiar with this project, you've probably already run it locally, but now you're looking for a way to put it on the web. Well, I have two things to share. First is that we actually have a website for this now. I'll link it in the description. It is useanything.com. That is mostly because some person bought anythingllm and then domain squatted it and is now trying to charge me like $30,000 for it. Um, no. If they watch this video. Anyway, aside from that, we actually have a way to run this for you.
Deployment Options: Hosted vs. Self-Hosted on Render.com
These are isolated instances, so if you upload anything, your storage is not shared with everybody else's. It's not like one giant S3 bucket or Google Drive. It's all isolated private instances, and you get your own subdomain, and you can actually get a free three-day trial.
Now, if you wanted to do this for yourself in a way that is definitely more expensive, but just something that I guess you can fully manage—this is managed and we update it and always push the latest changes for you—if you want to do it a different way, we always make that available because we are an open-source project. You can install this yourself on a server if you own it, or the easiest option is actually to just use a service like Render.
If you don't know what Render is, in summary, it's basically AWS but with a way better interface. And in just a few clicks, I'm going to show you how to launch an AnythingLLM instance that you can then access if you want to do it this way, because why not? At the end of the day, our objective at Mintplex Labs is to make great open-source tools that allow you or your business to use these new cutting-edge technologies for your business. However you do that is totally up to you, and I want to offer all avenues available.
Getting Started with Render Deployment
So to get started, you'll need a Render account and you'll need to attach some billing to it, because, you know, nothing in life is free. Once you have your account set up on Render, all you would need to do is go to New and go to Web Service. And it's automatically going to ask you to connect a repository or link a public one. What we're going to do is we're going to go link a public one.
So we should be able to just paste that in there and click Continue. Now, this is going to be the name of our service. They don't tell you this, but this is actually what the subdomain will look like. You can also add a custom root domain or subdomain using this method as well, but we're just going to call this 'sample-anything-llm-service'.
Now the next thing is you're going to want to choose the 'render-deployment' branch. This is a special branch that we allocated just for Render and it is always in sync with master. So whatever changes that we push, like for example, yesterday we made live the full developer API, that will be live in this branch as well and then it'll be up to you to redeploy it, push the changes, all of that stuff. But since you own this, this is your system.
Now the root directory we're going to leave empty, and also if you wanted to follow along, if you're a more, you know, visual learner, you can actually go to the render-deployment branch, and in Cloud Deployments, we have a Render folder. And then this README documentation will actually tell you exactly how to do it. So, root directory, as I said, is empty, and we can close this tab now.
Configuring Instance Type and Environment Variables
And for AnythingLLM, because there is frankly a lot going on with this service, you're going to need at least two gigabytes of RAM. This will keep your instance fast and performant without having to charge you a ton of money. So this is $25 a month plus storage, you're typically probably going to be looking at around thirty something dollars a month more realistically, but that's a low cost to pay for the amount of power that you're going to unlock. And obviously we're going to choose Docker because that's actually what we're running.
Now, Advanced is where we're going to need to actually kind of get in the weeds a little bit to make sure that we set this up right the first time. So we're going to need two environment variables. One called 'PORT' which is going to be equal to the value of 3001. This is just going to make the deployment process a lot faster for us on Render. And then we're going to need a 'STORAGE_DIRECTORY' component. And we're going to have that value be equal to /storage.
You're wondering what the hell is that, where is that coming from? I'm going to get to that in a little bit when we add a disk.
Adding Persistent Storage for Data Integrity
So, Render, much like other ephemeral services, do not offer persistent storage built in. So what that means to you is that if you have your AnythingLLM instance running and then for whatever reason you decide to restart it or you pull in the latest change, all of your settings will be gone. That's obviously not what you want because one of the benefits of AnythingLLM is the vector cache, so you don't have to keep paying to re-embed stuff in different workspaces. This solves that.
So what we're going to do is we can just call this 'storage' and then the mount path will be that /storage path that we set in ENV. The 'STORAGE_DIRECTORY' environment variable for these two values, if you wanted to change them, must be the same, but that's all. And then for size, I always recommend at least the smallest size you can. At least for this, one gigabyte is enough to get you started to see if you like using AnythingLLM like this or if you'd prefer using a local or our cloud-hosted version.
Finalizing Advanced Deployment Settings
For health check, health check is pretty... you don't have to use it, but the proper value is /api/ping. This will basically be something that Render can ping every now and then to determine is your instance still online, and I'll send you an email if for whatever reason it's offline.
For the build context directory, we're actually just going to put a period. So this will be relative to the root, so that means our build context is the root directory. You don't need to really worry about that. The build path will be this special Docker file, which is just render.dockerfile. It's just a slightly modified version of the original Dockerfile that allows this to run on Render without you having to debug it or anything like that. Same Docker command. Auto deploy, I recommend turning it off and then manually updating as new updates come about. The reason I say this is because if you are in the middle of maybe embedding a huge document and your instance restarts, you typically don't want that to happen because you'll wind up with a half-baked document or even worse, a corrupted vector database. AnythingLLM comes with a built-in vector database, you don't need to go and spin up another vector database, although you are totally welcome to do so and it'll work with it.
As far as LLMs go, OpenAI and Azure OpenAI are the two supported ones. We'd like to support other LLMs like Anthropic's Claude v2, but they have yet to email me back. I don't know if they just don't like me, but I've emailed them twice and they still won't give me access. So if you work for Anthropic or you know someone that does, tell them that I am very interested in integrating their LLM into this.
Launch and Verification
And then once we're done, that's actually the entire setup process. We are done. We're going to go to Create Web Service. And now what's going to happen is eventually we will have a running AnythingLLM instance that no matter how many times we restart it, all of our data will be persisted and it will be at this domain. Now, if you aren't familiar with Render but you're still excited to do this, if you go to settings and then you go to Custom Domains, this is how you add a custom domain. That being said, not going to get into how to do that because it's just frankly out of the scope of this video, but everything else should be set up for you.
And this process can take a couple minutes to boot up and get running, and so I'm going to restart the video once the instance is live just to show you that it's all there. Alright, and here we are only a couple minutes later and what we can see is that the service is now live. So it may take a couple seconds after the fact to boot, but if we go to the domain, here we are. We are treated with a beautiful fresh instance of AnythingLLM where we can create new workspaces. And just to be sure that this for example has the latest changes, there should be an API documentation endpoint at /api/docs. And there it is. So looks like we are all good to go. This would be your instance and you can do what you want with it.
Cost, Final Tips, and Conclusion
Just be careful to make sure that you don't exceed the storage capacity of the disk if you set it to something low at first. All in, this will cost you probably about 25 bucks a month for sure, plus 25 cents per gigabyte of storage regardless of if it's being used or not. Pretty simple way to set up AnythingLLM and hopefully you use this or consider our hosted option or even just run it locally and give us some great feedback, because that would be immensely helpful for us.
Hopefully, this video was helpful. Hope you enjoy your instance of AnythingLLM. Thank you.
