Introducing the Embeddable AI Chatbot
Everyone, Timothy, co-founder of Mintplex Labs and creator of AnythingLLM. And today I want to showcase a really awesome feature we just added into AnythingLLM that's available for everybody in the open-source repo on GitHub, where you can now publish an AI chatbot on any website or page that supports HTML that you own. And this can be an internal-facing website, it can also be public-facing with all of the controls powered by whatever LLM you want, all the data stored privately on your instance, and it just works.
Setup and Demonstration Goals
Now, today we're actually going to be using an AnythingLLM Cloud instance, but if you are on GitHub and you know how to use Docker, you can just run AnythingLLM in a Docker container. We publish an image, it's as easy as two steps to get it running. Today I'm going to use a cloud instance though, but it all works exactly the same.
Now, you may be familiar with chatbots. I'm on Product Hunt right now and you can see these little icons in the bottom. Now, this is Intercom, which is a helpful chatbot client where people can respond to requests. We want to actually use an AI LLM trained on information on our website and just make it happen with no code, just copy-paste some HTML snippet and just have it work.
Preparing the Website and AnythingLLM Instance
So that's what we're going to do. And we're going to actually do it on our website, useanything.com. Now I'm going to use the staging site because I don't want to publish this to the whole world, but if I wanted to, I would just click publish. Now we use Framer, but if you use WordPress or whatever, it doesn't matter. It all works the same, and I'll show you how simple it is.
So first things first, you need AnythingLLM running somewhere on the internet where you can access it. Hopefully you've put up a password protection and all of that. Now the workspace and the chatbot we're going to expose is public-facing. The documents in this workspace, however, the things that it's trained on, is not given back to the public user, but you can modify them and manage them within AnythingLLM as if you normally would if you've used our tool before.
Creating and Testing a New Workspace
So the first thing we need to do is create a workspace. A workspace is essentially a collection of documents that the LLM will use to answer user questions. The more you have, the better, but there is kind of a science, but in general, if you have documents, it'll work.
I want to showcase that I am using OpenAI. I already have my API key in, and I'm going to use 3.5. We could use 3.5 Turbo, GPT-4, even the 32k context, even fine-tunes if you have one available you want to use. We also have a whole bunch of other AI providers if you want to pick one of those, including Ollama, LM Studio, and LocalAI if you're trying to run your own model locally or on a server you maintain.
Training the Chatbot with Website Data
Let's create a workspace that we're going to publish to the world. And I'm just going to name it AnythingLLM. All right, so we're already here, we're started. And if I were to ask this chatbot just in AnythingLLM, 'What is AnythingLLM?' it'll either give me a wrong response or tell me that it doesn't know because our project is not yet big enough for GPT to have scraped it, which is sad. So I'm going to ask it on an empty workspace, and it says that AnythingLLM is an online platform for basically people pursuing a Masters of Law. That is not what we are.
So let's give it some information, but actually just use our website to fill in those blanks. The way we do that is we're just going to click on the settings icon, move me out of the way, and I already have a text document embedded in here that is literally just like, 'Hey, this is AnythingLLM.' I said we're going to grab information from the website, so let's do that. All right, so our website was pulled in. Now all we need to do is embed it. And you're probably wondering, how am I embedding information? AnythingLLM actually ships with a built-in embedding model that runs on instance, totally isolated and privately. That being said, if you want to use OpenAI's Text-Ada embedding or the newest generation, which I believe actually has an even higher dimension output, you can. This is just what we give you for free.
Verifying the Trained Chatbot's Accuracy
Now we've embedded this information, so let's go back to the main prompt and ask that same question: 'What is AnythingLLM?' So here we are asking the question again, and it says that AnythingLLM is the ultimate business intelligence tool, blah, blah, blah. And it got that information straight from useanything.com, which is right because it actually is in the tagline, and that is important information. So it looks like our chatbot is pretty much ready to go.
That being said, if we wanted to add more pages, we could add the AnythingLLM Cloud page, the docs, we could put the entire repo of the GitHub in here, or we could put the download page. Again, we're only doing this for the homepage for this example, but go crazy.
Configuring the Chatbot Embed Settings
So now that we have a workspace that's ready to go and we can publish to the world and basically just be hands-off for customer support, or at least that's the dream, we can go to settings, go to embed, embed chat, and we're going to create an embed. Now, you're presented with a couple options here, and that's only because we give you ultimate control as usual. And we're here, we can select a workspace. We only have one, but if we had more, they'd appear here. We want to use the AnythingLLM workspace.
Now, there's two chat methods. One is for chatting, which is basically like using ChatGPT: general knowledge plus your specific data, or query, which basically means the LLM won't respond unless there is something to respond with. For your safety, we put it on query, mostly because we don't want people asking random, unrelated site questions that have nothing to do with our product running up our GPT bill. And of course, you can restrict the domain as well to make sure that people don't copy your embed and put it on their website or anything like that. And it all gets handled on the AnythingLLM instant side of things.
Advanced Embed Configuration and Code Generation
Additionally, you can set the max chats per day or per session, enable dynamic model use, which allows you to specify the model that you would like to use. Maybe on one page, you would like to use GPT-4 versus GPT 3.5 or a fine-tune; here, you can make that happen. You can also modify the temperature per page. Sometimes we find that that parameter helps a lot. And then of course, enabling the prompt override through a simple string in the embed, in the actual code snippet, you can override the system prompt. Maybe you want to grab the actual description from the header HTML and inject that as a prompt. You can do that. This isn't visible to the client, it's the system prompt, but here you can override it. Anything here that you override will overwrite the workspace settings that you've already set.
Now I already know the URL of our staging site, so let's limit all requests only coming from a certain domain. And you can see that's the domain. You can have multiple here; having none here means you can have it anywhere. We really don't recommend it, but if you want it, you can do it. Now we'll just click 'Create embed' and our embed is created. You can, of course, go back and modify any of these parameters at any time if you'd like. Now for the next part of this video, we're going to do the really easy part, which is showing this code snippet, which really is just this part of HTML. We're going to click on this, which will copy it to our clipboard.
Embedding the Chatbot into a Website
And now we're going to go to Framer and embed it into our website. Now just to showcase, this is the same exact website but on our staging site, as you can probably see from the URL. Now from here, all we need to do is to go into Framer which, by the way, this is unique to however you're using it. It could be Webflow or GoDaddy or WordPress, whatever. What I'm about to show you is definitely possible in every single website builder. So we're going to go to settings because that's where we can access this. I want this embed to be present on all pages, and it's probably best that we put it at the end of the body tag. So we'll paste that in right there, scroll up, click save. Oh sorry, we have to publish it first. I will click update right here and then it'll be live.
Live Demonstration of the Embedded Chatbot
Okay, and now our embed is present. Now it's very plain-looking, and if we open it, you'll see that we have AnythingLLM branding, we have the kind of footer here. But let's ask the crucial question of 'What is AnythingLLM?' And here we get a description. Now notice we don't get the context that was used, because we want to keep your real source document right. And of course, the customer can go and reset their chat or anything like that, but I really do want to dive in to just a little bit of the functionality here that's possible.
Customizing the Chatbot Appearance and Conclusion
Now I'm on GitHub right now and I'm in the embed folder. There's a readme file in there, and this is all of the customization options that you can use that you can define with the embed. You can change the chat icon to a number of basically pre-supported icons. You can change the button color, the background color, the assistant response bubble, your response bubble, your brand image URL, the greeting, hiding the footer sponsor, or changing the link and the text of it entirely. And also showing should it open when the page first loads, what your support email is, and this is just the options that we have today. So while yes, out of the box, our chat embed just shows AnythingLLM, we give you the ability to totally clean slate and refresh this chat bubble to where it really works with the brand of your site and your brand's identity. We're going to be doing a lot more improvements in the UI and the UX of this so your customers have a really great experience. But that is the AnythingLLM built-in embed tool. You can publish this on any website you own and it comes with AnythingLLM. Thank you.
