Introduction to the New AnythingLLM
A long time back we explored AnythingLLM, an all-in-one AI app that allows you to chat with your documents, create and deploy AI agents for task automation, and even configure it for multiple uses, all while being fully local and open source. Now recently, AnythingLLM has gotten a big new update where they have partnered with Nvidia, and this is to enhance AI agent capabilities on RTX-powered PCs. So this will enable users to run customizable agentic AI locally.
The new Community Hub is also another new feature that lets users share prompts, slash commands, and even have the ability to create AI agents where these AI agent skills are used for tasks like managing email, calendars, as well as web searches, while ensuring that your data is private. Now with AnythingLLM, you can even integrate various large language models for tasks like content generation, summarization, and even automating complex workflows, which is making it an ideal platform for faster, smarter AI processing all in one area, powered by the RTX GPUs.
The Nvidia Partnership and Getting the App
I've actually made multiple videos on AnythingLM where you can use it to create AI agents as well as chatting with your documents locally. It's like a private GPT alternative that lets you chat with any document completely for free, locally, and private. But now you have so many new features where you have the ability to fine-tune models, create AI agents, and even automate certain things with this new update. And now what's also great is that you have Nvidia powering all of your automations locally.
The reason why I say that this partnership is really important and it's a big update is because your AI agents are going to be powered by consumer-grade RTX GPUs. And what this will mean is that it will deliver unparalleled speed for inference as well as task execution. Your agents will be operating locally while ensuring data privacy, but more importantly, the functionality of your agents are going to be more powerful as it's powered by these super GPUs. By the way, you can also access this on the web where you can access their Cloud portal which is going to be paid, but in my opinion, I highly recommend that you download it for desktop. I'll leave a link to this in the description below. Download it for your operating system. This is completely for free to go ahead and install.
Initial Setup and Configuration
Oh, and if you join our private Discord, you can access the cloud completely for free as well as getting access to lovable.dev, which is also another plan that you can access completely for free by joining the Patreon. So open up AnythingLM once you have installed it. And what you want to do first is click on 'Get Started' and select an LLM provider. You can choose OpenAI or many of the other choices over here. You have Ollama where you can run open source models locally, or you can even run Nvidia Nim. So go ahead and select the one that you would want to work with. Go ahead and select the model of your choice, then you just need to simply provide a workspace name. And once you have done that, you'll be then sent over to the main dashboard of AnythingLM. It's fairly easy to use.
UI Overview and New Agent Skills
This is your main chat interface where you can start chatting with your documents as well as your agents. You can simply upload your various file types. You also have the ability to run slash commands and you can even add different agents to provide the ability to have them execute certain types of tasks. For example, you have RAG search, web scraping, web browsing, as well as many of the other predefined agents that have been created.
Within the settings tab, which is the button that you can click here, you can configure things like your LLM provider, your vector database, you can also change your embedder, text splitter, change the configurations for your voice and speech as well as transcription. But what's interesting here is the agent skills. This is a new feature that has been added to AnythingLM. It's great because these are skills that you can create or enable from the base templates that are already set to provide your agent more functionality. For example, if you want to enable something like generating charts, you can go ahead and enable it and then you can click save. And then now your agent could use the skill of generating charts. In the same manner you can do that with web search where you can go ahead and have it use DuckDuckGo or many of the other providers for web search over here, and you can go ahead and click save. In the same manner, you can do that for SQL as well as generating and saving files.
Your browser and thanks to the RTX GPUs, this will make your AI agents even more powerful locally as they're going to be able to use the GPU's power to enhance the capabilities of the agents' efficiency in terms of generating, which is why this is the best ideal local solution.
Community Hub and Custom Skills
On top of this, you also have a Community Hub in which you can access readymade templates to AI agents that have been created by the community. Now one thing that I'll note is that you also have the ability to create your own custom skill, and you can actually refer to their documentation which showcases how you can do this. They showcase the prerequisites of developing your custom skill, and you can see their guidelines on how you can do this. So I definitely recommend that you take a look at this. But going back to the Community Hub, these are agent skills that you should definitely take a look at because they're easily readily made for you to easily integrate into your agent. So if there is something that resonates with you or what your goal is with an agent, you can easily import it. It's simply by just clicking on the import button. You can paste in the ID afterwards, you can then review the agent skill and if it is correct, you can go ahead and import it. And there you go.
Now there's also a website or their docs that showcase a couple of other agent skills like saving a file to the location. You have an Outlook draft Creator, a Gmail draft Creator and many other different agent skills that you can easily access off of this website. And this is something that I'll leave a link to in the description below. You also have a developer API and a browser extension that will enhance your capability of using this all-in-one platform.
Demo: Using the RAG Agent on a Document
But now that we have understood this platform, let's go back into our workspace and start looking at the new feature of tagging agents. Now what we're going to be doing first is testing out this RAG agent. This is where we're going to be attaching the agent with the RAG skill and essentially we're going to be feeding in this 137 page document. It's a research paper on alignment faking in large language models released from Anthropic today, which is super insane. This is something that you should definitely take a look at. But we're going to be feeding in this huge document and we're going to be having the AI use RAG to help better analyze and retrieve answers from it.
So let's go into our AnythingLLM application. Go ahead and click on the 'attach a file' and we're going to be attaching this research paper. So this will take a couple seconds and now that this file has been embedded, what you can do is click on this app function and then now you can have RAG search. You also have the ability to upload multiple documents. So in this case, you can see that we already have our custom document which is the alignment. And what we can do is you can even have it connected to various data connectors where you can provide information from GitHub repos, YouTube transcripts, and so much more. So we're going to tag the agent and we're going to have it summarize what this research paper is talking about.
Right away the agent has been invoked and it's going to look through the document. And there we go. It talks about how it discusses the phenomenon called alignment faking and it was able to report this answer back to me within 6 seconds. And this is where it will even provide the citations and it could even go back into providing which section it found certain areas.
Advanced RAG Agent Interaction
So now let's go ahead and use the RAG agent further. We're going to go ahead and tag the agent and we're going to have it focus on a specific area. So what I'm going to have it do is use the RAG capabilities to give me a more precise answer about the results. So this is where I'm going to tag the agent and then I'm going to ask it, 'Can you please give me a breakdown on the results, talking about the core metrics?' We're going to go ahead and send this in. And we can see that the agent was invoked and the agent found four additional pieces. And right away it's going to provide me the three different metrics.
I'm just playing around with it. I'm going to go ahead and make it write a thesis supporting the results, and within a couple seconds we should have it base an answer off of what we had found from this result. So we can see it created a really small little thesis on it. It focused on evaluating alignment faking in the AI systems metrics implication.
Demo: Web Search, Charting, and Code Generation
Showcase the capabilities of the web searching skill that could be attributed to an agent. Let's have it use it and where we're going to be asking who is the current prime minister of UK. So with that thought, it used DuckDuckGo and it was able to find that Keir Starmer is the current UK Prime Minister.
Now you also have the ability to create charts. So in this case, this was a random population chart that I created by just simply tagging the agent. So you get a good idea that these different AI agents powered by your RTX GPU can help you in so many ways. You can also have it so that this can actually create the front end of your application. This is where I had simply requested to create an application that focuses on time management and you can see that it goes ahead, generates the code for me, and there you go. You have a super simple time management application that was developed with my local solution.
Conclusion and Final Thoughts
If you like this video and would love to support the channel, you can consider donating to my channel through the Super Thanks option below, or you can consider joining our private Discord where you can access multiple subscriptions to different AI tools for free on a monthly basis, plus daily AI news and exclusive content, plus a lot more.
In essence, this is a great all-in-one solution where you have the ability to chat with your AI locally, fully private, and have the ability to use various large language models as well as various AI agents that you can create to help you in various ways. This is something that has been accelerated by the RTX GPUs and it is definitely something that will greatly elevate your workflow. So I highly recommend that you read through this blog post as well as take a look at AnythingLM if you haven't taken a look at this before, because this is a great solution that can help you in various ways.
But with that thought, guys, I hope you enjoyed today's video and you got some sort of value out of it. I'll leave all these links in the description below. Follow me on the Patreon, follow me on Twitter, and make sure you actually subscribe, turn on notification bell, like this video, and please take a look at our previous videos cuz there's a lot of content that you will definitely benefit from. But with that thought, guys, have an amazing day, spread positivity, and I'll see you guys fairly shortly. Peace out.
