The Problem with Cloud-Based AI Note Takers
Hey everyone, Timothy Carbat, founder of AnythingLLM, and today I want to show you a really cool feature that we've been working on for a while that is now available in our desktop app with the latest 1.10 release.
Like most people, I have a lot of meetings. And there are a lot of AI tools that can not join your meeting, but then also be able to record and transcribe and summarize your meetings. And in general, this is very useful. However, sending every meeting I have with both employees or partners or just sales, I don't really want to send that all to the cloud. And on AnythingLLM, our whole thesis is whatever cloud models can claim they can do, we can bring on device and give you a very good experience. Not to mention, because it runs on your computer, we can give it to you for free.
And there are tools like Granola that do start out free. They can do your notes. They can look at history. They can, you know, have templates. It does multiple language support. And then there's also tools like Otter that are free to start with that give you AI insights. But then also, you know, they only work with maybe a limited set of tools like Zoom, MS Teams, or Google Meet. What I'm going to show you today makes all of those tools completely irrelevant. And I mean that on any hardware.
AnythingLLM Meeting Assistant: Feature Overview
So, in the latest version of AnythingLLM, you can simply go to the settings and now you have the meeting assistant as an option on your sidebar. Now, from the meeting assistant, you can start a recording. You can also just upload arbitrary audio or video files that you already have. This is a team meeting that we had earlier this week. And as you can see, not only did we do speaker identification, of which I'm able to customize and label, I can also edit this transcript if it's not completely accurate, but I also got an entire summary that was also done fully on device. And you can see this summary is comprehensive. And not only is it comprehensive, that is with a basic template that we ship. If you have more customized needs, like maybe you want to do a performance review or a sales call that hits specific points that you worry about, then you can just make a template and have the LLM give you feedback on that. You can apply a template to any meeting after it's been done or while it's going on.
Leveraging Agents, Chat, and Search
As you know, AnythingLLM is very focused on agents, which is basically a fancy way of saying getting an LLM to be actually useful. AnythingLLM ships with a bunch of basic and pre-built agent skills like searching the web and connecting to databases, but you can code your own skills. You can use our node builder to create your own skills. And of course, you can use MCPs whenever you send a meeting or upload a file to the AnythingLLM meeting assistant. We actually will suggest agentic action items based on the skills available so that you can just do a one-click run and run that skill.
And of course, what would AnythingLLM be without the ability to ask questions? Clicking on this brings your entire transcript into a chat window where you can just send more nuanced chats or get specific details from a meeting. And then, of course, because AnythingLLM does do anything, our built-in search pipeline automatically indexes every meeting and transcript that you have entirely on your computer. And you can search for it here. You can search for keywords. You can search for semantic things like when did I talk about LM Studio and you can see that well I have one meeting but you can get the exact snippets in which I did bring that up.
Live Recording Demo with Google Meet
And depending on your hardware or your preferences you can turn off speaker identification but we also offer meeting started notifications so you don't have to go into AnythingLLM to start a recording and we also support live transcription if your hardware supports it.
But let's talk about the meeting started notifications. The AnythingLLM meeting assistant does not join your meetings. It is there in the background silently listening and transcribing and hopefully summarizing your meeting. But it also works with any software. For example, let's just start a Google Meet. Clicking this notification that we just got automatically starts this meeting. Now when I stop the meeting, this meeting should automatically conclude and then of course run the whole pipeline of summarization and agent items. So let's stop the meeting and see where that kicks off.
And as you can see, we're notified that the transcription is in progress and it'll let us know when it's done. In my specific setup, I'm actually using the Docker model runner, but this could be any other on-device LLM provider you want. It could even be our built-in LLM provider if you want. Of course, we support Olama, LM Studio, and everything else under the sun, however you like to run models. But it looks like that meeting just got finished. And as you can see, we get a pretty basic summary 'cause that was a pretty rudimentary meeting. It was super short and frankly there was really no good information.
But that was with Google Meets. What if I wanted to do something like Slack? Maybe I do all my stuff in huddles. So here I am in a test channel in Slack on my computer. AnythingLLM is also running. And if I join this huddle, we should hopefully get a notification just like we would any other meeting. And the same process applies. If I click this notification, we'll start recording instantly. But let's say that for whatever reason you do like to keep your meetings in some kind of recording software or whoever, who knows what it might be, you can always just upload that into AnythingLLM and it'll instantly be summarized and indexed and everything else that it already does.
For this example, I'm going to drag in an engineering meeting that we had actually earlier this week. And at the end, I did actually explicitly say, 'Oh, we're going to make an issue about that on GitHub.' And so, what I'm hoping to see is my MCP server that allows me to connect to GitHub and has all these tools to be able to queue up an issue request. So, all I need to do that is I have the MP3 of that audio already saved and I'll just simply drag and drop it into the window.
Processing, Summarizing, and Editing Uploaded Meetings
And again, this process will just kick off and start doing the transcription and the summary. I have speaker identification enabled. So, we hope to see some amount of unique speakers. Now, you don't have to stay on this screen after you've uploaded or you're waiting for a meeting. You can go and start sending chats and using AnythingLLM or just simply minimize it and go about your day. And you'll get a notification once everything is done.
Oh, so it looks like our recording just finished. So, I'm going to click on this and here's my summary. And you can see where we talked about basically how we were going to remove LangChain from our codebase. And so you can see that it was summarized using this model. And then checking out the transcript, you do see a dialogue between a number of speakers. I know. And also I can just click play from here and it'll begin playing the audio. But I know that this was me and this was Mac. And so now we've automatically kind of identified the speakers in this meeting without much fuss. And we can resummarize with these names. Ultimately speaker identification is purely for your enjoyment. We actually keep it off by default just so you can get faster summaries. And of course from this transcript, you can double-click to edit, make any spelling corrections or anything like that.
Automating Tasks with Agentic Action Items
And if you don't like the summary that you've got or the action items that got suggested, you can always just re-roll them and get a new one or change your model and then run it however you want to do this. On the action items tab, you'll be able to see every single agent skill that the model determined was relevant based on not only your transcript, but also your summary. And you can see that we have some basic built-in ones like emailing the summary to participants. Running this will simply open up your email client and you can just fill in everybody's addresses. But then we also have more specific skills like the GitHub MCP. And here you can see that it determined we did need to write an issue. And the name of that issue is deprecating LangChain with some markdown content that is relevant, targeting the Mintplex Labs repo and AnythingLLM because these are things I defined in that MCP. Clicking run will just execute this. So let's click run. And you can see that we get a successful result. And if I refresh the issue tab in GitHub, you now see that we have deprecating LangChain as an issue that was created by me because in the MCP I was using a personal access token. Now because this was a demo, I'm just going to close this issue.
Conclusion: The Power of On-Device Productivity
Hopefully this brief demo of the AnythingLLM meeting assistant has been really impactful and can show you the power of what you can do entirely on your computer. AnythingLLM is building all of these productivity tools for just your everyday job into one neat interface that runs either entirely on local or mostly on local.
That's not to say that cloud models don't have their place when you need really large context or a lot of compute for really complicated tasks. It makes sense to do cloud. But for your everyday tasks like summarizing emails or your meetings, just do it on device.
