Introduction to LibreChat with Ollama
yesterday I showed you how you can install Liber chat locally with integration of open aai and you can visit that video on my channel and I will also drop the link in video's description. But today I'm going to show you how you can use a Liber chat with AMA locally with any model which is supported by AMA so that means you can use liary chat with Gemma, mistral, 5, and heaps of other models.
Olama is one of the easiest and fastest tool to run and install large language models locally. I have done heaps of videos on AMA so if you're interested just search with ol Lama on my channel and you should be good to go. But for this video let's focus on Liber chat and how can we integrate it with AMA and then we will also see how to install it locally.
Before I delve deeper, Liber chat is simply a free open-source AI chat platform which offers you a lot of goodies such as you can customize it, you can integrate it with a lot of other models which are API based or which are open source as I just mentioned. And then there are lot of other customization which you can do with it.
It is open source and free. Uh, it supports multi-provider, it is quite customizable and extensible. It has a modular design and extensive customization option. It can be tailored to meet your specific needs. You can create and share custom presets, integrate plugins and even contribute to project development because it is open source.
One of the good thing which I like about it is that it is um multilingual, so it supports multiple languages, few European languages, few of the Asian languages, so it is quite good. Also Liber chat prioritizes privacy and security by offering secure authentication, moderation tools, and the ability to s force the application.
Normally the use case what I have seen for liary chat is that people install it in their local area network in their private company Network and then they expose it to their own developers or their own people and then people can use the large language model through a chat GPT similar interface and then go from there.
So let's go to my local system and see how can we get it installed.
Environment Setup and Repository Cloning
I am using ob2 22.04, one GPU card of uh 22 GPU of vram and my memory is 32 GP as you can see the specs of my G GPU card here. So let me clear the screen.
One of the prerequisites which I will highly recommend is Anaconda. I already have installed and let me show you its version. If you don't have Anaconda, so I would highly suggest you install it. I have a separate video and you can search it on my channel to install anakonda. So anakonda is installed and I will use anakonda to create a virtual environment which will keep everything nice and clean and separate so that I don't impact anything else running on my system. And this is you can see I am creating that environment with the name of Liber chat. So it is saying I'm removing the existing one, installing a new one which is always a good idea. Then press y, and that is done.
Let's activate that Library chat environment and you can see in parenthesis that it is installed. Now second step we need to do is to get clone the Liber chat repo and I will drop the link in video's description. So simply do get clone and then you will clone it in your local system. That shouldn't take too long, that is done. Let's cd into Library chat, let's clear the screen and let me show you the content of it.
Configuring LibreChat Files
Now this is where you need to be bit um, you know, conscious of few files and let me do ls -la. Now if you look at it we have this .env.example file, you need to rename it to .env. Then we have this docker-compose.override.yml.example file. We just have to rename it to this. And then we have a third file which is called as librechat.example.yml. We just need to rename it to librechat.yml.
I already have renamed these files and I'm going to copy these files and I will also put these files in my blog and I'll drop the link here because you would need them. And I have spent like 20 hours battling with this and eventually I was able to make it work. So I'm going to put those files in my blog and I will drop the link in video's description. Just copy those files and then uh just paste it here, all three file, and then we'll be good to go.
So let me quickly uh copy the file from my local post, and I'm just going to bring this file here. Similarly, I'm going to bring the live chat.gaml here. And then similarly I'm going to bring the doer compose file here. Let me clear the screen and show you the files again. There you go. So now I have my .env file, this one, then I have my librechat.yml file, and I have my docker-compose.override.yml file.
Reviewing the .env Configuration File
Let me quickly show you what these files look like. So I'm going to do VI editor. Now you don't have to change anything here unless you are using OpenAI. So this is a top of it, Local Host, the port will be 3080 and just keep everything as is. It is telling us where to run it, what is the mongod DP, we don't have to worry about it. It is going to do everything in its own.
Now, if your organization uses reverse proxy, just put the reverse proxy URL here. And then if you're using Anthropic and all that stuff, we are not using it so I, we don't have to change anything in this file. So this is, I'm just showing you so you don't have to worry about .env file, but make sure that you rename it. Okay, let me exit from here, colon Q exclamation mark, and I am exited.
Modifying librechat.yml for Ollama
Now let me show you liary check yl file. This is the file which you need to change. Now let me go at the top and I'll show you what you need to do. Okay, so now in this one don't change anything until here. Keep coming down and then keep coming down... keep coming down... don't change anything. Now from this line where it says - name, you won't find it in the actual uh libr chat.gml file, so you need to add all of these files, and this is these all lines from - name AMA to model display label AMA at the end. This is what you need to paste and make sure that all the da of yl are correct. And this file gave me the biggest heartache. Make sure that base URL is like this.
That is also done. Let me exit.
Finalizing the Docker Compose Override File
And let me show you the docker compose one. again in this one make sure that you have pasted... um you don't have to paste it you just um uncomment these lines from AMA to this. And then if you go up, keep going up, make sure that you also have uncommented these lines. Once you have done it, I think rest is all good. Now get out of this file. Let me clear my screen.
Now I already have Ama installed so let me show you what models I have in ama, ama list. Okay, so it seems AMA is not running. So let me quickly check if my system CTL uh AMA system CDL status service is not running which is fine. Uh, we don't have, if it is running, you need to stop it. This is what I wanted to tell you. All you need to do is to Simply do this to, system CTL stop. service because Liber chat is going to install it and run it, not install, run it by itself. That is all you need to do.
Launching LibreChat with Docker
Now, in order to start Liber chat, this is the command Docker compos you need to run. Of course you would need to install Docker and Docker compose. If you don't know how to do it again, I have already have it on my channel, so make sure that you have Docker compose install. Press enter and it is going to download and start all of these containers on your system. And you can ignore this warning for now. Okay, so let's wait. And here you would need to wait a little bit longer until all the docker containers are in the running state. So maybe you can run this command. And now you see everything is running. This is all you need to do in terms of installation. Let me take you to my browser now and we will see how it looks like in the browser.
Using the LibreChat Web Interface
So when I access it in the Local Host colon 3080, then you will be um presented this screen. So it is running local host on Port 30 80... 3080. Now, let me give it my email address here but first time you would have to sign up and this is all local sign up. So click on sign up and then give your name here and then it is going to fill out rest of the stuff. I'm just going to give it my name here and then this is my, I'm going to give it password and continue and I have signed up. Let me login now and once you click here, there you go. So you have Liber chat running locally.
On the top, just click on open ai, scroll down and you should have Ama here and as we already have mistol there so it has selected the mistol. I only have Mistral here and then you can chat with it. Just ask anything, 'Hello there'. And the first time it takes a bit of a time and you see mrol has come up with the response. Let me scroll up and you can ask it anything here, like, 'explain happiness to me'. There you go, so this is mistol for you, all running locally, all running on your own system.
Additional Features and Conclusion
And you don't have to worry about any API or anything else. And of course, you can attach files here, so let me bring a file. So you can just click on attach files, and then from here you can filter it, and then you can simply click on manage files and you can upload it from here.
And also, so if you want to change the settings, simply click on this bottom left and then go to settings and then from here you can change the theme and lot of other things which you can explore on your own, very simple. This is what is good about liary chat is once it runs it has one purpose, it provides you uh this chat GPT like service which you can use and uh you can have multiple users who can access it and then you can manage the users too.
So that's it guys, I hope that you enjoyed it. Now you know how you can run and use this liary chat with open ai plus also with the local uh models by using AMA. As I said I will drop all of this configuration in my blog and I will drop the link in video's description. If you still are facing some issues, let me know, happy to help out. If you like the content please consider subscribing to the channel and if you're already subscribed then please share it among your network as it helps a lot. Thanks for watching.
