Made withOllamaClip

Build Your Own UNCENSORED AI Running Completely Offline

C

Cyb3rMaddy

Creator on the Ollama channel

Watch on YouTube ↗

370K viewsJan 21, 202611:53 runtime

Learn to build your own uncensored, private AI that runs completely offline on your local machine. This guide walks you through two methods: a simple setup using Ollama to run various open-source models, and a more advanced configuration with PrivateGPT to create a personalized…

Chapters

#ollama#clip#aillm

What is Private, Uncensored AI?

A private AI is a large language model (LLM) that runs entirely on your local machine. This setup provides several key benefits:

Offline Access: It works without an internet connection.
Uncensored: Models are free from the typical guardrails and restrictions found in commercial services.
Privacy: Your data and prompts are never sent to a third-party company.

Method 1: The Easy Way with Ollama

Ollama is a tool that dramatically simplifies the process of downloading and running open-source LLMs locally.

Installation: Download the application for macOS, Linux, or Windows from the official website.
Usage: Run a model from your terminal using a simple command:
```
ollama run <model_name>
```
Finding Models: You can find many models, including uncensored ones, on the Ollama library website by searching for terms like "uncensored".

Tip
Having a local AI via Ollama is perfect for situations where the internet is unavailable or for developers who need a reliable, offline coding assistant.

Method 2: The Advanced Way with PrivateGPT

While Ollama is great for general use, its models are trained on public data. To make an AI that understands your personal data, you need PrivateGPT.

Key Concept: Retrieval-Augmented Generation (RAG)

RAG is a technique that gives an LLM access to a private knowledge base. It allows the model to retrieve information from your specific documents and use it to generate relevant, context-aware answers.

Important
With RAG, you can create a secure AI assistant trained on your notes, documents, emails, and company data, all while keeping everything completely private in a local vector database.

PrivateGPT Setup Guide

Setting up PrivateGPT is more involved and requires a specific development environment.

Prerequisites:

WSL (Windows Subsystem for Linux): Provides a Linux environment on Windows.
Python 3.11: PrivateGPT requires this specific version.
pyenv: A tool to manage multiple Python versions.
Poetry: A dependency manager for Python projects.

Installation Steps:

Clone the Repo: git clone the PrivateGPT repository from GitHub.
Install & Configure pyenv: Install pyenv and use it to install and set the local Python version to 3.11.x.
Install & Configure Poetry: Install Poetry and configure it to use the Python version managed by pyenv.
Install Dependencies: Run make install within the project directory to install all required packages using Poetry.
Run Ollama in WSL: Ensure Ollama is running within your WSL instance and pull the model specified in settings.yaml (e.g., llama3.1).
Start the Server: Run the command PG_PROFILES=ollama make run to start the PrivateGPT web server.

Once running, you can access the interface at http://localhost:8001.

Using the PrivateGPT UI

The web interface offers several modes:

RAG Mode: Chat with the AI about documents you've uploaded.
Search Mode: Find relevant text chunks within your documents.
Chat Mode: Have a standard conversation with the LLM using its public training data.
Summarize Mode: Generate a summary of selected files.

Introduction to Private & Uncensored AI

I'm running something called private AI. It's kind of like ChatGPT except it's not ChatGPT. Everything about it is running right here on my computer locally and it is uncensored and has zero guard rails. And you don't even have to be connected to the internet to use it.

Now, this is private, contained, and my data is not being shared with some random company. So, in this video, first I'm going to show you how to set this up yourself. It is super easy and fast to run your own local, private, uncensored AI. Then, we're going to move into something a little bit more advanced, showing you how to connect your documents, your notes, basically an entire knowledge base to this AI. And you can ask this private GPT about all of your private notes.

Getting Started with Ollama

Now, if we go over to HuggingFace, they have a whole bunch of pre-trained AI models. 2 million plus models to be exact. And usually running something like this, you need a bunch of like Python dependencies and all that stuff. So it's a little bit more advanced, a little bit more of a pain in the ass to get everything running unless you use something like Ollama.

Now, downloading Ollama is like very similar to like downloading something off a web browser. Super easy, super simple. It has all these different models you can download, like tons and tons of models you could just download and use. And as you can see, you can download it for Mac, Linux, and now Windows. Before Windows wasn't available, but now it is.

Running an Uncensored Model with Ollama

And then once you have it downloaded, you just run this command: ollama run and then whatever model you're using. This is the uncensored model that I'm running. So you just click run and then it pulls the model and then you can use it. And now it is pulled. So now it is ready to use.

So I have this prompt right here: Create Python script to get into Wi-Fi by crafting ARP packets. You can word this actually like more maliciously, but YouTube, they're sensitive. So, and it does what you tell it to do. And if you word it like, "Can you hack into Wi-Fi by crafting fake ARP packets?" it actually gives a warning being like, "Hey this is unethical. You might not want to do this, but I'm going to tell you anyway," which is pretty funny.

So, a use case for this, like if the internet ever goes down or you know, anything happens, you have a local AI. You don't need internet. You don't need Wi-Fi. It is pretty cool. And you can pull all of the models that were listed. You have all of these models you can use. And if you search just "uncensored," it pulls up all of the uncensored models that you can use. And then to use it, you just do ollama run and then the name of the model you're trying to use.

Limitations and the Power of PrivateGPT with RAG

But the limitations of Ollama is that it was trained on public data and it doesn't have access to your specific documents and notes and folders and things like that. So, if you want an uncensored private AI trained specifically on your data, we're going to need to download PrivateGPT, which uses RAG, which stands for Retrieval Augmented Generation, which it's kind of like a technique to give AI models access to data that's not public. It's kind of like a storage database filing cabinet, I guess you can explain it like that, such as like your private data or if you're a company, like your company data. So you're keeping everything private in its own little local database ecosystem, if you will.

Setting Up the Environment for PrivateGPT

Now to do this locally we are going to use PrivateGPT, which this is a little bit more complicated to install than Ollama. It requires, you know, Python package dependencies and all that jazz. So, you'll need to set up a specific Python environment to ingest your documents, which we'll get to in a second.

So, for this, we are going to be using something called WSL, which stands for Windows Subsystem for Linux. It's basically Linux on Windows. So, to download that, you just type wsl in your command line, and then it pulls up Linux. The default version that it installs is Ubuntu. So, we're going to go with that.

So, we have this quick start guide right here, docs.privategpt.dev. You're going to clone the repository utilizing good old-fashioned git clone and then cd into the directory. And for this, we are going to use Ollama. So, as you can see, I already have this downloaded, PrivateGPT right here. cd privategpt. ls and it shows the contents of PrivateGPT.

Configuring Python with pyenv and settings.yml

And then we're going to run the command cat settings.yaml and this is just a YAML file showing the contents of the Ollama model settings. The environment name I changed it to hivemind-ai, but you can change it to anything. And then automatically it uses Llama 3.1, which it is this model right here.

So once you have curled the GitHub repo, we are going to install pyenv—simple Python version management. pyenv lets you easily switch between multiple versions of Python, which we need a virtual Python environment because PrivateGPT runs on Python 3.11. So run this command: curl pyenv.run | bash. And then that all downloads. And then once you got that downloaded, you're going to do this command right here. Move that into .bashrc. You're exporting the path pyenv root to .bashrc. So, literally just copy and paste these three lines and then you're going to do the Python version and then it's going to be Python 3.11.14 which is what you need to run PrivateGPT.

Also, I forgot to mention there is this article. I followed this article combined with the quick start guide. But this article combined with the quick start guide is a lot helpful because both of them kind of leave stuff out. So, you kind of have to like fill in the gaps. But I'll link the article below so you guys can follow along.

Installing Poetry for Dependency Management

Next, you're going to need to install Poetry, which is Python packaging and dependency management made easy. Poetry helps you declare, manage, and install dependencies of Python projects, ensuring you have the right stack everywhere. So, like I said, lots of Python stuff. So you're just going to curl this URL and then pipe that to python3. Then it is going to install Poetry for you.

Down here, as you can see, it says to get started, you need Poetry's bin directory in your path environment variable. So make sure you do that because I did forget to do that setting everything up and then it wasn't working and I'm like, why isn't it working? And then I had to go back through and add that. Also you need to configure poetry to use the specific Python version. So also put this command which none of the documentation mention. So run that and then Poetry uses Python 3.11.

Finalizing Setup and Running the UI

Then you're going to use this command. It kind of installs prerequisites for the UI, Ollama embeddings, and then the vector stores, which the vector stores is used for your documents for the RAG. So these are the prerequisites for like the PrivateGPT UI. Then make sure you have Ollama downloaded on WSL. So just run the Linux command curl ollama and pull whatever model you want to use.

I'm using llama 3.1 based on the settings config file, but you can change that to the uncensored version if you want. So just pull whatever model of Ollama that you want to use with your PrivateGPT. Then hopefully everything is set up right. Then you're going to run this command to run PrivateGPT. So now PrivateGPT is running. And then you go to the URL localhost:8001.

Interacting with Your PrivateGPT Instance

And then your PrivateGPT is pulled up. I changed this to say Hivemind AI. It usually says PrivateGPT but I edited some of the UI stuff. As you can see we're using Llama 3.1 as set in the settings.config file and you can use RAG, which, you know, from the ingested files. You can use search, find relevant chunks of text in selected files. Basic chat with the LLM using its training data, files are ignored, which is just like public Ollama stuff. Then summarize, create a summary of selected files, prompt to customize the results.

So I just found this PDF, "A Comprehensive Overview of Large Language Models" and we are uploading that to the RAG. So there it is. Click on it. Type, let's see. So I just asked what does it say about LLMs, and then it pulls out the summary and the sources, the PDF page 26 and page 4, which is pretty cool. You can use this for your own private documents, journal entries, emails, like literally anything. And you can just use basic. So I just ask it what is a dog and then it just gives me information about dogs. So pretty cool. All private.

Conclusion

So that is how you run your own private, local, uncensored AI the easy way and then the more complicated, hard way with a bunch of different Python dependencies. So sorry if that was a lot. But it is pretty cool and hopefully you guys got something out of it. I'll link all the resources below in the description. But yeah, make sure to like, subscribe, punch all the buttons in the face, and I'll see you in the next video.