Made withOllamaClip

Claude Code + Ollama = Free Forever

L

Leon van Zyl

Creator on the Ollama channel

Watch on YouTube ↗

263K viewsJan 25, 20269:15 runtime

Learn how to run the Claude code CLI completely free and offline by using local large language models through Ollama. This guide walks you through installing Ollama, downloading recommended open-source coding models like Queen 3 Coder, GLM 4.7, and GPT-OSS, and launching Claude…

Chapters

#ollama#clip#aillm

Run Claude Code for Free with Local Models

You can use the Claude code CLI with local, open-source models via Ollama for a powerful coding experience that is completely free, unlimited, secure, and offline. This setup retains all of Claude code's native features, like slash commands, planning mode, and sub-agents.

Setup and Installation

Install Ollama: Download from ollama.com and verify the installation in your terminal by running ollama. Check for existing models with ollama list.
Download a Model: Browse available models on the Ollama website. Use the command ollama pull <model_name> to download your chosen model (e.g., ollama pull qwen2-coder).
Install Claude Code: Follow the installation instructions on the Ollama website's Claude code page to get the CLI tool.

Important
To run Claude code with local models, you must use the special command ollama launch claude. The standard claude command will prompt for an API key. To change your selected model later, use the --config flag: ollama launch claude --config.

Recommended Local Models

Your choice depends on your hardware, particularly VRAM. Here are the key recommendations:

Queen 3 Coder: A strong all-rounder requiring ~19GB of VRAM. It's a favorite due to its excellent performance and a large 256K token context window.
GLM 4.7 Flash: Similar in size to Queen 3 Coder (~19GB VRAM) but with a smaller 198K token context window that aligns well with Claude code's internal limits.
GPT-OSS (20B): A great starting point for less powerful hardware (~14GB VRAM or less). Its main drawback is a smaller 128K token context window, which can lead to performance degradation on larger projects.

Key Considerations

Performance Expectations: Local models are fantastic for learning agentic workflows and performing many coding tasks. However, they will not match the reasoning capabilities of state-of-the-art cloud models like Opus or Sonnet 4.5.
Context Window: A model's context window is crucial. Claude code starts to summarize conversations as they approach its ~200K token limit. Models with larger context windows (like Queen 3 Coder) handle large, complex codebases more effectively.

Tip
Troubleshooting: If you encounter errors (e.g., model not found, version conflicts), copy the text from the official Ollama documentation page and your specific error message, then paste it into a more capable AI like ChatGPT or a cloud-based Claude model for assistance.

Demo: A Free, Offline Weather App with Claude Code

In this video, I'm going to show you how you can use local models in Claude code. I just used Claude code to create this real-time weather app. This app calls an API to fetch real, accurate information from the web, so I can enter any city in the world and it will retrieve up-to-date weather information for that city, along with all of these beautiful little animations.

Now, here's the kicker. This is completely free. No need for any subscriptions or API costs. And these models are running locally on my own machine, which I can use for unlimited times and it's completely secure and offline. In fact, if you have a look at the CLI, you will notice that I'm actually using Queen 3 coder and not Sonnet, Opus, or Haiku. And because we're running in the Claude code CLI tool, these agents have access to all of the features available in Claude code, like slash commands, skills, MCP servers, sub-agents, you name it. We can also switch between edit mode and plan modes as well. Everything just works. And yes, this is running on consumer-grade hardware.

Setting Up Ollama for Local Models

The first thing we need to do is to download models onto our machine. For this, we will be using Ollama. If you're new to Ollama, it's a fantastic tool for running large language models on your own machine. So, go over to ollama.com, then click on download, and then download and install Ollama. Afterwards, you can verify that Ollama is actually working by opening up your terminal or command prompt and running the command Ollama. You should receive something like this, which shows you all of these available commands. And if you run Ollama list, it will show you all of the available models on your machine. If this is your first time using Ollama, you won't see any models at this stage.

Choosing the Right Local Code Model

So, how do you download models then? Again, on the Ollama website, simply go to models and now you have different models for different purposes. For Claude code, if we actually go to the Ollama website and click on Claude code, we can go down to recommended models and Ollama themselves actually recommend Queen 3 coder, GLM 4.7, this is actually GLM 4.7 flash by the way, and the GPT-OSS series of models from OpenAI. So, let's go back to models. Here's my recommendation based on your system specs. Let's go to GPT-OSS.

The smallest model out of all of these is the GPT-OSS 20 billion parameter model. You only need about 14 gig of VRAM. And in fact, this number is misleading due to the architecture of the model. You can get away with running this on way less VRAM. Then, if your hardware can handle it, I actually recommend using Queen 3 coder, this 19 gig model over here. Then, it's also worth mentioning that the GLM 4.7 flash model was recently added to Ollama as well. Just like the Queen 3 model, this is also a 30 billion parameter model. And this also clocks in at about 19 gig, just like Queen 3.

Model Recommendations and Hardware Considerations

Now, this is just my opinion, but I feel like I actually got better results from Queen 3 compared to GLM 4.7 flash. If you've got a more capable graphics card, you can definitely go for GLM 4.7 flash, the Q8 quantized version. This requires about 32 gig of VRAM. So, at this point, you probably need something like an RTX 5090 to run this model.

And if you don't mind a little bit of offloading and slower responses, you can definitely go for GPT-OSS, the 120 billion parameter model, which is the largest out of all of these.

Downloading a Model and Installing Claude Code

So, once you've selected a model, all you have to do is copy the name of the model, then back in your terminal, simply run Ollama pull and then paste in that model name. And after downloading the model, we can run Ollama list and you should now see the model in this list. Cool. We can now close the terminal and we should now be able to use these models in Claude code.

So, how do you install Claude code then? Well, from the Ollama website, simply scroll down to Claude code. Then, from this page, we can go down to the install section and then depending on your operating system, simply copy this command and simply run that command in your terminal. This will now download and install the latest version of Claude code.

How to Launch Claude Code with Ollama

So, now that we have the model downloaded and we've got Ollama up and running, all we have to do now is start Claude code and use these models. But, there's a catch. If you try to use Claude normally by running the Claude command, you will be taken to the setup instructions and you will be asked to sign in with either your Claude code subscription or your Anthropic API key. So, I'm actually going to press control C to exit out of this. And there's actually a special way to start Claude code using Ollama. What you have to enter is ollama launch claude. Optionally, you can add --config to kind of reset the setup process.

So, I'm just going to run it without the config flag. Then, let's select our theme. We can press enter and you will notice it didn't ask us to sign in with our subscription or provide an API key. We can simply press enter. We can use the recommended settings and that's really it. Now, we have access to Claude code and we are using an open-source model that's running securely on our own machines.

Switching Between Different Local Models

Now, what happens if you download additional models and you want the option to select between a different models? Now, since we've already run that setup process, this command actually won't ask us again what model to select. It will simply default to the previously selected model. So, in order to go through the setup process again, we simply have to add this --config flag. And now, we have an opportunity to select one of those models. So, let's go with GPT-OSS, the 20 billion parameter model, just to keep things interesting. And let's see if everything works by saying, "Hey." And look at that, we get our response back. And just to prove to you that we're not running any Claude models behind the scenes, I'll start the Claude command and you'll see this error message, "Missing API key." And that's because I'm currently signed out of my Anthropic account. So, this is indeed completely running offline.

Local Model Performance and Use Cases

And we can still go into planning mode and change mode. We can create sub-agents. We can hook up custom commands and everything else.

Hey, please can you describe this code base? And let's send this. So, while this is running, I do want to mention that these models will never compete with the likes of Opus and Sonnet 4.5. It's just unreasonable to assume that you can run something of that caliber on your local hardware. But, these local models are perfect for starting out with agentic coding and vibe coding and you might not have the means to pay for these expensive services. And it doesn't mean that these models are useless either. You can definitely use them to make quick little edits in your code base. And as you can see, GPT-OSS was perfectly fine with describing this application.

Troubleshooting Tips for Ollama and Claude Code

If you run into any issues using these models, what I recommend you do is simply go to this Claude code documentation on the Ollama website, copy this page, and then pass it to a more intelligent model like one of the Claude models or ChatGPT. It really doesn't matter. Just pass in this documentation and explain the issue that you're having. For example, while I was preparing for this video, I kept getting these 404 error messages in the terminal. So, it effectively said that it couldn't find the model. Or when I tried to download one of these models, I believe it was the GLM 4.7 flash model, Ollama said it couldn't download the model because I don't have the correct version of Ollama installed. So, I found that by passing in the content of this page to a more intelligent model, it was able to resolve all of these issues for me.

Comparing Model Context Windows

Another important factor between deciding on the different models is the model's... So, something to take note is the GLM 4.7 model has a context window size of 198K tokens. This is similar to the 200,000 token limit in Claude code. When you get close to reaching this limit, the conversation will be summarized or compacted. So, the GLM model does fit nicely into that context window limit. The Queen 3 coder model actually supports up to 256,000 tokens. This means the 200,000 token limit in Claude code doesn't come close to the limit of this model. Whereas, the GPT-OSS models only have a context window of 128,000 tokens. This means in Claude code, you'll see the quality of the responses degrade when using about 50% of this maximum context window size. And that's why the Queen 3 coder model is basically my favorite model at this point in time.

Conclusion

I hope you found this video useful. If you did, please hit the like button and subscribe to my channel for more Claude code tips and tricks. I'll see you in the next one. Bye-bye.