The Problem: AI Integration in VS Code
Usually, when working with code using AI, I use Zed editor or Google anti-gravity. But from time to time, I need to use Visual Studio Code. And in this editor, I really miss solid support for working with LLM models. You might say, "Just use Cursor. It's basically VS Code, but with AI features built-in." And yeah, that would be a decent option. But due to the specifics of some of my projects, I actually need to use the original Visual Studio Code. So last weekend, I decided to set up LLM integration directly inside VS Code.
And what's interesting is that this approach lets you connect almost any model, whether it's a paid one like Claude or Gemini, or a free one running locally on your machine. I'll be showing everything using Gemini for running locally as an example. Hi, I'm Nick, and I've been developing software for over 20 years. On this channel, I share my experience, insights, and thoughts about IT.
Initial VS Code Setup and Theming
So, if you don't have Visual Studio Code installed yet, it's the usual process. Just download it from the official website, literally one click. It's available for pretty much every operating system. To make sure I show the full process from scratch, I actually removed VS Code from my computer beforehand. So when I launch it now, it'll start with the default settings.
First, let me change the color theme. I use Everforest pretty much everywhere, so I'll start with that here as well. By the way, as you can see, a panel with an agent opened on the right. And at first glance, it looks similar to what you'd find in other AI-powered editors. However, unfortunately, this panel is designed specifically for Copilot, and I wasn't able to get it working with a local model. So I'll just close it and won't touch it today.
I'll also quickly make a few small tweaks to the editor's appearance. As I can see, the current color theme doesn't clearly separate the main code window from the other panels. In the VS Code settings file, I'll change the sidebar color so it's visually distinct from the editor itself. Then I'll make the border between the sidebar and the editor more visible. And while I'm at it, I'll also adjust the colors used to indicate nesting levels in the sidebar.
All right. Now, I'll open the project I used when recording my Claude code review. Just as a reminder, the goal in this project is to build a very simple task manager. There's a pre-prepared markdown file with a list of basic tasks, and completing them should result in a simple web-based to-do app.
Setting Up Gemma 4 with Ollama
Okay. Now, let's deal with Gemma 4 itself. I need it to be running locally. For that, you can use pretty much any LLM runner. For example, LM Studio, llama.cpp, or Ollama. Today, I'll be using Ollama. Since I already have Ollama installed, the only thing left is to choose which version of Gemma 4 to use. On the Ollama website, in the model section, I can find Gemma 4 and check which variants are available. Because I'll be running the model on my MacBook with an M4 Pro chip and 24 GB of RAM, the 26 billions and 31 billions parameter versions are a bit too large. So, I'll go with the 8 billions parameter version.
Let me quickly check if I already have it downloaded. Nope, it's not here yet. As you can see, I do have the previous version, Gemma 3, but that's fine. Let's download it. This is done with a single command, Ollama pull, followed by the model name. Once the download is complete, let's check if it appears in the list. Yep, there it is. Perfect. I'll also run it directly in the terminal in chat mode just to make sure everything works, that it downloaded correctly, launched properly, and is responding. All right, the model is replying. Looks like everything is working fine, so now we can move on to using it inside the editor.
Installing and Configuring the Continue Extension
Now, it's finally time to connect Gemma 4 to Visual Studio Code. For that, I'll need to install an extension. I'll go to the extensions tab, and if you search for something like local LLM, you'll find a lot of different extensions for working with local models. I'm going to pick one of the most popular ones called Continue. I chose it because it's widely used and also open source. According to the description, it allows you to use a local model both in agent mode and in other modes, like a regular chat, which is exactly what I need.
Before installing, there's the usual question about whether I trust this extension. Never ignore prompts like this. Take a moment and think about whether what you're doing is safe. In this case, since the extension is very popular and has a good rating, I'm going to install it.
Now, to start from a clean slate, I'll delete the HTML file that Claude Code generated last time when I was testing it with Gemma 4. And if you haven't seen that video yet, where I show how to connect Gemma 4 to Claude Code, definitely check it out. There's a lot of interesting stuff there.
All right. As you can see, something broke in the UI. The pop-ups have this weird transparency. I think it might be caused by the background settings I changed for the sidebar. Let me comment out that line and see if that fixes it. Nope, that didn't help. All right, then. I'll just switch to a different color theme so it's easier for you to see what's going on. Let me pick the Solarized Dark theme. I like this one a lot, too.
Connecting the Model and Submitting a Task
Yeah, now it's fixed and the pop-ups are rendering correctly. As you can see, the models I have installed locally are already available here. Let's compare them with what Ollama shows, just to make sure these are the same ones. Yep, as you can see, the lists match. So, VS Code is correctly picking up the models from Ollama.
If your local models don't show up automatically, you can always add them manually. This can be done through the model configuration settings. First, you choose a provider. It could be something like Gemini from Google or Anthropic, but since those are paid models, you'll need to provide an API key to use them. There are also plenty of local providers, like llama.cpp or LM Studio.
All right, looks like everything is finally set up. Let's take the first task from our markdown file and give it to the model. The model immediately started thinking. That's a good sign. But then, for some reason, it stopped and didn't create the HTML file as described in the task. Let's figure out what might be wrong.
Usually, this happens when the model can't use tools properly, so first I'll go into the tool settings and check what's going on there. All right, let's check the settings. Reading files is set to automatic. That's good. But for creating new files, the model is set to ask for permission. In my case, that's a safe operation within this repository. So, I'll switch it to automatic. Let's keep going. Running commands in the terminal. Yeah, that should still require confirmation. But reading the current file doesn't need approval, so I'll set that to automatic as well.
Next is reading skills. I don't really need that right now, but I'll leave it on automatic. It might be useful later. I'll also allow it to edit the current file automatically. And find and replace can run without my involvement, too.
All right, that should cover everything. Now, let's go back to our task and tell the model to proceed with what it planned. Okay, the model stopped again and still isn't creating the file. But since I've already checked the tool settings and everything seems fine, maybe Gemma is just waiting for me to approve its plan. All right, let me confirm it.
Executing Coding Tasks with Gemma 4
Oh, nice. Now, it finally started creating the file. Let me open it in the browser and see if it did what the task required. As you can see, the HTML file was created and it displays the list of tasks, exactly what was required in the first assignment. Great.
Now, let's take the second task and send it to the model again. The model started thinking and it looks like it's once again asking me to confirm the implementation plan. All right, I'll approve it. And again, it's asking for permission to make changes. Okay, confirming that as well. All right, looks like it finally started applying changes.
Okay. Gemma says it's done. Let's check in the browser if anything changed. Just a reminder, the second task was to add an input form to the page so that new items can be added to the list. And the page loaded without errors, and the input form is there. Perfect. Exactly as expected. Let's test it. Yep, everything works as it should. Great.
Conclusion and Final Thoughts
All right, I won't continue with the rest of the tasks since it's already clear that everything is working. I achieved what I wanted. Now I can use my local models directly inside Visual Studio Code, and I also got a chance to test how Gemma 4 performs in this setup.
By the way, as you can see, the performance of Gemma 4 on a laptop is decent. For small, well-defined tasks, this model works pretty well. Just a reminder, I was using the 8 billion parameter version, not the largest one. And for a model of that size, the results are quite impressive. But I'll repeat what I say in almost every video about local models. Don't treat them as a full replacement for paid models. They're more of a complement.
All right, that's it for today. If you enjoyed this video, don't forget to like and subscribe, so you don't miss the next one. See you soon. Take care.
