Introduction & Local Setup
The march of continues and we are also continuing our coverage of Qwen 3.5 small model series. We already have covered the 0.8 billion model and in this video we are going to install this nine billion parameter model. We are going to test it out on multimodality on images, videos and text. So let's get right into it and I will be talking further about this model.
I'm going to use this Ubuntu system. I have one GPU card, Nvidia RTX A6000 with 48 GB of VRAM. Now the tool which I'm going to use in order to get it installed is VLLM. I already have installed it and showed you how to install it in the 0.8 billion one. All I have done is run this command to install VLLM, this command to install transformers, and then we are going to serve the model locally.
When you first time run this, it just simply downloads the model for you. So I'm just going to run this. I have one GPU, that is where tensor parallel size is one here.
Qwen 9B Benchmarks & Architecture
Now as it downloads the model, let's talk a bit more around this nine billion and what exactly has happened here. So look, this is what happened today where Qwen has released all of these models ranging from 0.8 billion to nine billion. We're also going to cover this two billion and four billion by the way. And this is a very quick comparison. Every single model in this family serves a different purpose as you can see. This nine billion one closes a gap with large models. Plus the architecture is simply sublime.
Now if you look at these benchmarks, this gives you the true picture here. Every model has same foundation, same architecture principles, but this is where things get seriously impressive. This 9 billion model is the model which is competing with GPT, with Anthropic's Claude, and various other leading models even from China. Benchmark charts makes that very, very clear. Look at GPQA Diamond at 81.7. Look at HMMT Feb 25 at 83.2. MMMU at 70.1. The 9 billion is sitting right alongside or beating models like GPT-OSS and Qwen 3 next 80 billion on multiple tasks. That is a 9 billion parameter model competing with 80 billion parameter model. That is amazing.
Now, if I talk about the architecture of this 9 billion model, this is where also things get very interesting. The 9 billion shares the same hybrid layout as the 4 billion, which is 32, you know, 32 layers arranged in a pattern of three gated DeltaNet blocks followed by one gated attention block repeated eight times. But, the hidden dimension jumps to 4096 and the feed-forward network intermediate dimension goes to 12288, which in simple words mean it has more capacity. It defaults to thinking mode, meaning it reasons before responding just like a chain of thought model, and supports context up to 262K token natively, which can be extended to 1 million with Yarn scaling. And I have done a separate video on that. It is a full vision language model with video understanding built in from the ground up.
Serving the Model & Resource Usage
So, now you know what exactly this model is. Let's go back to our terminal. The model has been downloaded and takes around 18 gig of size on the disk. I will also show you the VRAM consumption.
Meanwhile, you can like the video and subscribe and consider becoming a member of the channel. Also, follow me on X if you're looking for AI updates without any hype. I will drop my X link in the first pinned comment. And the model is now being served on my local system. Let's access it through open web UI so that we could talk with the model in the web browser.
Coding Test: Bitcoin Hacking Animation
And you can see that the model is now loaded. Let's test it out. First up, I'm doing a coding test. I'm asking it to create me a self-contained HTML file that visually simulates a Bitcoin mining blockchain hacking terminal animation. We also tested it out with 0.8 billion. It was really good. I have made the prompt more challenging because it's a 9 billion parameter model.
So, it's a thinking model. It is going to think. It is going to take a bit more time. And default is thinking is on. You can also change the parameter in open web UI. While it thinks, let's check the VRAM consumption. So, again, it is fully loaded onto the GPU consuming just over 44 GB of VRAM. You can also just try to reduce the context window if you want lower VRAM consumption.
So, the model is loaded and it is now thinking. Let's check out its chain of thought. So, it is understanding the goal. It is coming up with a strategy of each and every component. Looks pretty good. It is also checking some of the alternatives and now it is creating a step-by-step implementation plan. And then it is drafting the code mental or scratchpad. That is nice. Final polish. And now it is producing the code. So, let's wait for it to produce the code and we will run it in the browser.
And Qwen has finished doing its magic. Let me open it in the browser. I'm just going to download this from here and open it here. Look at this. Wow, running live. Blockchain hacker, yes, absolutely. It has even surpassed the brilliance of its 0.6 billion younger brother or youngest brother and you can see it has done some nonce proto flow and then look at the top. It is so, so good. I mean, you can create a very nice website out of it.
It is, you know, activating Coinbase, some wallet balance, everything which I asked it to do, it is doing. And then it is doing the minings. And look, every graph is also moving if you look closely here. And then how many peer nodes are there? It is just a simulation, remember that. Look at this. How good is that? I mean, it could have been improved like it should have some things here, but I think it has done well. This is a target. And it is actually trying to meet that target. Even in the simulation, I have just given it some flexibility. In the 0.6 billion, I had hardcoded it to find the block, but not in this one and that is what it is trying to do. It is actually running some mathematical long chains in order to find the block. So, it is quite realistic simulation. Very nice. So, look, I'm very happy with the code.
Multilingual Test: Initial Failure & Prompt Refinement
Okay, coding is cool. Now, let's quickly do a multilingual test. For the multilingual one, I'm going to ask it to write the numbers 11 to 13 in all following languages. The 0.8 billion one, it miserably failed in most of the languages. Let's see if this one is able to do that. And again, I will seek your advice if that is your language. Please confirm in the comments if it has it was able to do that or not. You see it is just analyzing the request, verifying the languages, making some adjustments. It's always fascinating to watch this train of thought from the models and it is improving a lot. It is getting more targeted.
Um you know, less and less Chinese characters are appearing when you give it a prompt in English, which is really really good, I believe. Okay, so I will let it finish and then we will check it out together.
The model has completed its thinking, but you see a funny thing happened. Instead of giving me the numbers, it has translated the sentence or instruction itself in all the languages. And it has just put the number in this numerical format, which I don't want. So, what I'm going to do, I'm just going to click on new chat and give it again. I will try to make the prompt better.
So, this time I'm just going to say write the numbers 11, 12, and 13 spelled out in words in each of the following languages. And then, I'm also giving it an example like this is English and this is Spanish. Let's see if it is able to do it now. Let's wait for it to produce.
Multilingual Test: Successful Execution
And now when I look at this thinking, at least it is going into the right direction. So, let's wait for it.
After thinking for 7 minutes, as we can see here, it has come back with the response and tell you what, it has done wonderfully well as far as I can see. Some of the languages I have checked, some of them I know, and they are perfect. So, it is 11, 12, 13. And in Urdu 11, 12, 13, I think it has done well. Arabic looks good, French looks good, Spanish looks good. Um and various other languages which I have checked look good. I'm just scrolling down.
Please check your language. Let me know in the comments. Especially the low-resource ones. Like Nepali and then, you know, Icelandic, Greek, and all that. Look at the gibberish. Give club zip. So, it has done something.
Image Understanding: Traffic Light VQA
Okay. Now, let's try to do something different. I'm going to now do a video understanding. So, let me upload a video file from my local system. And I hope I have some videos here. Let me go with this one. Okay. Okay. So, MP4 is not supported for processing. Maybe I will access it through code later. I think video it is not supporting. I might need some plugin. So, let's do image. We will do the video through the code shortly.
So, let's go back to this and then let me select the image. So, for this VQA or visual question answering, I'm going to ask how many traffic lights are there. Let's see the answer because there are also some street lights. Let's wait for it to come back. It is going to think hard and long.
And there you go. The model has come back with a response and says that based on the visual evidence, there are two traffic lights. And now it is just giving us a short answer too. And the thinking is again very impressive.
Video Understanding via Code
Okay. So, that is done. Let's do a video one. So, for video, I'm going to use this AI-generated video where there are some cheerleaders having fun. Now, okay. So, you have watched it and enjoyed it. Let now show you the code which I'm going to use. So, this is a code where I'm just using the um this vLLM endpoint where we have served the model and then I'm asking it, "Who are these people?" and this is the video path.
So, let's go back to the terminal and run this. So, I'm just going to run it from my Python app.py. And there you go, it has come back with the response which is, "These are cheerleaders performing on a football field." Also, I'll show you one very quick thing. Let me rerun it. So, it is running and just see that it will be very quick answer. There you go. Different because it's a model with some creativity. It is giving us more detail.
Now, what I have done here is I have just enabled the thinking false. So, you can switch between the modes, as I said. By default, it is true. Let's do another video. I'm going to give it this again, yet another AI-generated video where two kangaroos are having some sparring. And I'm asking the model, "Which kangaroo will win the fight?" So, let's see what model says here.
Let me clear it. Let me play this and wait for it. Again, a non-thinking one. Shouldn't take too long. And the model has given a very grounded answer that it is uncertain. Both are skilled fighters. And their performance will depend on their agility, strength, and strategy. So, hallucination test passed. Model didn't make up anything.
OCR Test on a Spec Sheet
Let's do a bit of an image processing. I'm giving it this spec sheet where there are few items and this last one says to be determined. There is no photo available. So, I'm asking the model which item in this image doesn't have photo available yet. So I have turned on thinking so that it should be able to spend more time and give us a more grounded answer.
And there you go, another cool, great, very correct answer that on the far right doesn't have the photo available. You can tell this because the image placeholder for that card has a red stamp over it that says to be determined. So, not only it does fairly accurate OCR, it can also visualize the colors too.
Analysis of AI-Generated Handwritten Text
Okay, let's try to do another hard one on a handwritten text. Again, this is AI generated. There is some spelling mistakes and all that stuff. Some crossed out words. I'm asking it, did a human write this? If yes, what this text is about and how old would this paper be? Let's see what model does here. Very keen to see its thinking.
So, yes, I'm asking for analysis. And then, you see, it says that I need to look for signs of AI generation or messiness. And it says it looks a bit off. Some words are crossed out, the grammar is strange, and the layout is even. You know what? I think this is so, so good. It says it looks like a deep fake of a letter. Yep, that's correct. And it has extracted that text very accurately.
And then it is said, if it were real, and sorry, I'll just let it go and then we'll check it out. But look at this thinking. So, the thinking is done and look at the answer. It says, did a human write it? No, it is highly likely AI did that. Amazing, and then some, you know, reasoning there, which is awesome. You know, the flower, the red rose bud on the right looks slightly pasted on.
What is the text about? It's a love letter. Some mistake there. Or a note of longing. And it says that the sentiment is clear, so sentiment analysis is there. How old this would be? And it is talking about roughly mid-20th century. Like this. But actually, since it's a computer-generated image, it is effectively brand new. Zero year old. That's true. This is perfect. Love it.
Multilingual OCR: Polish and Arabic
Okay, so now this next test is in Polish language. I'll just You see that this is just some random Polish image which I generated from AI. I'm asking it which language is that and can you translate it into English? And hello to all the, you know, Polish viewers. And surprisingly and very very, how should I say, pleasantly, we have so many Polish viewer. We have viewers from all over the world. I'm also going to do a video on it very soon. But and at the same time, the number of Polish user is just amazingly very high. So, hello to all of them along with others, of course.
Okay, so let's see what it does. If you look at the thinking, it says it's clearly Polish and French. Yep. And then it is just going through it. I think it has already done that translation, which looks pretty good to me. What do you think? There you go. I think it's perfect. Really good.
Let's do one more multilingual. So, I'm giving it this Arabic one. Likely from a magazine or newspaper. And then I'm asking it what what this text says and what type of publication is this? So, let's see. And there you go. This is the response and you know, as far as I can see it has identified the personalities, some political events in Egypt, and some newspaper. It has got it right. But if you're Arabic speaker, please let us know.
And this is the actual newspaper. I couldn't find a reference. If you know what the reference is, let me know. I'll put it in the link in the video's description, but I think this is quite an old one, I believe. Anyway.
Mathematical Reasoning & Final Verdict
Let's do a one final one. I'm just going to check out a formula. Now, this question is not only going to see if it can read these formulas in LaTeX but also the math capability. So, I'm asking it to solve the question six and give me correct answer. Only focus on question six.
Now, I'm expecting it to think long and hard. Let's see, it has done the LaTeX pretty well. It is identifying that expression. It is checking on the both sides. Again, it has sliced and diced the equation. Grouping it. Going with the alternative in a typical fashion of chain of thought and then it is structuring it. It is solving question six, so instruction following is also quite good. It's not just wavering around.
And now it has Already I can tell you it's on the right track and yes, absolutely the right answer is five. So, look, I could go on and on, but I think it's already way over my real time of a video, but 9 billion masterstroke from 3.5 family, I think very, very well suited for small to medium-size enterprise. Let me know what you think. As I said earlier, I'm also going to cover this 2 billion and 4 billion. Already covered these two. Have fun. Please like the video and subscribe and consider becoming a member of the channel because members are who keep the lights on off the channel. So, huge, huge thanks to them. It is growing steadily, not as much as I would like. Also, please follow me on X if you're looking for updates without any hype, without any fluff. Thank you for all the support.
