Introduction to Deepseek OCR & Data Crunch
Deepseek OCR is the latest and hottest release from Deepseek which is much more than OCR. And in this video, I'm going to show you how you can run this at scale. To make it easier, I've partnered up with Data Crunch, who's a European cloud provider. And Data Crunch has kindly agreed to give $75 for you to get started with Data Crunch for users who are signing up from the link in the description with this particular coupon code. And in this video, I'm going to show you the step-by-step tutorial about how you can use Data Crunch to run Deepseek OCR at scale using vLLM.
Before we move further, a little bit about Data Crunch. Data Crunch is a European cloud provider focusing on privacy, transparent, fair pricing and it is independent of big US hyperscalers. Data Crunch also focuses on sustainability. That means using 100% of renewable energy and it is also ensuring that it is compliant with GDPR and other local regulations. You can sign up to Data Crunch from the link in the description and use the coupon code onelittlecoder-dsOCR for you to get $75 to exactly follow this tutorial. Soon Data Crunch will be under a different name. Stay tuned for more details.
Setting Up the Cloud GPU Instance
After you sign up, you will see a dashboard something like this and then you can select the project. Either you can create a new project or select an existing project and then click create an instance. Once you click create an instance on the right hand side you can see how much it is going to cost. And on the left hand side you're going to configure the computer that you want. Select on demand and pay as you go. And then I'm going to select Tesla V100 machine. This is just a 16 GB memory machine. And once we have that you can select the right configuration you want. And then we're going to start with fixed pricing for now. And then you can select the location you want. So I'm going to select Finland. I'm going to select with Jupyter Lab. So if you have Jupyter Lab enabled then you can just do everything in your browser. Otherwise you might have to SSH into it. So just something for you to keep in mind. You can select the storage as well. So I'm going to go ahead and then give 100GB as the storage. So this will be by default. You don't have to separately mount it. And then finally you have to enable the SSH key. It's very clearly documented how you can create an SSH key from your computer and then add it here. So you can go to keys here and then just create a key and then add the key here, so that that key is available within your interface. Once that is done, your machine is now ready to be deployed and it is going to cost you just less than 20 cents for 1 hour and that is what we're going to use with DeepSeek OCR powered by vLLM so that we can do batch OCR at scale. So I'm going to just deploy it here.
Preparing the Jupyter Lab Environment
Once your machine is up and running then you can click open Jupyter lab and once you do that your Jupyter lab is going to be available. So once Jupyter lab is available then you have got multiple options. If you're not familiar with Jupyter lab, just host a Jupyter notebook with some other settings where you can access terminal and such things. So I've got Python 3 kernel that I can just start from scratch. But in my case I already have the vLLM notebook which I'll link in the YouTube description for you to get started. So you can straight away go ahead and then use it. I'm going to just simply run this for you to see the machine configuration. So you can see we have got a Tesla V100 machine and it's got 16 GB memory. So it's not a very, very powerful machine that we are going and that also helps us in reducing the cost. But if you want to do it for like 10,000 documents then you can rent A100s, H100s and then really, really rock with Deepseek OCR. Once we have the machine up and running, the next step is for us to install vLLM light nightly.
Installing Dependencies and Sample Data
So by the time you watch this video, maybe you can just install the latest vLLM, but to be on safer side you can install the vLLM nightly. It's going to take some time. So I'm using pip but if you are a fan of UV you can go ahead and do it with UV, that might save you time. But I just wanted to be easier for everybody who's watching. So I've gone ahead and then done it with pip. So now you can see that it has done everything, all the things that are required including PyTorch, including torchvision, everything required for us to get started with this project, including vLLM. At this point we are going to download the input images. We're going to just download two images, one with handwriting, one with doctor's prescription, and we're going to just keep it here. So you can open it and then see this is the first image we're going to use and this is the second image that we're going to use. You can, this is just for sample, but you can have like hundreds of images as, and the code is written in such a way that you can easily optimize it. Once you have the input images, you have vLLM setup ready, the next thing is for us to import the required libraries.
Loading the Model with vLLM
You all you need is vLLM and you need Pillow. Pillow is to do image processing in Python. After we have successfully imported all the required libraries in this case vLLM and Pillow, then we have to create a model instance with vLLM. Now when we create the model instance for the first time, it is going to download Deepseek OCR. But for the second time and the third time, because Data Crunch gives you persistent storage, you can keep it ready. So if you want to do batch OCR processing over a couple of days then you don't have to every time purge the entire thing and then reuse it. So you can keep it. It doesn't cost you a lot of money, few cents, then you can just use it. So it will be easier for you to kickstart and then start doing the OCR process. So in my case I've not deleted, I've not purged my instance. So you can see that it just took a couple of seconds to completely load the model. Once we have the model successfully loaded and the LLM instance created, then we have to prepare the input files.
Configuring and Running OCR Inference
In this case, I've got two input files as you know. So the first input file is being loaded using the Pillow and then converted into RGB. The second input file is being loaded and converted into RGB. Then you have to give a very small guidance prompt. So in this case we're saying image and then free OCR and that goes inside this model input. So you can also do like markdown and other aspects that you can try out but because we are focusing exclusively on OCR I'm going to just simply do OCR. Then you have, you're going to have sampling parameters, very simple LLM parameters like temperature, maximum token and everything. Once we have all these things then all we have to do is ask LLM to generate the output and print the output. I'm going to run this so that you can see in real time how much time it takes on a machine like this. But if you have a machine like let's say A100 and H100, it is going to be extremely, extremely faster than what you're seeing here now. So it is about 16 tokens per second. Input is about 76 tokens per second and again it depends upon the kind of compute that we have got. We have got a very simple 16GB Tesla V100 machine and if you have got a powerful computer then it is going to change the picture. So the first one is have you heard of Iron Man? I'll tell you about Iron Man.
Analyzing the OCR Results
So we can see here, have you heard of Iron Man? I'll tell you about Iron Man. And even like the handwriting where it is not like very, very clear, you can see Deep Seek would have done a very good job. Like for example here, steep, high, dangerous cliff. So I didn't know that until I saw that, steep high dangerous cliff. So you can see that this is a pretty pretty good model. Now let's look at the doctor's handwriting here. So we have got Mr. Sachin. I'm going to go back here. Mr. Suchin and uh it says 28 male and the date is 12-10-2022. 12-10-2022 and you can see all the other medicines as well like Enziflam, Augmentin, Augmentin 625 and Pandi. So we have got Pandi and there is a Hexigel gum paint and we have got a Hexigel gum paint. As you can see, this is a very good model. You can use it for medical transcription. You can use it for handwriting recognition and you can do a lot more with this particular model and the best part is like you can run this model at scale.
Cost Management and Conclusion
Now let's say you have done everything. You can modify the code to pick up everything from your local drive and then also do it. But let's say you have done everything, for you to save your GPU cost, what you can do is you can just go back and then click here and then shut down your instance. So once you shut down your instance you're not going to pay for the compute. You're going to pay only for the storage and that's what I was talking about because the storage has the model downloaded so you can reuse it every time you want. And because they have given you $75, you can do like thousands of documents using this particular credit that you get only from this video.
I would like to thank Data Crunch for sponsoring this video. Again, sign up with the link in the YouTube description using the code on the screen. you will get $75 and then you can right away start using Deepseek OCR at scale. Soon Data Crunch will be under a different name. Stay tuned for more details. Whatever Deepseek has done with Deepseek OCR can bring wins much beyond than OCR. It could be a new way of dealing with LLM. Anyways, a huge thank you to Deepseek for open sourcing the model and also releasing the technical paper. See you in another video. Happy prompting.
