As I get older I realize money is not everything, but it's kind of almost everything. So every year or every other year, I download all my bank transactions and review my incomes and expenses. The other day I came across someone who made this income and expense breakdown and I feel really inspired to do the same.
Usually the most tricky thing in the process is to classify the expenses from my buying transactions into appropriate categories. A lot of times I just use my manual labor or some low-tech ways to do that. This year I decided to ask ChatGPT to crunch the number for me and maybe tell me when I can retire, only then to realize that I can't just upload my bank statements to the ChatGPT website. It's so sensitive information about places I've been to, shops I visited, how much I spent on buying secret items, and other personal data. Although using OpenAI APIs may help, still the data sent via the API is stored by OpenAI for a duration of up to 30 days.
So eventually I decide to download and run an open-source large language model locally on my laptop, and the best thing yet, they are free.
In this project video, we do a few exciting things. First, we'll learn to install and run an LLM, for example, Llama 2, locally on our laptop. Then, we use the LLM to classify all the expenses in my bank statement into categories such as groceries, rent, travel, and so on. We then analyze the data in Python and create some visualizations to show the main insights. I shared all the codes on GitHub, so you can check out the GitHub repo in the video description.
This video is sponsored by Coursera. Coursera is running a discount with $200 off the Coursera Plus annual subscription if you sign up. This is half the regular price for the whole year. With this subscription, you can get access to tons of courses and certificates in data analytics, including the Google Data Analytics and the Google Advanced Data Analytics certificates. These certificates teach you all the fundamentals you need as a beginner in data analytics, so check out the offer below.
How Local LLMs Work
There are a few different ways to run a large language model locally on your laptop, which means you can run the LLM without internet connection, without a third-party service like an API service or website like ChatGPT. As you can imagine, this is secure if you want to protect your personal data, and it's free.
There are now a few different frameworks developed to help us run an open-source language model locally on our own device. Some popular frameworks are Llama.cpp, GPT4All, and Ollama. So you might be wondering, why the heck do we even need these frameworks? Remember how large language models are trained? They are basically the result of taking a huge amount of internet data and training a large neural network on it. And the model that comes out is basically a zip file with a bunch of numbers that represents the weight of all the parameters in the neural network. This model file can be quite large, depending on how many parameters the model has. So frameworks like Ollama and Llama.cpp basically try to do two things. The first thing is quantization. It tries to reduce the memory footprint of the raw model weights. And the second thing is it makes it more efficient for us, the consumer, to use the models.
Installing and Running Ollama with Mistral
If you have a Mac or Linux machine, I'd highly recommend installing Ollama. It's super simple and I'll show you in a bit. If you have a Windows machine, you can also run Ollama through the Docker desktop. Okay, now let's go to the Ollama website. And here you can download Ollama, which is available for Mac OS and Linux, and Windows will be coming soon.
Also have a quick look at the list of models that are available. So here we have Llama 2, Mistral, we have a bunch of other code models, and so there's a lot of options here for you to try out. And if we click on any of them, we can see the description, also how to use the API, what is the memory requirement, so how much RAM you need in your laptop in order to run these models. So let's download Ollama and install it. It's very straightforward, just like installing any app on your laptop.
So once we've installed Ollama, we can start using it through our terminal. In order to install a language model locally through AMA, we just need to run the command ollama pull and then specify the model that you want to install. So for example, I will install again Mistral. It's pretty fast because I've already installed it last time. And you can see here the model is around 4 GB. So in order to use a model through the terminal, we just need to do ollama run mistral. And here we can start typing our message or our prompt. So let's say, "hello." And the model replied with, "Hello there, how can I help you today?" Okay, so that's a lot of things. Okay, so I can ask something a little bit stupid. What's 2 + 2? Okay, it comes back with quite an elaborate answer.
Now let's try another question. I want to see if it actually can do math properly. Let me ask what's this times this. Oh wow, that's amazing. Okay, it even tries to teach me how to find the product of two large numbers step by step. Now let me check if this is actually the correct answer. So let me go to Google. Okay, so what I have here is around 426 billion. However, if I look at the result here, it's actually not correct. 45 millions. You can check this result using a calculator to make sure it's correct. Well, I've checked it and it's not correct. I kind of have to say I'm impressed, but in terms of basic arithmetic, I think large language models out of the box are not probably not the best option.
Comparing Mistral and Llama 2 for Classification
All right, the next thing we want to test, and that is very important for this project, is whether the Mistral model can properly classify all the different expenses in my bank statements into different categories. So I just try out a prompt here to see how it performed. Now I ask, "Can you add an appropriate category to the following expenses?" For example, Spotify as entertainment, Beta Boulders as Sports. And here I just give a list of transactions as shown in my banking statements. So let's see what it comes up with.
Okay, so it replies with three categories. It's roughly correct, I would say it's reasonable, but it missed one transaction here, which is Bistro Bar Amsterdam. And also, it doesn't really reply in the format that I wanted. So the transaction together with the category separated by this hyphen. So I feel like Mistral, it doesn't really do the task as I expected it to do. So let's try another model, which is Llama 2. Let's exit this model and we will do ollama run llama2. But if you haven't installed Llama 2, you can do ollama pull llama2 and the Llama 2 model will be installed locally in your computer. Now we can start using the Llama 2 model.
Now, let's ask the same question as we asked before. And it does it pretty well. It gives me a list of the expenses together with the categories. Although the first two are my own example, but it does give me the correct format for the answers. And so each of this transaction has a category added next to it, separated by a hyphen. So I'm pretty happy with Llama 2 and it actually understands the task. Although if you keep asking the same question to these language models multiple times, these models may come back with different answers each time. And so there's definitely a certain level of randomness in the responses.
Creating a Custom LLM with a Modelfile
If you want to take a step further to customize these language models to your specific use case, you can do that by specifying a model file. And a model file is basically the blueprint to create and share language models with Ollama. So you can specify the base model you want to use, and also you can set parameters like the temperature for the model. Now let's go back to the terminal and exit this model. Let's clear the terminal and I'll go ahead and create a model file with Nano, and I'll name this model file as expense-analyzer. And we'll go into the text editor.
Okay, let's first specify the base model as Llama 2, so FROM llama2. And next, we set the temperature parameter as, let's say, 0.8. Temperature closer to one is more creative and the lower the temperature, the more coherent and less creative the model behaves. And further, let's also specify the custom system message. So my system prompt is quite basic: "If you're a financial assistant, you have to classify expense and income from buying transactions." Okay, let's save this file by Control-X, yes, enter.
Now that we have the model file set up, we can use this model file to create a custom model. We can do this by ollama create, we specify the custom model name, -f, and then specify the name of the model file. So that is expense-analyzer. So now if we run this comment, basically what it's doing under the hood is that Ollama will pass through this text file, this model file expense-analyzer, passing through all the parameters and the custom message that we put in, and then customize all these different layers in the base model, which is Llama 2. And now we can start using this custom Llama 2 model that we just created by ollama run expense-analyzer.
Classifying Transactions in Python with LangChain
I also forgot to mention that we can also now look at the model list available by ollama list. And you can also see that now we have the expense-analyzer Llama 2 model available in our list.
Now, interacting with these local LLMs through the terminal is also fine, but I find a more convenient way to interact with these models is through the Python environment, and more specifically through Jupyter Notebook. Now let's create a project folder, and I'll move my bank transaction data inside this folder, and I'll just start up Visual Studio Code from this folder. In order to access these language models from Ollama with Python, we need to install the LangChain community library. And so if you haven't done so, we can use pip install. And now we can access all the language models we have installed through Ollama by specifying the name of the model with this Ollama method. For example, the first man on the moon was... and after a few seconds, we get back the completion of this sentence. So that means our model is up and running. That is a good sign.
Now let's move on to reading the transaction data that we have and take a look at it. You can see that here in the name description column, we have all the transactions that we want to classify. We have the indicator whether it is expense or income, and we also have the amount in Euros of these transactions. Now I have anonymized a lot of these transactions, so this is not my real income and real expenses.
Now we need to find a way to insert all these different expenses into our prompt, right? Let's get all the unique transactions from our data. You can see that this is quite a big list, and this may exceed the token limit that the large language model has. So if we try to insert this huge list into our prompt, there's a risk that the model comes back with a completely nonsensical answer or incomplete answer because your question is already taking too many tokens. So after many trial and errors, I found that around 30 transactions would give the most optimal response. So here you can see an example response with 30 transactions. With this approach, we are going to create a for loop to basically loop through all the 300 transactions here in our bank statements, and we take in 30 transactions at a time. And so a handy way to handle this for loop is to get an index list. So this index list is basically giving us a sequence of all the index from zero until the last item, hopping 30 items at a time. And with this, we can also conveniently handle the last group as well, which might not be 30 items but maybe less.
Parsing and Validating LLM Output
Now let's initialize a data frame to store all the unique transactions and their corresponding categories. And with this for loop, I'll call a custom function that I created, and this function takes the names of the transactions, of the LLM that we are using, and the only thing we need to do is to properly parse the output from the language model into a format that we can work with. So if we look at this output, we definitely only want to keep these lines, right? And we don't want the rest, like, "Certainly, here's the categories." But we don't need to worry about this for now, because at the end we can always remove all the rows that have categories being none.
The complication is that sometimes the language model might use a different format for the answer. So we had to use some kind of like validator for the output in order to make sure that the output actually is in the format we want. For example, in the response, we have the hyphen in between the transaction and the category. So one handy Python library for this is Pydantic. So the idea is that after getting the response, we will run it through a validation check. If the validation fails, we will rerun the language model to get another response until we get the right output. If you're interested in how to do this with Pydantic, you can check out the code in the GitHub repo link in the description.
Okay, now let me run this for loop. And I also print out the transaction names and also the output by the models. Also notice that when I'm running this for loop, for some reason, if I want to stop it, I really cannot stop it. So if for some reason we want to interrupt this process, we can go to the terminal and do ps -a and in the back end all the processes will be stopped. Okay, after a while, if everything goes well, we should get back the data frame with all the transaction with the categories that Llama 2 has categorized for us. I'll just save the CSV here. Now let's open the CSV file and quickly look at the output. Overall I'm quite happy with the categories that Llama 2 came up with. However, some categories may be quite similar to each other but not completely the same, and I want to group them together. So I just quickly group these categories by hand.
Building a Finance Dashboard with Plotly
Now the last step is to clean up a little bit this data frame and then we can go ahead and merge this data frame into the main transaction data frame using the transaction name. So after this, we should have a data frame with all information, the transactions and the categories.
So that was exciting, but the next part of the project is even more exciting. We create a personal finance dashboard based on the transaction data that we just obtained. The idea for this dashboard is we want to show the income breakdown and the expense breakdown for both years 2022 and 2023. And at the bottom, I also want to show how much I earn and how much I spend per month in each year. So for creating interactive visualization, I love using Plotly Express. For the creation of the dashboard, we also use Panel. Panel is an awesome library for creating data dashboards very easily and very fast.
Now let's read in our transaction data, and for the income, I don't have a lot of income sources, so I can just use the name of the income as the category. Firstly, we want to make the pie chart to show the income and expense breakdown. I create a function here to basically take the data and select for the year and whether we are looking at the expense or the income in the data set. And similarly, I also create a function to make bar charts that basically gives us a histogram of the income or expense per month over the year. And you can see that it's super nice with Plotly Express that you can also hover on the different sections on the graph and see the data label.
Assembling the Dashboard with Panel
Now we're almost done. We just need to combine and organize all these charts on our dashboard. So here I just create a panel layout with the tabs which consist of two tabs. We have the 2022 and for the 2023 tab, we have exactly the same charts and graphs but for 2023 data. And to show all this nicely on the dashboard, we will use a template from Panel called FastListTemplate. We can specify the header of our dashboard, we have the sidebar, which is some information and also you can put in different elements or pictures, etc. And in the main here, we will put in the tabs that we just created. And if we do template.show, we can see that this is our dashboard.
And here's my income and expense breakdown, and here is how much I earn and how much I spend per month in each year. Here you can see that I do save a little bit more in 2022 than in 2023, which is not really a good sign. But if you're into personal finance and things like that, you may notice that this overview is probably not complete because we also have assets. So if you transfer your money to an investment account, or if you pay the mortgage towards your house, then part of that money is also your asset as well, and not only your expense.
Final Thoughts and Next Steps
Although as you can see, I can't retire anytime soon, I hope this inspires you to do your own projects and experiment with open-source language models. I think in the future, it will be a norm for us to be able to use and run large language models locally on laptops and other personal devices. If you enjoyed this project, also check out other data science projects on my channel. Thank you for watching. Bye-bye.
