RAG Tutorial 2025 #4: Document Retrieval Implementation with LangChain

Introduction to the Retrieval Pipeline

Hello guys, welcome to this video where we are going to be coding out this retrieval pipeline right here. Okay, so in the previous section we saw how we can implement the ingestion pipeline and we did successfully implement it as well. So if you remember we actually loaded all of the documents and then we split it and then we stored it in the vector store. Okay, so all of the embeddings live in this particular directory right here. So right now we are going to go ahead and implement the retrieval pipeline.

So to give you a quick refresher, so we have the user query. We're embedding that and then we are going ahead and creating a component called retriever. And this retriever is going to go and find similar chunks that match this particular original user query's embedding and it is going to use an algorithm called cosine similarity. Okay, so don't get confused. This is something that we will cover in the next video. But it is going to find the most similar, you know, cosine similarity scores are highest. Okay. So it's going to go ahead and shortlist those first five or 10 or whatever it is that we specify. It's going to shortlist those first five chunks and then we are going to get it. And finally we are going to take the query as well as the original chunk English passages and then we are going to send it to the LLM. Okay. So I hope that makes sense. So right here you can see I've already gone ahead and created a file called retrieval pipeline and you can see this is all the code there is. Okay so these are all some synthetic questions that we are going to be testing the system on but actually the code is going to be very minimal. Okay so this is all the code.

Code Walkthrough: Setting Up the Retriever

So as you can see we have imported Chroma and the embedding model. Okay so we have OpenAI embeddings which we imported and then we are loading up the environment variables because we have the open AI key right here. Okay and Then this is going to be the persistent directory right? So this is where our embeddings live we would want access to this particular, you know, data inside of this particular file. And also I'm initializing the embedding model right. We are also, note guys that I'm going to be using the same model right here that I have used here. Okay so we cannot use something different or we cannot change the dimensions. All right.

So the next thing is that I'm going to recreate the vector store okay so If you remember in the previous file, we created all this data, right? So I'm just pointing the new vector store to this particular data. So we're just basically recreating it. So I'm providing the persistent directory and then I'm providing the embedding model and then I am specifying which algorithm we are going to be using or which algorithm that this retriever needs to use in order to do the matching in this case. Okay, always remember whenever you're dealing with RAGs, always use cosine similarity. Okay, so why we'll talk about that in depth in the next video. But we are going to always use cosine similarity right here. And this is what this particular line means. All right. So we have the instance of the DB right here. And now we can use this DB and then create the retriever component.

Configuring the Retriever

So I'm just going to say okay DB as retriever. And I'm going to set the K value to three. So what this means is that this retriever is going to retrieve the top three chunks with the highest similarity scores to the user's query embedding. Okay. So that is exactly what we are mentioning right here. So we can also configure the retriever in a slightly different way as well. Okay.

So what we are doing here is going to be slightly different. So we still have the the number of chunks K to be five or three or whatever but we are also setting a score threshold okay to be 0.3. So this is something that we will cover soon exactly what it means but on a very high level you can think of the scores, okay, so I told you the cosine similarity scores that gets calculated right for every single chunk, so those scores range from 0 to one. So zero if we call it as zero it means that you know there's nothing similar there, don't even bother to shortlist it. if it is one it means it's a strong match, it's like an equal match. So we're just saying if the score threshold needs to be at least 0.3 only then you can shortlist a particular chunk. So if we set it to be very high it is very likely that you might not get any chunks at all. So this is something that we will have to arrive at through trial and error but for now to keep it very simple I'm just going to use this particular line right here. And then finally we are going to say retriever.invoke and then pass in the user query. Okay, so let's look at the user query. So which island does SpaceX lease for its launches in the Pacific. Okay, so this is something that you can imagine the Wikipedia page of SpaceX is going to have the data. Okay, we don't know where it is going to be. We cannot possibly go through every single line and find out. That is why we are using RAGs.

First Test: Tesla Roadster Production Year

So we are going to invoke the retriever. Okay. So this is going to be like a keyword that you will come across a lot. So whenever you have a component like this or whenever you have a chat model instance, you always, you know, call it by saying invoke. So retriever.invoke and we're passing this particular user query. And this should give you the top five chunks. All right guys, so it is time to finally run this file and check if the retriever actually does a good job, if it can actually retrieve relevant chunks from the vector database. So also note that I've increased the number of chunks retrieved from three to five. So let us go ahead and run it.

So the query is in what year did Tesla begin production of the roadster. So if I run it okay so there's a couple of things happening right. So the users this thing was user question was getting converted to embedding and then the retriever goes off and fetches the top five chunks. So right here you can actually see document four, document five, document three. So these are all the chunks, document 2 and document one. And finally we have the user query. So let us see if the chunks actually have the answer to the user's question. So in what year did Tesla begin production of the Roadster? Tesla began production of the Roadster in 2008. It it has the answer, right? So let's look at the second document. Uh the original Tesla Roadster was a two-seater sports car evolved from Lotus this thing. Okay. It was produced from 2008. So this also has the answer. It started from 2008. And then let's look at the third document. Okay. So the third third document also talks about the Roadster. The fourth document also talks about the first Tesla Roadster. Right? It does not have the answer but still the retriever fetched it because it is somewhat similar to the user's question. And then document five. Okay. So this also talks about roadster and uh you know this also has information. Maybe it does not have the exact information or the exact answer for this user question. But still the retriever fetched it because it thought that okay this could be somewhat similar. Okay, this could have the answer. But you can see the first three or two documents actually has the answer to the user's question. And one more thing guys all these different documents that have been fetched this is not being generated by LLM. There is no LLM involved here. It is pure embeddings and similarity matching.

Validating Retrieval Quality with an LLM

So if you uh if you actually like you know go ahead search for this in the tesla.txt, right? Okay. So, you can actually see Tesla began production of the Roadster. Tesla began production of the Roadster, right? So, this entire thing, apparently, this entire thing was one chunk. Okay. So, this entire thing was one chunk and then this particular chunk was retrieved as the first document. This is pretty powerful if you think about it. Okay?

And usually when I'm building out these very simple prototypes, what I tend to do is to check if the chunks that are retrieved are good quality, if it answers the user's question. To check it, what I do is I don't go through every single chunk and test it, okay? To tweak the rack system. Instead, what I do is I just copy this entire thing and then I ask chart CPT or claude. Okay, I tell it, okay, I'm testing a rack system. This is the user's query. These are the context. just tell me if the rack system is good, if it's working good. So, this is the prompt right here. I'm just saying, okay, I'm testing my rack system. I'll give you the following information, the user query and the context. I want you to tell me if the context is good and has the answer to the user's question in one or two lines. If you understand this, just say okay. So, let's go ahead and paste this and let's see what Claude has to say. The context is excellent and directly answers the question, right? Document one explicitly states Tesla began production of the Roadster in 2008. So, let me go ahead and try out a different question of a different company, right?

Testing with Nvidia and Microsoft Queries

So, I'm going to say, what was Nvidia's first graphics accelerator called? So, let's replace this query right here. And let's go ahead and run it.

Okay. So, we've got again we've got, you know, document one and right here. Uh, I'm not really sure I'm not really able to uh because there's a lot of information. So, I'm a little lazy. So I'm going to make the use of claude to tell me if the user query is actually I mean if the context retrieved are actually high quality. So I'm going to post it right here. Okay. So you can see we've got the user query and we've got the entire context here as well with document one, document two, document three. Okay. So again the context is excellent and directly answers the question. Document one clearly states Nvidia's first graphics accelerator the NV1. Right? So that is exactly what the first context here talks about as well. Right?

I have also got a lot more questions for you guys to test it out. Okay. So these are called synthetic questions and uh you can test it out by yourself. All this code is already available in the repository. Okay. So I can test it with okay how much Microsoft pay to acquire GitHub. So let's replace this query right here and let's try to test this out again. All right. So we've got five different chunks coming all the way up. You can see, you know, it's got something about GitHub acquisition on January, right? Okay, this actually has the question. I don't even have to go ask Claude because I already see the answer right here.

Conclusion and Next Steps

All right, guys. So that is it for the retrieval pipeline. But in the next video, we are going to dive deep into, you know, how exactly does the similarity matching happen. It's called cosign similarity formula. And that is exactly what we're going to be diving deep into in the next video. So I'll see you there.