Introduction: Why LangChain?
Hi everyone. So in this video I'm going to be talking a little bit about an intro to LangChain. And actually, over the series of videos, I'm going to go quite in depth about what you can do with LangChain, all about how different parts of it work, and how you can build applications with this.
In this particular video, I'm going to just start off by talking about why we want LangChain and then jumping into a little bit about what it is and how we can get started with the basics of using it to access large language models.
So first up, large language models as apps. So large language models are great at generating something new from scratch or via sort of a conditional generation. So for example, basically all they're doing is giving us a probability distribution of the next token based on a set of tokens that we're putting into this. And it turns out this is great for making stories, for doing a whole bunch of different tasks. Only recently in the past year or so, people have worked out ways to fine-tune these with RLHF and a variety of other sort of add-ons in the sort of prompt space to get the best out of these kind of models. And it turns out once we can do that, we can get some really good results from these large language models.
The problem is that they still can't access the traditional software stack in a normal way. So if we want to build like an app with one of these things, we need some kind of way to interface between the large language model and the traditional software stack. If we want to build things like chat bots or something that uses search or external data, then often a large language model alone is not going to be enough.
A perfect example of this is the way that large language models are not storing state. So each call to the large language model is individual. If we want to have a conversation state going with the large language model, then each time that we call that model, we're going to have to pass back in, at least if not all of the conversation we've had, at least the key parts of that. And that raises a whole bunch of challenges. Or do we just pass it in verbatim, which might be fine if the conversation only goes for a few rounds, but if you've got a long conversation, you're going to probably have to summarize it or find other ways to do this because the large language models have a very finite span of tokens that we can pass into these things. Most of these things are going up to sort of 1024, 2048 tokens that we can pass in. Some of the new ones can go much bigger, and maybe this is something we'll see or get better over time, but it's still, you know, that's still a finite amount. And if you wanted to do something with large documents, with other things, you need ways to think about how do we interact with the language model.
What is LangChain and Why Use It?
So what is LangChain? So LangChain really is a tool or a framework that allows us to build fully featured apps that interact with the normal software stack. It allows us to manage our use of large language models and prompts and then how we deal with both the language models and prompts. It also allows us to integrate to the traditional software stack like APIs, tools, things like calculators, etc. and then also things like databases and various data sources that we want to look things up on.
So often people ask, okay, why not just make our own for this kind of thing? And while it can be easy to build something simple with prompting a large language model, once you start doing things that get a little bit more complicated with manipulating the prompt, with adding in memory, etc., things get complicated very quickly. So the open source community has really picked up on LangChain as being an exciting project. It's been developed really quickly by a large group of people and it's becoming a de facto standard for ways to build apps around these large language models.
The Fundamentals of Prompt Engineering and Templates
Everything starts with a prompt. In fact, everything in LangChain is built around prompts. They're key to everything that you're going to be doing with LangChain. So let's talk a little bit about what prompting is and the idea of prompt templates in LangChain.
So what is prompting? Basically, prompting is just getting the model to generate text conditioned on some input text. So these models are generally built to just keep predicting the next token, which is often going to be the next word, and assemble a text that way. So it's taking the input that you've given it, the prompt, and then continuing from that. The prompts themselves have a massive influence on the output, so much so that this has led to a whole field of what people are now calling prompt engineering.
Early on, prompts were very simple. You would have very simple questions that people would use. When was Marcus Aurelius emperor? And you would hope that the large language model had learned enough in its weights to basically answer that and just generate an output for the end. At the end of that, it would have some sort of end of a sentence token so it would stop generating too. And as the models have gotten better, they've gone from these kind of simple prompts to now much more complicated prompts. And now we see prompts that people are giving whole sort of context and a variety of things in the prompt to generate the output.
One of the papers that were key in this was the InstructGPT paper. And while this wasn't the first paper to use the reinforcement from human feedback technique, it certainly was one of the main ones in establishing that this was really good for fine-tuning your models to take prompts that could be much more useful than just very simple sort of prompts. And we can look at these kind of prompts from the InstructGPT paper. We can see that there are different types of prompts in there. The paper itself has got a whole bunch of different prompts in there. You can also see in this one here that we've got the idea of what we call few-shot template going on. So the idea of where we're not only just going to tell the prompt something, we're actually going to give it some examples as well, and then we're going to get it to generate a new answer or a new example there. So in LangChain, this is all handled by prompt templates.
So one of the key things is to sort of know what the different parts of prompts are, because what you will do is in LangChain you will build a prompt and you will inject different things in there. So an example of this would be, here we've got a simple sort of prompt that starts off with a context. It tells the model that you're a digital assistant who helps users create business names based on descriptions of businesses. You provide catchy, short, catchy names. Then it gives some examples, and the examples in a special way. So it's basically giving them in a way that the model will then learn to generate given this example. This is, this would be an example of a short catchy name. Then we can pass in the question or the task and then we can prime it for the output here. And this is something that is done a lot in LangChain, but it's also done in a lot of the other prompting systems that are out there. So you'll see things like this where here it's basically saying, here's a message to me, injecting in some information. Here are some bullet points for a reply, again, injecting some information and then prompting the model to give us a response with this.
So this can be done for a whole bunch of different tasks. It can be done for classification, it can be done for a variety of different things.
Getting Started: Colab Setup and Basic LLM Usage
Alright, let's jump into the code and have a look at how LangChain handles both normal prompts and few-shot learning prompts where we're going to be giving it examples as well.
Okay, so let's jump into the code. So here you can see I've basically got a Colab setup. The Colab will be in the description so you can just play with this yourself. First off, we're just installing some packages. So I'm installing the OpenAI package so that we can use the GPT-3 DaVinci models. We're installing the LangChain package itself. We're also installing the Hugging Face Hub here. So this basically allows us to ping the endpoints for OpenAI with the OpenAI one and also ping the endpoints for the Hugging Face Hub here as well. So you will need to go in, get your keys for these, and then once you've basically got a token for each of these set up, you can just drop them in there and then this will basically work for you. Alright.
So if I come down here, first off, all we're going to do is basically load a language model and prompt it with some text. So you can see here in this first one, I'm just bringing in the OpenAI model. The OpenAI model that I'm going to use here is the text-davinci-003, which at the time of recording is the latest one. I'm going to use quite a high temperature so we get a random output. I'm also going to set it to just give us a max tokens of 256. In this case, it's probably overkill, but this is how you can determine how much output you want from the model.
So now we've basically got our prompt here of, "Why did the chicken cross the road?" and all we're going to do is basically just pass that prompt into the large language model and then print that out and see what the output that we get is. And we can see that, okay, it came back with, you know, "To get to the other side." Obviously not massively original. We could even run this again, it may give us something different, although that answer is probably, you know, pretty, pretty common to get there. If we change this to a duck or something, we may even get something a little bit different. Okay, we're still getting the same thing. By playing around with this though, you can then start to see like, okay, how much effect your prompt is going to have.
So that's using the OpenAI model. If you wanted to use one of the free Hugging Face models, we can basically do the same thing. And here I'm just setting up a Hugging Face version of the Google Flan-T5-XL model. So this is not the biggest one, the biggest one is XXL, but I'm not sure they're actually supporting that on their endpoints at the moment unless you're paying for it. So this one is definitely nowhere near as big a model as the DaVinci, OpenAI DaVinci model, but you'll see that we'll still get some sort of coherent response from this. And you'll notice that this one, that this response is different. If we run it again, okay, we're still getting the same sort of thing, but we're getting different responses from each of these. But it's basically doing the same thing. We're picking a large language model and getting a response back from it.
Building an LLMChain with PromptTemplates
So that is just a sort of raw prompt. If we want to start actually setting up some prompt templates, this is where LangChain starts to become really useful. So here we can sort of say, okay, we want to make a little app that our app is going to take in a description of a restaurant and create a name for that restaurant. So this is my prompt. My prompt is, "I want you to act as a naming consultant for new restaurants. Return a list of restaurant names. It should be short, catchy, and easy to remember. It should relate to the type of restaurant you are naming. What are some good names for a restaurant that is..." and then we're going to pass in a restaurant description here. And this is where LangChain will basically enable us to get our description into the prompt here. So if we had a user typing something in, they wouldn't actually see this prompt, right? They would actually just type in what their description was, and then using this prompt, we would ping the back-end model and get the output for this.
So to do this, we set up a prompt template. And the prompt template, so you'll notice this is very similar to an f-string in Python here. The prompt template, we basically just work out what the template is, which in this case is going to be our restaurant template, and what are we going to inject? What are going to be the input variables here? So this is going to be the restaurant description here. So we can just go through, set that up, and now we've created this prompt template. And, you know, here's actually just creating it again.
So now if we want to, you know, try something out, we can basically look and see, okay, what's this going to do? So one of the ways we can do it without pinging the actual large language model itself, we can just basically say, "All right, for this prompt template, format it with the restaurant_description equals..." and we're going to go for this description here. So you'll see that when we do this, we're going to get the whole prompt plus our restaurant description that's been injected into that prompt. So our restaurant description was, "A Greek place that serves fresh lamb souvlakis and other Greek food" is what we want there. And you can see that we're getting the full thing there.
So now if we basically take this, we can, and you see I've done a few of these up here so we'll play around with a few of these. Now I can basically say, right, okay, the LangChain, I'm going to set up a chain. Now, I'll talk about a lot more about what chains are in the next video, but this is basically just a very simple large language model chain that we're going to prompt it and it's going to give us, you know, it's going to return the output of the model here. So our chain is going to be a large language model chain. We're going to pass in the language model that we're using, in this case that's the OpenAI model, and we're going to get back a, you know, we're going to pass in our prompt template that we've created, and then we're going to inject into that this description. So we'll start off with just the description, the basic description. It's for the Greek restaurant. So let's see when we run this, what are we getting back from OpenAI that we can use?
And we can see sure enough, it's given us 10 restaurant names that we could go for: The Olive Tree, maybe. Yeah, I guess, you know, Athena's Kitchen, Aegean Delights, Souvlaki Grill. All of these are related to our description up here of the Greek place.
So if we wanted to try something like, you know, a burger place that is themed with baseball memorabilia. So now all I'm doing here is just injecting in the different restaurant description into the exact same prompt template that we had before, right? We haven't changed the prompt template, and we can see now, sure enough, we've now got things related to burgers and to baseball that we expected here.
All right, for a third one. So the third one here was, I was trying to basically get it to sort of say something like a cafe that has live hard rock music and memorabilia. Some may say even Hard Rock Cafe, but let's see, will it actually, you know, generate something like that? It's generating things sort of around this and they're certainly, you know, appropriate names for what it is that we're going for here.
All right, so that should give you a sense of like the basics of just setting up a standard prompt template and setting up your standard language model to get started.
Using Few-Shot Prompt Templates for Advanced Control
The next level for doing this, and I've taken this from the LangChain examples, is what we call a few-shot prompt template. So here we're actually going to pass in some examples, and it's not drastically different than what we've done before, right? We've already got it, you know, so that we're going to set up a prompt template. And in this case, we're going to basically, we're going to basically give it a word and we want it to return the antonym for that word. So we've given some examples here of like the word 'happy' it should return 'sad', the word 'tall' it should return 'short', etc.
So now, because we're passing in multiple things, we're passing in, you know, some examples and stuff as well in here. What we're going to do is we're going to have our prompt template, alright, and we're going to have a prefix and a suffix to this. So we're now just sort of building up the parts of the prompt so that it all goes together with this. So we've got our, you know, our standard sort of example of the, you know, where we're setting up the template. We've then got our few-shot templates where we're going to pass in examples, we're going to start, you know, a prefix for this, we're going to start with a suffix for this.
So you can see that what it's going to do is that it's going to basically create a prompt, and we should be able to actually, you know, we should be able to see this by by running this. We can actually see that, okay, this is the whole prompt that it's creating, right? It's created, "Give the antonym of every input," that was our prefix that we had there. Then it's basically giving, you know, some examples of these, right? These are the examples that we set up here. And then finally, we're passing in the word that we want to do this for. We're going to basically return this back, right? So for what we've got here, now if we come down and run this, we can basically do the exact same thing that we did with the first prompt. We can just pass in here our input is going to be 'big', and I can just basically run this and we're getting back 'small', right, which is the antonym.
In this case, the idea here though, is that we're not actually seeing the whole prompt. So you often won't want to show your users your whole prompt for something like this. So this is a simple way of, you know, making prompts with the prompt templates. You've got your standard prompt template, and then you've got your few-shot prompt template as well. Remember the key thing with the few-shot one is going to be the examples, so you could actually have a whole series of examples in here. All right, and that's the example is just showing an example of what the input is and then what's going to be the corresponding output that you would expect for that particular thing.
Conclusion and Next Steps
Okay, in the next video, we're going to be looking at tools and chains in LangChain and how they come together to start being able to make the building block of apps. So if you're interested in finding out more about this, please subscribe to get notified of the latest videos, etc. See you in the next video. Bye for now.
