Made withFlowiseClip

Prompt Chaining with GPT-4o and Flowise AI (Tutorial)

E

Elvis Saravia

Creator on the Flowise channel

Watch on YouTube ↗

35K viewsSep 19, 202414:04 runtime

Learn how to implement prompt chaining with GPT-4o and the low-code Flowise AI platform. This tutorial guides you through building a multi-step chat flow from scratch. You'll see how to configure an initial LLM chain to extract specific information from a document, and then…

Chapters

#flowise#clip#aillm

Overview of Prompt Chaining

Prompt Chaining is the technique of breaking a complex task into a sequence of smaller, more manageable sub-tasks. Each sub-task is handled by its own prompt, and the output of one prompt can be used as the input for the next one in the chain. This modular approach simplifies development, improves accuracy, and allows for more sophisticated and customized AI responses.

Building the Chat Flow in Flowise

The tutorial demonstrates a two-step prompt chain built in Flowise AI to answer questions based on a document.

Tip
This example uses a single ChatOpenAI model (GPT-4o) for both steps, but you could use different models for each chain. For instance, a powerful model for analysis and a faster, cheaper model for final formatting.

Chain 1: The Extraction Step

The first chain is responsible for extracting relevant information from a source document.

System Message: Instructs the model to act as a helpful assistant. Its specific task is to extract quotes relevant to a user's question from a document provided between ### delimiters. The output must be wrapped in <code> tags.
Human Message: Contains the static source document and a dynamic {user_question} variable. This variable is populated from the user's input in the Flowise chat interface.
Connecting Input: The {user_question} variable is linked to the chat input via the Format Prompt Values setting on the prompt template node.

Chain 2: The Response Refinement Step

The second chain takes the raw extracted quotes from the first chain and formats them into a polished, user-friendly answer.

System Message: Instructs the model to compose a short, accurate, and friendly answer based on the extracted_codes and the original document.
Human Message: Contains a placeholder for the {extracted_codes} from the first chain, the original document again for context, and the original user question.

Connecting the Chains

Linking the two chains is the core of the technique in Flowise:

The first LLM Chain (named extraction) is configured to have an Output of Output Prediction instead of LLM Chain.
This Output Prediction from the extraction chain becomes available as a variable.
In the second prompt template, navigate to Format Prompt Values.
Create a new value with the key extracted_codes (matching the placeholder in the human message).
Set its value to the extraction output prediction from the dropdown list.

Important
This connection passes the raw output of the first chain (including the <code> tags which may be hidden in the UI but are present in the data) as input to the second chain.

Key Takeaways

Task Decomposition: Breaking down problems (e.g., extract then format) makes prompting easier and more reliable.
Improved Iteration: With chains, you can isolate and refine one part of the process (like the final response tone) without altering the other parts (like the data extraction logic).
Customization: Chaining enables highly customized and personalized responses that would be difficult to achieve with a single, monolithic prompt.

Setting Up the Flowise Canvas

In this demo, we're going to be using Flowise AI to compose a chat flow that applies the concept of prompt chaining. You need to go to Flowise and again, you should be running this locally already. And what we're going to do is we're going to add a new chat flow. So click on add new and you will see this blank canvas.

The first thing we're going to do here is we are going to need a chat LLM. So same as before, we're going to be using this Chat LLM, and the one that we'll be using is the one from OpenAI. So I'm going to drag this right here and it's using again, Chat OpenAI. You can search it here or you can find it under chat models. I'm going to set my credential, and then I'm going to set the model that I'm going to use. So I'm going to use the latest because I think these are more advanced tasks that I'll be performing for this particular prompt chaining example, and everything else will be left as default.

As you recall, the next component is our prompt template. So we're going to go to prompts right here and we're going to use a chat prompt template. This time, I'm going to drag it here. I'm going to put them close to each other.

Crafting the Extraction Prompt

So this shot prompt template is asking for a system message and a human message, and both of these are required because this particular test that I'm trying to perform is a bit more complex. We're going to be using both of these messages.

So for our system message, I'm going to paste this here. This is a system message that we will be using, and I'll paste all of these messages and prompt templates down below so you have access to them. What we're trying to do with this particular prompt chaining example is that we want to build this helpful assistant and the task is to help answer a question given in a document. The first step is to extract codes relevant to the question from the document provided, and document will be provided using these delimiters, these pound signs here. "Please output the list of codes and surround them with these code tags. Respond with 'no relevant codes found' if no relevant codes were found."

In summary, what we're trying to do is we are going to give the model a document. You can pick any document, we're going to provide you text for that document, but you can try it with any type of document that you may have around. And for this first step of this prompt chain, the idea is to extract codes that'll be useful to answer questions that users will be asking. This is why we're using Flowise, because we don't need to build that chat interface. This is already provided for us and we can directly test our prompt chain.

So I'm going to save this here and then down here I will provide the document. So document is the input and I'm also going to configure where the question is going to go. So I've pasted here my human message. I'm going to expand this a bit and you can see here that I'm using the delimiters and I've just chosen this document here, which you can use or you can use any document that you want. And this one is just talking about some prompting techniques. So I will ask a question about these prompting techniques, but this could be again, any document that you have lying around. And then finally here at the bottom we have this question and we have this special variable here that we're using.

Connecting the User Input Variable

So as part of Flowise, what we can do is we can pass information as part of these prompt templates. And this particular one will be identified by Flowise as some special variable, and you may be asking what goes into this variable? Well, since this is a human message, what we want to pass here is the actual question that the user is asking. So I need to make that connection. And the actual question that the user is asking is the question that would go right here. So this question that the user will type, but I just need to tie it up into this template. So the way you do that is you go to Format Prompt Values down here, and then you can see here that it says user question. So you can see it was populated for you. If it's not, you need to create it yourself. You can just go in this plus button. So I will do that because sometimes Flowise does that. So I'm going to cancel this one and I'm going to create that one. Okay, go to the plus button and then add this key name, user_question. And then it's not for now, but I need to give it a value. So I'm going to give it a value and I'm going to pass the question. So it's already there, right? So I just can select that variable right away. So this is the user's question from the chat box, which I just showed you. I can select that and that ties in those pieces of information. And now the prompt template has access to whatever question I'm asking through the chat. So that saves automatically. I can double check, it's right there. And again, this user question right here, you can give it whatever name you want, make it something that is representative, but you can give it whatever name you want.

Building and Testing the First LLM Chain

And then finally, to complete this first part of this prompt chain, what I can do is I can use the LLM chain. I'm going to go to chains, and I have this LLM chain here. I'm just drag it here and then this is asking me for a language model. I will connect my language model here and I will connect my chat prompt template. I can connect that right there. And now I can save this, and I'm going to save this as prompt_chaining_example, you can give it whatever name you want. All right, so that's saved.

Now I can test. I can also give this chain name a name just to make it again, very easy to understand. I'm going to call this extraction because it is extracting quotes that will help answer that user question. I'll save that. I can go to chat now and I can ask a question.

So the question the user is going to ask based on those documents is something like this: What are the prompting techniques mentioned in the document? There are some prompting techniques mentioned here, so it should be able to collect them. I can hit enter. As you can see here, that the model responds with the prompt methods mentioned in the document, and it's only mentioning those prompting methods, which is nice because that is exactly what it should do, and that is what we tasked the model to do. Okay.

One thing you don't see here is these <code> tags. It turns out that with Flowise, it doesn't show it because when it's displaying the message here, it does some reformatting so it doesn't show the raw code tags. But if you check in your terminal, you also get the logs and you will see that it shows the codes there in the log itself. So that's pretty neat. You can always look at the logs and it will show you the raw output. So I know it's there. It's just not displaying here.

So once we have that first part, then the second part of this prompt chain is going to take the extracted outputs of this first chain, which is this result here, and then prepare a nice response to the user. This is not really helpful as a response to the user. It's not very user-friendly. So we want to refine the output that we're getting from this first chain of this prompt chain. So now we're going to do that.

And this is why I was saying prompt chaining is very useful for customizing and personalizing those responses because now you're separating these different tasks, you're splitting the task, and you're paying closer attention to how the final output are going to look. That's the benefit that you get. All right, so I'm going to clone this just to make it a lot easier here. I'm going to clone, you have that clone option, and then I'm going to clone this one because I need another chain here. And once I've done that, then pay very close attention to what I'm going to do here.

So this LLM chain has LLM chain as the output, but it also has an output prediction as the output because I was initially using this first chain as the final output, I needed to set that as LLM chain. So in order for this to be a first component of this entire chat flow, I will need to convert this to an output prediction, and basically what I'm telling it is that I will connect it to another LLM chain, which is going to be the second one that I'm designing. So I'm going to make that connection one time here. You can see I made that connection, and what I'm going to do is I'm going to connect this one to prompt. And then I'm going to connect this model here to the LLM chain.

Configuring the Response Generation Prompt

Since I'm using the same model for both these parts, then I could just connect it like this. But I could also, if I wanted, which makes a lot of sense in a lot of applications, I could just use another model here. I don't have to use this model. I could use a different model, especially because this last part is going to be about refining the response to the user. So I don't really need to use a very advanced model for that. I could use a very simple model. I'm going to keep the example simple. I'm just going to use one model because I really want to focus on the concept itself.

So the LLM chain here, the second one here, I'm going to name this response. Okay, I'm going to save that. And then here, I need to edit this. So I've cloned it, so it's giving me the same system message and the same human message, but I need to change this part. And so what we're trying to do here is we're trying to take the information that we got from that first step, this entire thing, and we need to prepare a response for the user. So here is the system message that I'm going to use here. I'm going to delete this one and I'm going to paste this is the one that I'm going to use for the second part, which is going to prepare the final response for the user. "So given a set of relevant codes delimited by <code>, this is the answer that I got from the first step, extracted from a document, and the original documents delimited by this." So this is assuming that I will pass the original document as well, just in case the model needs that additional context, right? "Please compose an answer to the question." So this will be the final answer. "Ensure that the answer is short, accurate, has a friendly tone, and sounds helpful." So we're trying to refine that answer. And if you notice this particular system message is tasking the model with a couple of things, right? It's asking the model to compose an answer, then it's asking it to make it short, accurate, has a friendly tone, sounds helpful. That might look very simple, but that's actually something that's very challenging for the model to complete. But because we have now separated these two tasks of extraction and preparing a final response, it becomes a lot easier for the model. You could even go farther and divide this into simpler subtasks, but I'm going to keep it as is because I just want to focus on giving you a simple example of how to apply this prompt chaining technique. And I think the example that I have here is good enough. So I'm going to save this.

Linking the Chains Together

And then I need to change my human message. Well, I have the original document here. I'm just going to leave it there, but I need to change this. So this second part here, I need to provide it the extracted codes that come from that first chain, that first chain in that prompt chain that I'm building. So I'm going to save it and then I will explain to you what this is. Save that.

Basically what I want are these codes. I want this information to be the codes, extracted codes. So the way I will do that is I need to make that connection. And so the way we make that connection is by going here, Format Prompt Values, and I need to erase this and then I need to add my key name. And the key name needs to correspond to that key name I'm using in the human message, which is extracted_codes. And then I will fill this in and notice it's giving me 'extraction' here. Okay. So extraction, what is this? This is saying that it's an output prediction from an LLM chain, which is this one here. So I'll select that. And basically what I've done with this step is I am now passing the response or the result from this first chain, which is all of those codes with the code tags included, because that is what this particular prompt template is expecting. So I'll save that and that's basically completes the prompt chain here. So I have two parts, the extraction and the final response, and that completes my prompt chain example.

Testing the Complete Prompt Chain & Conclusion

So now I can test it. Can go here, I'm going to delete this one and then I will ask it again. So this is just good practice just to make sure that everything is going well. And I can go here and now I can ask it a question. So now when I ask the question, the user question will go through this first chain, do the extraction of the codes, and then the codes will be passed to this chat prompt template, and then it will try to finalize a response for me. So all of that, you don't really see in the chat interface. What you get is a final response, which is the response that the user will see. The user doesn't need to see all of the steps here, that's more for you. And here we go.

So this is the response that we're getting. It says, 'What are the prompting techniques mentioned in the document?' That's the question from the user. And then the response that we get here, the final response, 'To maximize the performance of LLMs, you can use advanced prompting techniques such as these ones. These strategies help improve the quality, accuracy, and complexity of LLM outputs.'

You may think, well, this response doesn't look that great, and you can keep refining that. That's the beautiful thing about this. Now you can just focus on refining that response that you're getting by just focusing on this particular part, which is in charge of the finite response, as opposed to having to refine this as well. So this really simplifies how you iterate on prompts. And this is why prompt chaining is a very important concept. We will be applying this concept of prompt chaining again. Please make sure that you're able to reproduce the example here because I think that practice will give you more intuition into what prompt chaining is and why it's so powerful.