Building Effective AI Agents with Dify

Introduction to AI Agents and Dify

I am on Dify's blog on agents. It's amazing. It tells us how to define agents by their type, what are the building blocks of agents, and the most common patterns and use cases of agents. So in today's video, let's build these common patterns of agents in Dify.

Dify is an open source agent development tool. You can check out our GitHub repository and use it locally by Docker Compose, or you can try our free cloud version at dify.ai.

So as first introduces the basic building blocks of agents. These building blocks ensure an LLM can access the right context when it needs it. And there are a few ways to do that: using tools, retrieval-augmented generation, and memory. So let's use Dify to build an agent that can interact with all of this from scratch.

Building a Dynamic Knowledge-Based Agent

Here I'm at the cloud version of Dify. So I want to build an agent that knows everything about the TV show The Office and I will create an agent called The Office Wiki Agent. The first thing we're going to do is to prompt our agent. So it basically tells the agent, you are an AI assistant that knows everything about The Office. You can use the information retrieved at the knowledge base. If the retrieved knowledge is not sufficient for answering the question, you should use Google Search to generate the answer. Here you can see information about season one, episode one is available in the knowledge and the Google Search tool is ready.

Before I talk to the agent, I just need to make sure I'm using the right model. In the right upper corner, you can choose whichever model you like. For an agent, we want to use a model that supports function calling and has better reasoning capability. So if I ask, what's the iconic scene in The Office season one, episode one, which by the way information is in our knowledge. Once the prompt is triggered, you can see it will use the knowledge and generate a pretty decent answer. Well, if I ask what's the iconic scene in The Office season 2, episode 2, which it can't retrieve from the knowledge base because we only have season one, episode one available. Our agent will just use the Google search and give you the right answer about it.

When to Use Agents and How to Debug Them

So the agent we just built can actually choose to interact with different building blocks in different environments. If you give the context of the answer and know there is sufficient information for the query, and if it doesn't, it will ask for help from Google search by itself.

So this falls into the definition by a16z. Agents are systems where dynamic data with their own processes and tool usage, maintaining control over how they accomplish tasks. Agents should be used when you're facing problems that you don't even know how many steps are going to take. Agents typically cost more to run and can accumulate errors across steps. This is why testing and guard rules are essential before deployment. In Dify, you can debug the agent using multiple models at one time. It can inspect the chat history and the building blocks usage, and you can read the query and response by using keyword filtering and a moderation model.

Workflow Pattern: Prompt Chaining

Now let's go into the second category, which is called workflow. Workflows are systems where tools are orchestrated through predefined code paths. What this means is the pattern between input and output doesn't have to be a single thread like the agent loop. So it gives us a bunch of patterns of workflow. Let's build all of this in Dify.

Prompt chain is a workflow that can break tasks into crucial steps. The LLMs in prompt chaining will use the output from the previous step as input. And the gate here would use code to validate whether the previous output is qualified for a subsequent process.

I'm going to create a recipe generator and at the start point, we use ingredients as the input. The output of the recipe generator will go through an if/else block. In this if/else block, it can tell if the recipe contains the word garlic. If the recipe doesn't use any garlic, we can proceed with an LLM that will help develop a more detailed recipe, and another that performs cooking tips addition. If there is garlic in the recipe, the workflow will instantly fail.

Let's use tomato, potato, and chicken as an example. When the workflow is running, you can see the response time, output of each block, and token consumption. So here we can see the workflow failed because there is garlic in the recipe.

Workflow Pattern: Routing

Now let's take a look at routing. Routing is a workflow that classifies input and directs it to a specialized sequence. It will use LLMs to evaluate the input, determine the category, and direct the input to specialized substeps.

Again, I will create a recipe generator workflow and take ingredients as input. The question classifier block, which can make a decision based on the prompt of classification, will take the ingredients and decide what is the perfect way—whether fry, stew, or bake—can produce the best dish according to the ingredients. So if I run this workflow with potato, steak, it will direct the ingredients to the specialized fried recipe generator.

Workflow Pattern: Parallelization

Let's now look at the parallelization. It's a workflow where multiple LLMs work simultaneously on one task. It can break a task into independent subtasks that run in parallel, or it can run identical tasks multiple times to get diverse outputs.

Again, let's use the recipe generator to implement parallelization. I'll make the ingredients go into these three recipe generators all together, and I will use a variable aggregator to get three recipes into one piece.

So parallelization is useful when you need different opinions on a certain question, like you want to decide which is the best way to cook potato and steak but you want to make sure you heard all three cooking methods first.

Workflow Pattern: Orchestrator-Workers

Now let's talk about something more interesting: the orchestrator-workers workflow. An easy thought for this workflow would be when you're writing a report. I use one LLM to generate the outline of the report. Then there is the orchestrator that distributes all the outlined sections to different workers to write independently.

So in the recipe example, we first use the recipe outline generator, and then we have this parameter extractor block. It will use an LLM to extract an array of recipe steps, and then an iteration block will take this array. Inside the iteration block, there is an LLM which will take the name of recipe steps and generate each part of the recipe. This LLM will repeat as many times as how many steps are in the array. In the end, we have a synthesizer which will take the output array from the iteration block and synthesize it into one big recipe.

This workflow is useful when we can't predict the subtasks at once. So we don't know how many steps there will be to cook ingredients like potato and steak. But it doesn't matter. The orchestrator will help us distribute each of these steps to the workers, and workers can handle your task as many times as there are steps.

Workflow Pattern: Evaluator-Optimizer Loop

For the last workflow, we have evaluator-optimizer. It has a feedback loop between two components. The optimizer generates initial responses and iteratively improves them, and the evaluator will assess responses and give feedback.

So here we use this loop block as a container of our evaluator and optimizer. We create two global variables which are the recipe and feedback. The optimizer will generate the recipe based on the ingredients and feedback. The evaluator will generate the feedback and the condition for qualification, which is ending the feedback with the word 'success'. The variable center here will assign the output of each LLM into the container's global variable. And the loop container will check if the global variable 'feedback' ends with the word 'success'. We can actually see it took our model four times to generate a qualified recipe.

Publishing, Collaboration, and Final Thoughts

Now you've learned these common patterns of agents. You can actually implement your own use cases and make everyone know about the project by simply clicking on the publish button. Go share this link with your collaborator and let them help you to validate if your idea is right or wrong. The prototyping and validating of an AI agent, it's a very iterative process. You should always find someone to build with you to get involved.

In Dify, we allow you to invite up to 50 team members and build up to 200 apps. This is where your AI agent idea starts. So now go to dify.ai and sign up to host your first AI agent.