Multi-agent swarms with LangGraph

Introduction to Multi-Agent Swarms

Hey, this is Lance from LangChain. There's a lot of interest in multi-agent systems right now, and I want to show a popular architecture called the multi-agent swarm. Now, with the swarm architecture, you can have any number of agents in principle. In this particular example though, I'm just showing a case of two to keep it simple.

This represents a customer support system with a flight assistant and a hotel assistant, so one agent that will do flight bookings, one that does hotel bookings. And the central idea behind swarm is it gives you a mechanism for these agents to hand off requests to one another and also share state when they do that handoff.

Swarm Architecture in Action

Let's see this in action quickly in LangGraph Studio here. So I passed a request related to booking a flight from Boston to New York. This is processed by the flight assistant, which responds to me. I can answer that I do want to book this flight. The flight assistant then goes ahead and books it.

Now, let's say I ask the flight assistant, "I'd like to book a hotel as well." What's going to happen? Well, if this was a single-agent system, the flight assistant wouldn't have any capability to book hotels, and so it would probably just return with a message telling you that it doesn't have the ability to do that.

But this is a multi-agent system with the ability to hand off information between the flight assistant and a hotel assistant. So let's see what happens here. The flight assistant hands off to the hotel assistant. The hotel assistant then interacts directly with the user and follows up exactly on that request. We say, "Yes, please, we'd like to book that hotel." The hotel assistant then goes ahead and books the request.

So we seamlessly transferred or handed off between the flight assistant and hotel assistant based upon the user's needs. This is one of the powers of a swarm-style multi-agent architecture. We're releasing a lightweight library called LangGraph Swarm, which helps you build this style of agent swarms, and I'll walk through that right now.

Swarm vs. Supervisor Architectures

First, I just want to differentiate the different architectures for multi-agent systems. So we talked a bit previously about supervisors. This is a case where we can take some number of agents and basically bind them to a supervisor. Now the key point here is that the user interacts only with a supervisor, and each agent only interacts with the supervisor. So what happens is a supervisor can hand off tasks to one particular agent. The agent does some work, responds back to the supervisor, then the supervisor decides what to do next.

The swarm is a bit different, where the swarm allows any individual agent to interact with the user directly and any agent to hand off to any other agent. So you're kind of removing the supervisor from the loop in the case of a swarm.

So again, what are the key differences? With a supervisor, the starting point is always a supervisor. With a swarm, you can set a default agent to be the starting point or, in the case of multi-turn interaction like we'll see shortly, the last active agent will be the starting point.

Now in terms of the flow between the agents, agents always interact with the supervisor in the case of the supervisor architecture, but with swarm, each agent can decide where to hand off information to any other agent.

In terms of user interaction, the user always interacts with the supervisor, versus in the case of a swarm, the user can interact with any agent. Each one is self-sufficient and can engage directly with the user. So swarms can be really good for systems that require many different agents to interact directly with the user, like customer support is a classic example when you have different sub-agents that are responsible for independent specializations.

A supervisor is better in cases where you have a bunch of agents you want to do work, but you don't really want to expose that work to the user. You really only want a single supervisor to actually orchestrate the work that the sub-agents do, kind of under the hood, unbeknownst to the user.

The Handoff Mechanism Explained

And so here's a diagram of a multi-agent swarm example. First, the user asks, "Book me a flight." The flight agent is active, responds with the flight. And until the user asks, "Book me a hotel," we directly engage with the user. When we see "Book a hotel," then that flight agent will call a handoff tool to hand off over to the hotel agent, which then engages with the user directly. This handoff is the central concept that's very important to understand when thinking about these multi-agent swarm architectures.

Building a Swarm with LangGraph Swarm

Now let's show that in code. Here's a notebook that you can see in the repo. First, I'll just find a model. Now I'm just going to set some mock data for different tools related to searching flights and booking flights and searching hotels and booking hotels.

This is really the new thing: we're going to find two handoff tools. You can see this is from the LangGraph Swarm library, and it just creates a tool that can transfer the user to a hotel booking assistant or a flight booking assistant.

We define a helper function to generate the prompt, and we create two different assistants: a flight assistant, a hotel assistant. And here's the key point: we go ahead and bind those transfer tools to each of the agents. So the flight agent will have a hotel transfer tool, the hotel agent will have a flight transfer tool.

Now, finally, we just take both of those agents we defined and we add them to our swarm, as you see here, and we set a default agent, in this case, a flight assistant. Now remember, this default agent doesn't particularly matter because if, for example, the first question's related to hotels, the flight assistant can just transfer to the hotel assistant.

We go ahead and compile. You'll notice that we compile with this checkpoint. So a checkpoint is a mechanism in LangGraph to save conversations to a thread, which is persisted, in this case in memory, across things like interruptions or, importantly in this particular case, across handoffs between the different agents. So the main idea is the checkpoint allows you to save message history to this thread, which is then passed to each agent during the handoff procedure. You'll see why that's interesting shortly.

Running the Swarm and Visualizing the Flow

We can go ahead and run, and this is exactly what we already saw in Studio. So we're looking for a flight from Boston to New York. We can see our flight assistant is the default, which we set right here, and it calls the search flights tool. And so we respond, "Yes, please." We call tool to book the flight. That's all fine.

Now for the interesting part, we say, "Hey, let's book a hotel as well." Now, this is where the flight assistant sees this request to book a hotel and transfers to the Hotel Assistant, which is the tool it has bound. And the hotel assistant seamlessly responds by searching hotels and responding directly to the user: "I found a hotel for you in New York. Would you like to book a room?" This is a classic example of how we transfer or hand off between the flight and the hotel agents.

We can respond with, "Yes, please," and we continue to engage then with the hotel assistant who books our hotel. And we can say, "I want to talk to the flight assistant now." Again, the hotel assistant sees that request and calls the tool to transfer to the flight assistant, and that's done.

Now, we can see that follows exactly what we saw in Studio. We went from the flight assistant, search flights, the human requested to book in that second turn. Then the human requested to book a hotel, so the flight assistant performs a tool call, transfer to hotel assistant, which transfers over to the hotel assistant. Hotel assistant then seamlessly answers the request, saying it found a hotel for us in New York right here. So this is just kind of a nice visualization of that transfer as it occurs based upon the tool call that the flight assistant makes.

Tracing Interactions and State with LangSmith

Now I want to show something else that's pretty interesting. If you go to LangSmith, you can see all those runs are logged. Open one up. Now go up here and click thread. Remember, we're using a checkpoint here, so all those independent interactions with the different agents resulting in handoffs are all saved to the same thread. We can click that here and we can see each of those turns.

So you can see our first turn was just we're looking for a flight, and again the flight assistant called the search flight tools and responds with the results to the user. We can see that right here: search flights and the response from the agent about the flight that it found and do we want to book.

Turn two, again in the notebook we said, "Yes, please." So again, on that same thread, then we pick back up and our flight assistant goes ahead and books by calling the book flight tool.

Now turn three, again on that same, we also said, "I'd like to book a hotel." This is where we called that tool to transfer to the hotel assistant, and the hotel assistant then processes our request. You can see the hotel assistant then calls a tool to search hotels. That tool call is made and it responds directly with some hotel options.

Turns four and five follow exactly what you saw in the notebook, where we continue to engage with our multi-agent system, and we finally end with a handoff back to the flight assistant based upon our request. So again, you can see that you can follow along with this flow in Studio really nicely or in LangSmith if you want to look at the underlying traces.

Another thing I'll show you in LangSmith that's pretty interesting is, for example, when we transfer to the hotel assistant, let's look at that model call itself. The hotel assistant will search hotels, but it actually has access to the full history of messages that came before, for example, that whole interaction with the flight assistant. And this is one of the key properties of handoff. Like we said, when you perform a handoff, you actually transfer the full message history, which is all saved to our thread. And so this agent, even though it's responsible strictly for searching hotels and booking them, it has access to the entire history. It has full context around what happened previously with the user, which can be very useful in certain cases.

Handoff Customization and Final Comparison

Now I do also want to call out there could be different implementations for how you hand off messages between agents. In our case, we basically just passed the entire history, but you can imagine many different ways to do it. You could pass some number of recent messages, you could pass a summary of all prior messages. So that handoff mechanism or the information specifically that's handed off could be modified.

Hopefully, that gives you an overview of the multi-agent swarm architecture and this simple library that we have for it. And again, just showcasing some of the tradeoffs or differences. Supervisor kind of centralizes control in a single supervisor which can call sub-agents to do work but always engages directly with the user via the supervisor. With a swarm, every agent is autonomous and self-sufficient. Every agent can interact directly with the user and hand off to one another accordingly based on the user requests. So those are two different ways of structuring multi-agent systems, and hopefully you found this to be informative. Feel free to leave any comments or questions below. Thanks.