Fully local multi-agent systems with LangGraph

Introduction and live demo of a multi-agent flight and hotel assistant

Hey, this is Lance. LangChain just released their agents SDK which builds on some prior work that they released called Swarm. This was a library that allows you to build multi-agent workflows, and the SDK extends it with a few different things, notably improved handoffs, guardrails, and tracing. Now I want to talk about building multi-agent systems using fully open source and local models. I'll show using a Swarm approach as well as a supervisor approach.

So here in LangGraph Studio I have a simple multi-agent system with a flight assistant agent and a hotel assistant agent as a toy example. I can ask a question, the flight assistant addresses the question. It calls this search flight tool based upon my input. It finds a flight for me. I can tell that I want to then book this flight and it books my flight.

Now I can tell that I want to book a hotel as well. This is the key idea behind many of these multi-agent libraries such as Swarm: there's a handoff mechanism that allows you to hand off from one agent to another. So I say I want to book a hotel and we can see the flight assistant hands off to the hotel assistant, and the hotel assistant finds a hotel for me.

Tracing the conversation and running models locally

Now I'm going to open up the trace and show you something interesting. So I'm in LangSmith now and I can look at all the turns with my agent. We can see turn one: the flight assistant made a call to the search flight tools in response to our input.

Now I want to draw your attention to one important thing here. What's pretty cool is that I'm actually running this locally on my laptop. So I'm running Cado Llama with Qwen 25 14B instruct as my model of choice, and this model running locally on my laptop is able to orchestrate this multi-agent workflow using LangGraph's Swarm implementation.

So first it makes a tool call to search flights. We can see the tool call made here that goes back to our model which then identifies the flight is found. In that second turn we said "book it." It calls that book flight tool. The flight is booked. And that third turn — this is the most interesting part — we said let's book a hotel and we can see it can then call this transfer to hotel assistant tool, which is the key idea in the Swarm architecture. It allows for different agents to interact with the user and transfer the conversation to one another when they deem fit. For example, the flight assistant sees that the next question from the user relates to hotels and it can transfer over to the hotel assistant which then addresses the user's needs.

What an agent is and local agent requirements

That's the key idea behind all these multi-agent libraries: there's some handoff mechanism between the agents. Now I want to pull back briefly and explain a bit more about what's happening here. First, it's worth level setting what actually is an agent. An agent is an LLM directing its own actions based upon environmental feedback from the action or tool call. In simple terms, you're going to think about this as tool calling in a loop.

We have a nice tutorial, and I do want to call out that agents are different than workflows. Workflows are some kind of directed control flow in a predefined code path. They can involve LLM calls. The key point of an agent is it can freely make tool calls in a loop until no tool call is made, in which case the agent typically finishes.

Okay, so that's an agent. Now one of the most interesting questions that I've been looking into for a long time is how can I actually run agents locally on my laptop? To do that I need small models that actually can perform tool calling effectively. Now I will call out I've done a lot of work on local workflows. Workflows don't necessarily need tool calling; you can build very simple routers with structured outputs and this actually works really well for a lot of applications. But as noted, the classic definition of an agent is tool calling in a loop, so we need the ability to call tools locally if we want to build locally running agents.

Function-calling leaderboard and Qwen models for local agents

Now I'll show you a nice resource here that I like to use. If you go to the Berkeley function-calling leaderboard right here you can scroll through models based upon their rank on this tool-calling benchmark. Now you'll see the state-of-the-art or the best performing are of course large and often closed-source models like GPT-4o, Gemini, but you'll see something interesting pretty high up is Qwen-25 which is open source.

Now this is a larger version — 72 billion parameter model — and it's difficult to run locally unless you have a GPU. But if you scroll down you'll see something interesting: Qwen-25 14B instruct is actually ranked 30, which is quite strong for a 14 billion parameter model. Now 14 billion parameters is a model size that can often be run locally depending on your machine. So I have a Mac M2 Max 32GB and I can run this locally with reasonable latency. It's not super fast but it's acceptable.

If you scroll even further you see rank 52 Qwen-25 7B. So basically the Qwen-25 models are quite strong at function calling as we see here and they can be run locally. So these are the models that I've chosen to work with in general when I need to do local function calling or build local agents.

Why multi-agent systems matter and architectures overview

Now why do we care about multi-agent systems? The main point here is that particularly when running things locally it can be useful to separate concerns. If you take an LLM and give it a very large number of tools you can increase the likelihood of confusion; the model may call the wrong tool given a request. However, if you have a subset of agents that have specialized tasks — one for flight booking like we saw, one for hotel booking with independent tools — it can often result in better performance, especially if using smaller models. So that's kind of the intuition behind multi-agent architectures.

Now an interesting point is there's different ways you can set up multi-agent architectures. So we have two separate videos on this talking about supervisor and swarm as two approaches, and we also have open-source packages that implement both of these with LangGraph.

Now I want to highlight the supervisor approach first. This is pretty intuitive. The idea here is simply you have a supervisor model that has access to some number of... okay, the supervisor interacts with the user and farms out tasks or hands off to the sub-agents which do their work, and they always respond with their final output back to the supervisor. So those sub-agents never actually interact with the user; you can think about them as subcontractors. The supervisor hands off a task, they do the task, and the output is passed back to the supervisor. That's one approach, pretty intuitive.

Now another is what we saw previously which is Swarm, where there's different agents and they can both interact with the user and they can all freely hand off to one another. So we saw that previously in Studio: the hotel agent starts interaction with the user, the flight agent starts interaction with the user. When the user asks for hotel booking the flight agent transfers over to the hotel agent which then executes. Some of the trade-offs: supervisor is always a starting point versus Swarm it's the last active agent or default agent in terms of the flow and interaction. The supervisor architecture always interacts with the user via the supervisor, whereas Swarm — each agent has the ability to hand off to any other agent and interact with the user. It kind of depends on your application. If you have an application where you really want to centralize the interaction with the user with an overall supervisor and subcontract out tasks, supervisor is a good architecture. If you have a problem that benefits from multiple agents being able to engage with the user and just freely hand off between one another, then you may consider Swarm. We have two packages: lgraph-supervisor and lgraph-swarm which you can have a look at, and previously in Studio I was using lgraph-swarm, so you can see it works really nicely. What's pretty cool is these are simple enough that they can be run with open-source models locally on your machine as we just saw.

Notebook example: installing packages, creating agents and running supervisor

Now let's see how to build multi-agent architectures in a notebook using these packages. So you can see I just have a few pip installs and I grabbed lgraph-supervisor and I also grabbed lgraph-swarm. So you previously saw Swarm working in Studio so we have a sense for how that works. I'll go ahead and show supervisor here in the notebook just to give you a kind of concrete sense for how to set up these type of systems.

So as mentioned I'm going to use LangChain and Olama and I'm going to go ahead and initialize two different models. I'll use 14B instruct which I previously used in Studio as we saw. Now here's where I'm going to find my agents. First I'm going to set up some tools and add a multiply and mock web search tool, and I'm going to go ahead and use this create_react_agent pre-built.

Just to explain this a little bit more: this workflows and agents tutorial I'll link, go to the agent part, you can see this is a simple diagram of an agent and right here we go ahead and show implementation of a simple agent. We define some tools — now this is very precisely what happens under the hood here. We defined an LM node which will go ahead and invoke the LM. We'll define a tool node which will basically just execute a tool and some conditional logic to continue until no tool calls are made. We construct that and we test invoking it.

Now this logic that you see right here is so commonly used that we do have a pre-built for it, and that's exactly what we just imported in our notebook as well: it's called the create_react_agent from LangGraph prebuilt, and it's literally just this stuff. So again I want to be careful with abstraction; if you want to implement yourself from scratch just follow what's in this tutorial, but sometimes they use this create_react_agent just as shorthand where all you need to do is go ahead and pass the LM and your tool list to this create_react_agent.

So just talked about create_react_agent and we'll use it to create two different agents: one for math, one for research. Each one we simply provide a prompt instructing it what its role is and what tools it has access to. We pass in the tools — that's really all we need to do. And again remember for the model we're using Qwen 14B instruct.

Now here's where we define the supervisor and this is where we're importing from that LangChain supervisor package. We give it an overall prompt indicating the tools it has which are these handoff tools and the two different sub-agents that it's overseeing. We bind the sub-agents here. We provide our model again being Qwen and we provide this prompt.

Here's a visualization of a workflow where the supervisor can conditionally call either expert or decide to end. Let's go ahead and run this. And again I want to show that it's actually possible to set up these simple multi-agent systems in this case using supervisor but previously using Swarm that can actually run locally if you choose the right local model. And again as mentioned based upon the Berkeley leaderboard I do like the Qwen-25 models: both 7B instruct and 14B instruct are two that I've tested and I found that these actually work pretty well. We saw Swarm working previously; now let's see supervisor working.

So it ran in 45 seconds. We can see the full set of messages here where here's our input: the supervisor executes a transfer to the research expert. The research expert makes a tool call — here's the web search tool. We go back to the supervisor. The supervisor provides a summary to the user. We look at the LangSmith trace to convince ourselves that everything was working properly. So we can see again we're running Qwen 25 14B locally on my machine.

The first call to supervisor takes the question and transfers to the research expert. The research expert then performs a tool call to web search. The web search tool returns the results and the search agent provides our answer which gets transferred back to the supervisor. The supervisor then provides a summary as we see right here which is returned to the user. So again all running locally in this case using a different architecture — supervisor versus Swarm — but clearly illustrating the ability to perform handoffs between supervisor and sub-agents and within each sub-agent make correct tool calls.

So what's pretty cool about this is both of these packages, Swarm and Supervisor, are two different approaches for building multi-agent systems. They are pretty simple to set up. They can work even with local models as we've seen here. I think the key points to recognize are two things when working with local models:

Make sure you're very thoughtful about which one you're choosing. In particular I like to use this function-calling leaderboard to guide that decision. Some local models don't even implement tool calling at all and you'll see an error thrown, and some implement it poorly. I found the Qwen-25 models to be the best from my testing, but we should keep an eye out on new open-source models being released that are capable at function calling.

In addition, when you're using LangGraph you can easily use Studio to visualize your multi-agent systems and you can use LangSmith to see the traces to look under the hood at everything happening. We did that both with supervisor and with Swarm, showing tool calling within sub-agents as well as handoff between agents or between supervisor and agent in the case of supervisor.

So hopefully this is a simple introduction to building multi-agent systems. We can even build them with local models, in this particular case Qwen-25. Feel free to leave any comments or questions below. Thanks.