Introduction to LangSmith and Tracing
Hello, my name is Robert and I'm an engineer at LangChain. Welcome to our intro to LangSmith. LangSmith is an observability and evaluation platform for AI applications. And today we'll be using LangSmith to peek under the hood of an application that we've made using a feature called tracing. Let's get started.
To use LangSmith, the first thing you need to do is create an account. I've already connected my Google account, so I'll be using that. Once you've logged into LangSmith, you should see a screen that looks like this. There's a lot going on here, but don't worry, we'll be covering it all in future videos. For now, let's focus on this tab on the left called tracing projects.
Setting Up Your LangSmith Project
A tracing project is just a collection of traces or logs associated with your application, and each project corresponds roughly one-to-one with the application that you're building. The natural next question is how can we send traces from our application to LangSmith so that they can be recorded in these projects?
To do so, you first need to create an API key. You can create one by heading to the settings and clicking this create API key button in the top right. Personal access token is fine. I've already created some API keys for this demo, so I'll be using that. Next, let's hop over to my Jupyter notebook to see our application.
The Example Application & Environment Setup
This notebook contains a basic AI application that I've created. It's designed to explain a concept like I'm five. It's going to take in a question, do a web search on the answer, and then summarize that answer so that even a 5-year-old could understand it, at least theoretically. It's then going to give me that answer so that I can understand it. We're going to use LangSmith to trace this project.
And I want to highlight that LangSmith is framework agnostic. The LangChain engineering team strongly believes that you should be able to effectively monitor and observe your AI applications no matter how you've built them under the hood. So, we've made LangSmith easy to set up and use no matter how you've chosen to build your application.
The first step is to import some environment variables into your project. This includes LANGSMITH_TRACING, LANGSMITH_ENDPOINT, and your LANGSMITH_API_KEY. You can also optionally specify the LANGSMITH_PROJECT you would like to use. My project uses OpenAI and a tool called Tavily to search the web, so I'm going to import some API keys for them as well. Let's load them all in through an .env file.
Implementing Manual Tracing with @traceable
Next, before we start tracing, let's set up our project. So, I'm going to give our application access to the web and I'm going to define our prompt. You can see that the prompt for our application is that it's an expert in explaining complex topics in a way that's easy to understand. And its job is to answer the provided questions so that even a 5-year-old could understand it.
Now, we're ready to actually create our application and trace it with LangSmith. So once you've set up LangSmith by importing the environment variables, it's very easy to trace your application. The only thing you need to do is to add the traceable decorator to any function that you want to trace. In this case, I have a couple of functions that I want to trace for my application. The first is the web search step where I search the web. The second is the explanation step where I ask OpenAI to summarize the answer. And then finally, I want to trace our entire process from end to end. The full explain like I'm five experience.
Running the Traced Application
Let's put that together and actually test our application. And I'll give it a real question of what is complexity economics.
And after running for a little bit, you can see it's come up with a pretty decent answer. It's used simple, friendly terms like Lego blocks to explain the concept of complexity economics. Let's move back to LangSmith to see what happened in our tracing project.
Analyzing Traces in the LangSmith UI
Back in LangSmith, let's check out our projects. We can see that there's been a new project generated for us, the explain like I'm 5 bot project representing our application.
Clicking into it, we can see what happened when we ran our application. There's a new trace that has been generated representing the question that we asked our chatbot. If we want some more detail, let's click in. Here we can see a high-level breakdown of everything that happened when we ran our application. At the top level, we can see the input, which is the question, what is complexity economics? As well as the final output, which is the answer our AI generated using Lego blocks as an analogy.
If you look into the left-hand sidebar, you can also see a detailed breakdown of every single step that happened within your application. This is known as the run tree and it gives you a breakdown of each individual run that comprises your application's overall flow. For us, there's two steps that we had in our application. A search step where we searched the web and an explanation step where we asked OpenAI to summarize the answers. If you want to see more detail on a particular step, you can click into it as shown. For example, in our OpenAI step, we can now dig into exactly what happened when we called OpenAI. We can see the exact input we provided, including our system prompt and the context we supplied from the web, as well as the AI's output, which was the final answer we eventually returned to the user.
This isn't the only thing that LangSmith shows in its tracing capabilities. If you hover over any individual step, you can also see some helpful telemetry on that step. This includes how many tokens that step cost you or the overall latency of that step. And a great way to view latency is through the waterfall view accessible through this button.
This gives a nice visual representation of where the highest sources of latency in your project are. For example, if we wanted to make our chatbot a little bit more responsive to the user, we should probably start by looking at this explanation step because the slowest step with the highest latency was our call to OpenAI.
That's tracing with LangSmith. And next, I want to go back to our project to show how using one of our open-source libraries, LangGraph or LangChain, can make tracing even easier.
Automatic Tracing with LangChain and LangGraph
If you're not familiar, LangChain and LangGraph are LangChain's open-source libraries for building AI agents. And while they're not required, they do work out of the box with LangSmith. If you are using LangGraph or LangChain, you don't even need to set the traceable decorator in your application. All you need to do is import those same environment variables we imported earlier and LangSmith will work out of the box. To demonstrate this, I've created the same application using LangGraph. Let's go through the setup steps.
You can see that once I've compiled my application, it follows the exact same flow that we showed earlier. You ask a question, conduct a web search, and then summarize the answers using OpenAI. That final answer is output to the user.
Verifying the LangGraph Trace and Conclusion
Let's see it in action. We're going to ask the same question about complexity economics and we'll be able to view the trace in LangSmith afterwards.
So, it took a little bit to run, but you can see it's also generated a pretty decent answer to the question, what is complexity economics? Let's take a look one last time back at LangSmith.
Back in our project, you can see we have a new trace representing our call to our new application implemented using LangGraph. If we click in, we can see that we have all the same tracing capabilities. We can see the input, the output, and every step in between.
That's tracing with LangSmith. I want to thank you for joining me today and I'll see you in the next one.
