Introduction to Dify for AI Apps
What if you had an idea for a new AI app that would take a query, get some information from a web search or some other tools, pass that info and the query to the model, and spit out the answer? How long would it take you to build that app? It's not that difficult, but how long would it take you to spin up the infrastructure to host that app, including the web search API, a vector database if you need it, a front end, and anything else your application requires?
Using a tool called Dify, you could have it do it for you. So all you need to do is connect the components in a graphical interface, answer a few questions, and click a button to deploy it. You will instantly have a UI your users can work with right away, as well as an API for the app for when you want to build a more bespoke UI. It's kind of magical. It isn't perfect, but I kind of dig it.
Hosting Options and Self-Hosting Setup
The one thing you have to set up is the model provider. I'm going to assume that you have Ollama installed for this one, but if you don't, you can go to ollama.com and follow the instructions in the middle of the homepage.
There are two main ways you can use Dify. You can pay them to host the application and anything you build, and the prices seem pretty reasonable, but I'm more excited about self-hosting and using it with Ollama.
Configuring Dify with Docker
So to spin it up, you'll need to have Docker installed. I'll assume that you've done that already. If you don't, there are plenty of tutorials for that. Then you need to clone the GitHub repo to your local machine or wherever you want to run Dify. So, git clone and this URL.
Now go into the Dify directory and then into the Docker directory. Copy the env.example file to .env. If you already have something running on Port 80 on your machine, there are three settings you need to change in the .env file.
Setting Environment Variables
First, decide what port you want to use. In my case, I'm using Port 8088. So in the .env file, if you're connecting to Local Host, change APP_WEB_URL and SERVICE_API_URL to http://localhost:8088. This will be used in the UI for any links to other parts of the app. And way down at the bottom, change EXPOSE_ENGINE_X_PORT to 8088. There's a lot of other stuff you can set here, such as the vector database to use, storage integrations, and more, but let's move on to running.
Launching Dify and First Login
At the command line, run docker compose up -d. It'll take a bit of time to download all the images and start everything up. There are at least nine containers running when everything is set. So, open up a browser to localhost, port 8088. The first time you log in, you need to set an admin account. Then you can log in as that account.
Navigating the Dify User Interface
There are a few different parts of the product, and I hope they reorganize the navigation because there are some things you only need rarely that are in the top navigation and other things that you need every day that are buried several clicks away. You typically start in the Studio. This is where you design apps and can see the apps you've already created. Explore lets you discover examples of Dify apps. Knowledge lets you add content to a vector database. Tools is a list of all the tools you can use, and this is where you set up the login information for those tools. Thankfully, there aren't a huge number of tools because they aren't in any order, and there's no way to see which are enabled and configured and which are not.
Configuring Ollama Model Providers
Well, let's start by configuring a couple of things. The first is going to be Ollama, our model provider. I'm going to assume that you have Llama 3 and Nomic embed text models installed. If you don't, run ollama pull llama3 and ollama pull nomic-embed-text.
So back into Dify, go to your name at the top right of the window. Under there, you can find Settings, and right at the top is Model Provider. Of course, this also has no sort order, so you just have to look for it. When you find it, click Add Model. So we want to add a regular chat model like Llama 3. That's going to be the first option. For model name, enter llama3. This actually has to be a model you have, and it will error out if you don't.
Base URL can be a little tricky. If you installed Ollama locally on your machine, you might be tempted to put localhost port 11434. But Dify is running in Docker, and so localhost would probably refer to that Docker container. What you need to do instead is put http://host.docker.internal:11434. That's how this container can find the actual host of the system. If you put your Ollama server somewhere else, then point to that location.
For completion mode, leave it at Chat. For model context, we need to be a little bit careful. The original model released by Meta has a context size of 8K, and there are lots of fine-tunes that bump that up by enormous amounts. But all models in Ollama, with a few exceptions, default to 2K, since many users have limited memory. It's easy enough to increase that by setting the num_ctx parameter in a model file or in the CLI or in the API call, but since we're just using the default, let's set this to 2048.
The next thing is Max token limit. Some models have a context size that's a lot bigger than what it can output. There was a recent Anthropic release that has something like a 200k context but output maxes out at 4K. Normally, we can just set this to the same as the context size. Finally, this does not have vision support, so leave that unchecked.
Since we're here, go back to the top and click Add model for Ollama again. For model name, enter nomic-embed-text. Enter the same URL which in my case is http://host.docker.internal:11434. It's reasonable to have multiple Ollama servers, each hosting different models, so that's why we have to enter this each time. Finally, model context size is a little different. You can see on this page for the model num_ctx is already set to 8192, so enter that in the size text box. Now, I don't understand why, but most text boxes in this UI seem to take one character and then take the focus away. Totally bizarre. Okay, for each of these two models, you may need to click the switch to enable them.
We're going to be using web search, so the next step is to go to set up the tools, which probably should be done in settings but isn't. Up at the top navbar is Tools. Again, there's no sort order. I really like using SearchNG, that's search with an X instead of a CH. I have it running as a Docker container all the time and use it often. If you don't have it, visit this site and get it up and running.
Throughout Dify, they use very inconsistent phrases for configuring things. Here we need to just point to the URL for the SearchNG API, but instead of anything that makes sense, we need to click 'To authorize'. So click that. Our Dify containers are probably not on the same Docker network as SearchNG, so enter the same Docker host name we used before, assuming it also is installed on Docker. So, http://host.docker.internal:3333 since my SearchNG host is using port 3333. You may be using something else. Then click save. If you get the URL wrong, it won't let you move on.
Creating a Workflow and Understanding Blocks
With all that set, let's build our app. So go into the studio, create a new app by starting with a blank, and we have a few choices. There's a chatbot, text generator, agent, and workflow, but the only one that seems to be really interesting is the workflow. So let's choose that and give it a name.
The start box allows us to define the first inputs to the flow. When using a model, it's important to define a prompt, so create a new input field called prompt with a label of 'What would you like to know?' and a decent length.
Now click that plus sign on the right side, and we have a number of choices to use for our next block. LLM will send a prompt to a model. Knowledge retrieval pulls relevant content text out of a vector database. End is the final block of any workflow. Question classifier lets you use a model to determine if any variable fits a certain class. You define the classes, which are just buckets to group possible matches. Maybe you want to look for queries that have a date or an artist name or a place name or that need a tool to answer or anything else. For each match class you could continue down a different path. And then there is if/else where you can compare any variable against any other value and define different paths for each option.
Advanced Workflow Blocks Explained
Iteration lets you take any variable that has a list or array of items and then run the same block or multiple blocks on each thing. Maybe the previous block generated a list of topics and you want a set of tweets for each block to be generated.
Next is Code. This lets us run any arbitrary Python or JavaScript code using any variables as input.
Template lets you apply a Jinja2 template to any variable or set of variables.
Variable Aggregator lets you combine variables to create a new variable.
Parameter Extractor will use a model to try to extract one or more parameters from a variable.
Finally, there's HTTP request. This lets you make any restful API request with API keys, headers, and parameters.
We can also run lots of tools here. So if you switch to the Tools tab, you can find lots of ways of running web searches and lots of other things. You may have to set these up first, and there's no way to see what's set up and what is not. But one common example is adding the current date to a request, because otherwise the model has no way of knowing.
Building a RAG Workflow Example
So for our first block, choose SearchNG search. At the query string box, type a slash and then choose query from the start block. Leave everything else as is. On the right side of the box, click the plus sign and choose LLM. Choose Llama 3 from the model list. You can add whatever context you like, a system prompt, and then for the user prompt, just type in slash and choose query from the start. Now I'll add a period just in case the user didn't type one, and then add 'use the following information to help answer the question: ' slash and then choose text from the SearchNG section. Before we chose the plus sign on the right side of the box, but you can also go down to the bottom of this dialogue and click select next box and then select End. Now set the output to Output and specify the text from the model.
This is a pretty simple flow, but let's try it out. Click the Run button at the top right. Enter in a question then click Start Run.
Testing, Publishing, and Monitoring the App
If there are any problems along the way, it'll let you know. You can also go over to Tracing and it'll show you the input and output of each box. You can also publish and then run the app as an end user. It's not the prettiest app, and there isn't much customization that can happen, but it's a good start. But go back to the studio and choose Monitoring. Change analysis to Today and we see the runs from today. That's pretty nice. We also have access to the API where we can trigger the start and get the result at the end.
Review, Critique, and Final Thoughts
I also love the fact that you can build a flow one time and then reuse it in your other workflows by clicking 'Publish this as a tool'. There is a lot more to Dify and it's probably worth continuing to dig into it, but again, it's not perfect. The navigation in the app needs a bit of work, and nothing is is sorted in any way that makes any sense. The published UI is always the same, and it would be nice to be able to tweak it. But as I say that, it sounds a bit nitpicky. Some bigger complaints is that it's a pretty heavy app. There are a lot of moving pieces, maybe more than it really needs. It feels like it started simple, and rather than removing code, they just kept adding and adding, which gets a bit dangerous. I would love to see this get cleaned up a bit. Labels should be a bit more consistent. But again, this is getting nitpicky, and if I have to be nitpicky, then maybe I think it's really awesome.
For beginners, this is an amazing way to get started, and for more experienced devs, it's a great way to prototype an idea quickly without getting stuck in the weeds of deploying to a test group of users. We didn't really touch the RAG stuff built in here, but it defaults to using Weaviate and you can easily swap that out to using other vector databases as well. There is a lot you can do just by tweaking that configuration file, which unfortunately is only minimally documented. There's a lot in there that isn't very clear, but with a little trial and error, you can probably figure it out.
Conclusion and Outro
What do you think? Is this a tool that you could use? Do you have any ideas of what you might build with it? I'd love to hear about it in the comments below. Thanks so much for watching. Goodbye.
Start with a coffee and then some water.
