Introduction and Project Overview
Today let's build an OpenAI application using SvelteKit and Vercel Edge Functions. To do this I'll be referencing a project I recently built so you can see an example of how I can build your own GPT-3 application.
Live Demo: How the Recommendation App Works
First let's take a quick look at the application we'll be working with. It recommends shows or movies based on the criteria the user inputs. You can select whether you want a show or movie or both, select some criteria that interests you, and then you can even put as detailed of a description as you want. So for example we can say it must have at least two seasons and be on Netflix. Now if we click the button it'll use GPT-3 to generate five recommendations based on our criteria.
HTML and UI Structure with Tailwind
So now let's open up the code and take a look at how this works. First I'll briefly go over the HTML. Let's scroll down a bit in this file and you'll notice we're using Tailwind for styling. Now down here you've got our header and the title, but if we keep scrolling we'll get to the more interesting stuff. So here we have the drop down to select the media type we're looking for. Next we're looping through the category types which is just an array of 50 different categories and we're just laying the checkbox for each one. And then we have our text area for the user to add any specific instructions on what they're looking for. Finally we've got the 'Curate my list' button which calls our search function when it's clicked. So really quickly I want to point out that each field in this form is bound to a reactive value. For example the text area is bound to specificDescriptors and I'll go over these variables in just a bit.
Svelte State: Variables and Reactive Data
Now moving on we see we have some loading state and if we keep scrolling down we have some error handling and then finally we're looping through our recommendations and we're displaying each recommendation in a card once it's returned from OpenAI. So now let's scroll up to the top of this file and we'll take a look at our JavaScript. First let's go over some of the state we have. We have our categoryTypes which as I mentioned earlier is just an array of 50 different categories. Then moving on we have some reactive state. First in the list is cinemaType. This is bound to our drop down. By default it's set to TV show but the user can update it based on what they select in that drop down so it could be movie or no preference as well. Next we have selectedCategories and this is bound to our checkboxes so anytime a category is selected it'll be added to the array or removed if they deselect it. Then we have our specificDescriptors and this is bound to our text area as we just kind of mentioned so if the user types anything in this text area this value will be updated. Next we have our searchResponse which will be the recommendations returned from the OpenAI API and we've also got some loading and error state and then finally recommendations which is an array that will eventually contain all five of our recommendations returned from OpenAI. And this array will actually be created later on using our searchResponse string and we'll go over that in a bit.
Search Function: Preparing the Prompt and Sending the Request
Now let's move on to our search function. Remember this is called when the user clicks the 'Curate list' button. The first thing we see is if loading is true we return. This is so the user can't spam the button with multiple requests when there's already one in progress. Next we reset recommendations, searchResponse, and endStream to clear out any prior requests the user may have made and we set loading to true. In this project we're specifically using GPT-3 from OpenAI which functions really similarly to ChatGPT if you're familiar with that. So we have to provide it with a prompt. Here we are setting the prompt just based on the fields that the user provided. We're basically saying like, hey give us five recommendations of this cinemaType, so movie, TV show, or either that fit these categories and it's whatever categories they selected with this specific description if they provide one. I even go on to describe how I want this data to be returned to me or how I want the response to look. So when we get the response it'll be a string with each recommendation in a numbered list with a title and description and eventually we will transform this data into our recommendations array which we'll go over in a little bit. Okay so now is when we finally make the request. You can see we're calling our getRecommendations endpoint with the prompt and we'll open this endpoint file in a bit and take a look, but first let's move down here where we get our response. If for some reason the request fails we handle the error. Otherwise we're setting our searchResponse to the response returned.
Why Streaming Improves UX and Enabling Edge Functions
Now clearly there's a bit more going on here than just setting searchResponse to the returned response. If we look back at our application you'll notice we're instantly showing the response as it's generated rather than waiting for all five recommendations to be created and sent back to us. This creates a much better user experience because you can actually see what's being generated as it's being generated and you don't have to wait the few seconds it takes for OpenAI to create the entire response and send it back to us before we can display something on the screen. Now this happens because rather than using a normal service function we're actually streaming data from the API using Vercel Edge Functions. To turn a serverless function into an edge function is extremely simple. In our svelte.config.js file we need to ensure we're using adapter-vercel or adapter-node rather than the default adapter-auto. To do this we simply can install adapter-vercel and then update this file to use it. Next we just set edge to true like we did here and now our service functions will automatically become Edge Functions when deployed to Vercel.
Edge Function Endpoint and OpenAI Streaming
So now let's open up our endpoint and take a look at this Edge Function. We're going to ignore what's going on up here for just a second and scroll down to where we're exporting our POST handler. Now down here it's just a standard SvelteKit endpoint. As you can see we're getting our prompt from our request param. Next we're constructing our payload where we tell OpenAI to use this model which is GPT-3, we're sending our search prompt in as the prompt, and finally what's important and notable about this payload is setting streamed true. Next we're creating the stream using a helper function openAIStream which is defined above. Now this function opens up a stream with OpenAI so OpenAI can send us back chunks of text as they come in instead of waiting for the entire request to finish. So basically for every word that GPT generates it is instantly streamed to the client where we can display it to the user. So that is our generate API function. Let's go back to our frontend so I can show you how we use this response data.
Frontend Streaming Handling, Reactive Parsing, and Conclusion
Back in our search function this is where we're getting the response from our stream. As I already mentioned this response will be chunk values which will typically be a word, maybe two. Here we are taking that chunk value and adding it to our searchResponse state. So searchResponse is our single source of truth. Every word that is returned from OpenAI is added to the string and then using the string we work Svelte magic to actually display the recommendations on the screen. So let's scroll back up and check out how we use searchResponse here. We see this dollar sign followed by a colon which is special Svelte syntax that creates a reactive statement. All the code in this block will rerun anytime a referenced value changes, and since we're referencing searchResponse every time a new chunk is streamed in from OpenAI and searchResponse is updated this entire block of code will rerun. Now OpenAI returns a list of recommended shows with an empty line between each so we are splitting the string wherever there's an empty line so that we can create an array where each item is a single recommendation and then we are mapping this array to create our recommendations array that we are eventually looping through down in our HTML. Now there are two different data types that we're going to have in this array. One is a complete recommendation that has a title and a description and the other is an incomplete recommendation that is still being streamed in from OpenAI. Here we're checking if we have at least one full recommendation or if the stream has ended in which case we have all five recommendations and we are returning these recommendations in the form of objects that each have a title and description. If it's an incomplete recommendation it means that this is still being streamed in and we're simply returning the incoming string as the item in the array. If you remember down in our HTML we are looping through our recommendations array and displaying each in a card. If it is a complete recommendation we have the title and description and we display that; otherwise we show the incoming stream of the incomplete response. Looking back at the app this is what allows us to watch as a new response is being generated and then it reformats to have a title and description. So that is how you can use OpenAI and Vercel Edge Functions in this SvelteKit application. I hope I did a good job demonstrating the different streaming. Data from an Edge Function creates a much better user experience because you don't have to wait that extra few seconds for the entire response to be created. This is an open source project so if you want to check out the code I will link the GitHub repo below. You can also start from our Vercel template; you can instantly clone and deploy this project with a click of a button. I hope you enjoyed this video.
