Intro: First impressions and why this matters
Hello my friends and welcome back to the channel. This is Noah with Learn Meta Analysis and today we're not talking about meta analysis. We are talking about Unsloth's new desktop app. This is going to be my first impressions on this and let me just say this upfront. I think I'm uninstalling LM Studio. Unsloth desktop app seems freaking awesome so far.
Creating projects and folder-based sources
Okay, so let's walk through this and why I think this is going to be basically the thing that's going to help a lot of people with a lot of things. First of all, if you all remember, we had a project a while ago. We were trying to set up connecting local RAG for academic RAG settings. And I know I just did a video about why I don't use RAG anymore. I still really don't use RAG, but Unsloth listened to our problem. They may not listen to us specifically, but they have addressed our problem. So, what we can do is if we go into Projects and we do a New Project, I'm just going to call this Testing Two. Okay, I'm going to hit Create. And look, if I go down here to Sources, you can now link a folder, okay? And it will keep it synced. Look. So, I'm just going to choose one here. I know it's got a popup you can't see right now, but I am just choosing a folder that is on my desktop with a bunch of PDFs in it. And look, what it's going to do is it's going to embed them all right now.
Embedding model and sync behavior
Now, I will say there is a pretty easy way to look at your embedding model. If you go into the Settings and you go into General, then you scroll down to Documents and RAG, it does have this here. This is what it's using by default. I have not been able to actually change this. I tried to change this and it always throws complaints and I'm just assuming it's because this is the beta. But here's the thing. Okay, this is already getting set up. It's going to put all these files in there and then I'm going to show you what happens when I remove one or I add one because this is what we've been waiting for, right? Like this is what we have been waiting for. We tried to get this working with Open Web UI and it just never did. And this does it and I am so excited that this actually worked. It's mind-blowing. Okay, so look, we got 17 files that are now there. So, what I'm going to do, I'm going to open up that folder and I'm just going to remove half of them. And then I'm going to go over here and I'm going to say Sync Changes. And look, now there's only six. I guess I got rid of a little more than half. I really did only leave six, but look, now there's just six. So, if we want to add them all back in, you could certainly do that. I deleted them, so I can't really do that right now and show you. But anyway, all you got to do is click this Sync Changes and it's going to sync it and do it. And I tested it and it worked and it was awesome.
Model Hub: discover and download models
Okay, so that's the first thing. Next thing, how do we download models? We go to ModelHub here and it's very, very similar to what you would see on LM Studio or something like that. So On Device is the ones that you already have downloaded. Discover is what is on Hugging Face and they do default to the latest Unsloth models which I guess you would kind of expect given that this is an Unsloth software, right? But you can search for anything you want in here. So you can see I was looking for the new Neatron 3.5 earlier today. Fun fact, this thing is pretty dang good at tool calling and I've been impressed with it so far in the little tiny bit of testing I've done. But here we go. So you can see this finds it here. If you want to download it, you pick your quantization and boom, you're done.
Model settings and context window auto-sizing
So next thing, let's choose this model. Okay, so let me eject it first and show you what's going to happen here. It's going to default to 4096 as the context window, which is pretty dang useless for most of us if we're trying to do anything particularly useful. But what you can do is when you highlight over this, you can click on the Settings and here you can turn the context length all the way up and it's going to auto fit the experts and everything to where they need to be. No more of this guess and check like we were doing on LM Studio. I just crank that sucker all the way up. I hit Remember for this model and I click Load Model and it just works.
Let's see. Load it up. All right. So, keep in mind I have a very slow system. I have 8 GB of VRAM, but we'll just say hi. And I'll show you guys that it works. It goes through, it does its thinking or whatever reasoning it's going to do, and it's going to output a response, and it gives us very similar information to what we'd see in other apps. We see tokens per second up here, we see context window, all that kind of fun stuff. Other things that I think are absolutely fantastic: built-in web search. If I remember correctly, this is using duckduckgo. I could be wrong about that. It's got code. It's got the thing that I got excited about when I was looking through here. It's got Deep Research, right? And this actually works. So let me show you how this Deep Research works because it's pretty cool.
Deep research pipelines, recipes, and training UI
And then I'm actually going to probably just show you one or two more things and we're going to call this good because this is definitely meant as first impressions. I've used this software for maybe an hour, hour and a half so far and it was launched this morning and I can already tell you I'm planning on uninstalling LM Studio and just sticking with this. And I'm probably going to uninstall Open Web UI too because this is doing everything I want to do plus more. Okay, so let's say Deep Research here. Do pedagogical agents improve learning? Only use academic sources and we will hit enter. And it's going to show us in the sidebar here that it is going to come up with our plan. And my computer's a little bit slow. So as it does this, I'm going to show you something else and we're going to come back to it. Some of the other things that are on here: it can do images, it can do video, it can do audio, it can do recipes. So let me show you the Recipes thing while this continues to work. So what this is is you can create all sorts of different things, but I found this particularly helpful for creating supervised fine-tuning data sets. And so that's what I thought was pretty cool about this. And speaking of the thing Unsloth really got known for is their ability to do model training. And you now have a really nice GUI here to set up and run model training locally. So one of the questions that you guys might be wondering is, if you watch the channel you know that I like using—oh okay look so this local model it failed right so all we got to do over here go ahead and click Retry and it's going to restart again and it's going to go through and retry that last step that failed. I would say of the—I don't know, I probably tested five or six models with this Deep Research so far, I would say maybe one out of every five times that I do this, it will run into an error like that where a tool call failed and I just hit Retry and it tends to work fine the second time through. So, generally speaking, it's working pretty well. It is just about done. It's planning and it's going to let us edit the plan, which is something that I really thought was pretty cool. And then it actually listened to my edit, which was even cooler. Okay, so you can see here this is the research plan that the model came up with. It's 12 steps. And all we would do is hit Start Research or we can go in here. We can click and we can edit any of this. So I'm going to go ahead and I'm going to click Edit Plan. And then here we'll say Definition: educational technology. I don't have anything I really want to edit. We can just go ahead and hit Start Research. And you will see I'm not going to let it run the entire time during this video because on my PC to run through this entire pipeline takes roughly like 20 minutes. That has to do with the model I'm using and the context window and a bunch of other settings about my specific computer and my task.
API, connected web models, and limitations
All right, so last thing I want to show you. It does have an API that can open up and you can use it just like on LM Studio. And the other thing that I thought was really interesting is you can actually connect it to web-based models. And so I connected this to ChatLM which is something I've been using for a very long time and it does work. We can go over here and I can go to my Connected Models and I'll just choose like, let's see, I got everything in here from ChatLM. You can see they have a bunch of different models. Let's try OSS120B and we'll just say hey and we will let it respond which hopefully it will do in a reasonable amount of time. Yeah, there we go, 3.81 seconds. So, the only thing I've noticed so far, and again, I think this is probably just a product of this being beta software, is I do not have access to the tools for these connected models. And when I was going through and looking through my settings here, I have not been able to find where to set that up. So not really sure what's going on with that. I don't know if that's an intentional decision on their part or something that is specific to this provider or if it's something that is just part of being beta and it's going to get fixed in the future.
So something else I just want to show you quickly: it was this Tool Permissions. So when we do have these tools you can set it up for—they have to ask for permission for absolutely everything through—you can just let it run everything itself. And I chose kind of this happy medium of Run Tool Calls but Ask for Permission before high-risk things. And that seems to be working pretty well for the types of things that I'm doing so far. And just to show that this is still going through and running here. You can see it's going through and it's running through all these different steps. So generally speaking, I would say my first impressions of Unsloth desktop app: very, very good. Much better than Unsloth Studio when it first came out for Docker only. I tried that. It was good, don't get me wrong, but it needed some refinement. This has a lot of those components already refined, and even though it's a beta, I think they are well on their way. I am really excited about this so far. And like I said, I plan on uninstalling LM Studio, plan on uninstalling Ollama if I haven't already, and I'm probably going to go ahead and uninstall Open Web UI as well because this is doing everything I want it to do, and it's one thing, right? It's one app that does everything.
Open source, installation, and closing thoughts
The only thing I think I'm probably going to keep at this point is Lemonade. I really don't use Lemonade that much, but I have had fun with it because it can load up some Omni models and things like that, and I don't know if this can do that or not yet. I haven't tried it. But anyway, suffice it to say, I like the Lemonade app. It's fun. And yeah, overall I am super pumped about Unsloth and their new desktop app here. Oh, the other piece I guess is important that I meant to say in the very beginning: this is all open source. So this is all up on GitHub. You can see everything about their code. If you were one of those folks who wanted to look through there and the install was flawless for me on Linux. I just downloaded the .deb. I have like a little program manager that I use for myself that helps me know when things need to be updated and things like that. It installed absolutely no problem whatsoever. So, that said, if you are sick of some of these closed source apps out there but you want a really easy to use chat UI that has some additional nice tools like Deep Research and web search and code and things like that, Unsloth Studio might be a good option for you. Like I said, I'm really impressed with it so far, and it's only been an hour or two, but I am really impressed with it. So that said, I'm going to end the video here because this was really just meant as a first impression and you guys know me, I will ramble forever if given the opportunity. So, that said, I hope you guys have an absolutely wonderful day. I plan on having some meta analysis and systematic review content for you guys next week. I'm thinking about a couple different videos, haven't landed on which one I'm going to do first, but suffice it to say, we will be back to some systematic review and meta analysis things probably within the next week to ten days, as long as everything goes well on the recording side. So that said, I hope you all have an absolutely wonderful day. If this is helpful for you, please hit Like and Subscribe. It really does help the channel. And I will see you all in the next video.
