Introduction: Building Budgie from Scratch
There are tons of Elixir tutorials out there, mine included, that walk through a specific feature or capability, but few prepare you to build a full web app. They're typically lacking in user experience, test coverage, or real world applicability.
For years my wife and I have been keeping track of small personal budgets using, believe it or not, a shared Apple note. Stock you might be thinking things like, "There's a million apps out there for this," or, "Why don't you use a spreadsheet?" and so on. And I hate to break it to you but most non-technical people don't like spreadsheets. It's also a hard sell to replace a free note doc with a huge financial management platform. As an engineer, I knew what I had to do.
My name is Christian and I'm a polyglot staff engineer with an affinity for Elixir. Over the last few weeks I've been building an app called Budgie, and I'm excited to walk through how I built it, sharing concepts and considerations that will help you in your Phoenix development journey. Expect to learn about data modeling, forms and validation, testing, resource scoping, user experience and a touch of security.
This is a change for the channel, my first multi-part series, so bear with me as I figure this out. All of the code for the project will still be made available on GitHub as episodes come out, with a link in the description.
Project Setup with UUIDs
With that out of the way, let's start building.
Every Phoenix app starts with a simple command: mix phx.new and then the name of your app, in this case Budgy. This scaffolds out an app with a specified name, preparing all of the dependencies needed for development. By default Phoenix uses numbers for IDs. These guessable IDs end up in URLs and can result in unintentionally sharing your usage stats to the outside world. While there's a ton of debate about it online, I'll use UUIDs instead of numbers in Budgy for IDs. To make sure the whole project knows about this preference in advance, I'm going to go back and add the --binary-id option. There are a ton of options available in the generator, like which database you want to use and specific features to turn on or off. I'm going to take the defaults with the exception of the ID and hit yes to fetch and install the dependencies.
Running the App and Exploring the Codebase
Now it's time to open it up in the editor. To run the app, first run mix setup. This will make sure that all the dependencies and the migrations are in order. And then finally run iex -S mix phx.server. If everything went correctly, we should be able to open up a browser at localhost:4000. And there it is. This server doesn't do much, but it does show some of the most basic structures of Phoenix. Those of you coming from Rails or Laravel will be more familiar with this than others.
Everything starts with router.ex which defines the views that are served in each route. By default, we have a GET at the root that is bound to the home function of PageController. And if we look at PageController, it just says when this runs, render a template that is called home and disable the layout because the homepage is generally a little different than the rest of an app. Now this home refers to a template in the page_html folder, and this name matches the name of the controller by convention. So if we go into home.html.heex we see a template that actually shows what we see on the screen. This controller method and HTML view combination is what's called a dead view, fully rendered on the server. Most of the pages in Budgie will be interactive LiveViews. Still server rendered but interactive through a websocket connection. But it's important to remember that not every page needs to be live in a Phoenix app.
Generating User Authentication
The foundation of most apps, and typically the first addition to the database, is the concept of user accounts. A few years ago the Phoenix core team built a code generator called phx.gen.auth that provides a great foundation for user authentication while avoiding some very specific security mistakes that beginners and experts alike may make. This generator is now included with Phoenix and is just a command away.
This command is mix phx.gen.auth Accounts User users. This will generate an authentication system where data is stored in a context called Accounts with a user schema called User stored in the database table users. And one more change before we run this: the default hashing algorithm for user passwords isn't the most secure option. Since hashing is expensive, just ask Bitcoin miners, the default option is bcrypt on Linux and PBKDF2 on Windows. These can let you run on less powerful server hardware at the cost of some robustness. But we want a secure app here so let's add --hashing-lib argon2 to make sure that we have the most secure option available.
Now, it'll ask if we want to use LiveView for the authentication system, to which I'll answer yes. Looking at the output of the command, this generator just wrote a ton of code. I'm not going to walk through it file by file, but I broke down some highlights. There are now two database schemas in the accounts context, User and UserToken. We also have new capabilities like user registration, user sessions (that's signing in and out), password resets, email confirmation, user settings (and it's just limited to changing email addresses and passwords), and some middleware that are used to load the current user, making it available to the rest of our app's routes.
Handling Database Migrations
Since this added a new library dependency, namely the password hashing library, it's necessary to stop the server and run mix deps.get. Then we'll restart the server. And well, we get an error. As with many web frameworks, Phoenix database structures are defined in files called migrations. This error message is telling us that the database doesn't have the latest schema.
Migrations are stored in the priv/repo/migrations folder, and now you can see that we have one. These files contain a function that describes how to get the database from where it was to where it needs to be. A lot of these can be written just with a change function, and sometimes you need to split them out with up and down functions if you're doing something really advanced. But this one isn't very advanced. Now these files start with a timestamp to enforce the order of operations, but be careful not to mess up the ordering if you're working with others. Git can't detect this kind of conflict. Anyway, this defines tables and indexes along with installing a Postgres extension that lets text be case insensitive, like the email address. Back in the browser if I hit this Run migrations for repo button, it'll make sure that all of the database stuff is up to date. All right, there's no more complaint and now there's the subtle little Register and Login button.
Establishing a Test Baseline
We'll get to that in a second. Before we go making changes to this app, let's make sure that all of the tests that generated are succeeding. For that, I'll run mix test. It's going to do a little bit of compilation because this is the first time in the test context that these packages have been encountered. And just like that, we have 128 tests, all succeeding in half a second. I'm actually going to rerun it real quick. Yeah, two seconds that time. All right, so they all succeed, and this will be a great baseline going forward. These tests are found in the test folder under budgy and budgy_web with live and all of these different live views being tested.
User Registration and LiveView Feedback
So, let's go to this tiny little register link at the top right of the homepage. This will lead to a very basic registration page. Now if I start typing, notice that I'm getting some feedback right away. This is part of the advantage of using a LiveView instead of a dead view. The server is sending little diffs to the browser for feedback like, this should be red in color and should say, "must have the @ sign and no spaces." So, I'll type in an email address, let's just go [email protected], and give it a 12-character password. I guess that's the default. Uh, after doing that, I'm in.
Extending the User Schema with a Name Field
But something's missing. I don't know about you, but I don't like being referred to solely by my email address. It's the result of trying to find a name that wasn't taken, and I'm not usually very proud of them. Let's extend the generated code to add a name property to each user.
Let's take a look at the user schema module, which you might be inclined to call the model if you come from another framework. This file is really busy, but it starts with the definition of fields, then we get to changesets. These are the definition of how records can be created or updated. Notice the password validation rules; that's where that 12-character minimum came from. And there's a whole bunch of others that are just there for examples that we could enable or tweak as we see fit. So let's go up to the top here and add a name field to the user schema with a string data type. So field :name, and it's a string.
Now, I'll go down here and run mix test and see a nice amount of failures. Most of these errors seem to be coming from the database. The column 'name' doesn't exist. We're going to need a database migration. So for this I'll run mix ecto.gen.migration and we'll say add_user_name. That last argument is just the name of the migration file. It's not that smart. So we'll go into that newly generated migration, add_user_name, and it contains nothing. So I'll paste in the migration which just says to alter the users table, adding a name string column. Now if I run the tests, I should see some successes. In fact, 128 of them. It looks like that fixed everything.
Updating the UI to Display the User's Name
Now to check our work, let's add the user's name to the page header. Imagine I don't know where this is at. I can do a search across the whole project and look for the @current_user.email variable. @current_user refers to the binding of the user that was loaded by convention in phx.gen.auth. So this is our header, and it looks like it's in layouts/root.html.heex. So let's swap this out with code that will show the name followed by the email address in parenthesis. So I'll just paste that into place here and do a little bit of formatting. Now this is using Phoenix 1.7 template syntax, which lets you use curly braces like you do in React. In the browser, well, we got to run the migration first. Uh, we're rendering properly. Of course, our user doesn't have a name yet. And if I sign out and then try and register a new user, I don't see name on the registration page. Let's take a step back.
So a Phoenix LiveView form goes through several different steps and has a few different components. So far, we have updated the schema, and we have migrated that schema into the database. But there's still more things that we need to do. A LiveView will render a form, the form itself referencing a changeset. And then a changeset is defined within a schema, which migrates to the database. So we need to find this changeset. Let's go to the registration changeset in user.ex, and it just says we want the email and password fields. So let's add the name field.
Now over in user_registration_live, this is the LiveView for the registration form. There is a render function and within it a HEEx view. HEEx views can be within a LiveView directly, or they can be in separate template files. It all depends on the complexity and what you're going for. So in this case, I can copy the email field and then change this to be the name. This will be a type text, and the label will be name. We're going to set it to required here. And if I open up the browser, the name field is present. Now let's just set the name to the letter c. And we'll do [email protected] again. We're going to need a 12 character password. And when we close this success message, the user's name shows up.
Adding Server-Side Validation for the Name
And let's check our tests. All of our tests pass, which is a bit worrying. In the form, we said that name is required, but our tests aren't using a name. This is because the required property on our input is just for the browser's HTML element. If we want to make sure that the user has a name, we're going to need to add the validation to the changeset.
Now, I wrote a little helper that matches the same style as validate_password and validate_email. And in this case it will be validate_name. validate_name is going to say that name is required. And let's say that a name has to have at least two letters and at most 100. It's always a best practice to set a maximum on user-defined inputs so that your database doesn't blow up when somebody decides to be malicious. And then I'm going to pipe this into validate_name. And that should be it.
Fixing Tests by Updating Fixtures
So I'm going to run mix test, and there should be some failures, in fact, 104 of them. So we're going back and forth on everything is great and everything is terrible. But I think this is fantastic because it means that our tests before, while they were all passing, weren't actually covering this concept of the name being required or even existing.
Generated contexts in Phoenix projects typically come with a fixtures file. This file defines how to construct things like your schemas or any other information that's necessary within that context. And this one is structured such that it has functions that give you unique or valid inputs, and then a fixture for creating a user that's relied upon in most of our tests. So what we can do is replace this definition and make sure that it's not just email and password, but also includes the name. So now we have an email, we have a password, just like before, but we're also making sure to pass the name property calling the unique_user_name function that just returns a string that is my name and then a random number.
Now, if I run the tests again, we are down to one failure. And this one actually looks pretty straightforward. It's saying that on accounts_test line 98, we are saying that the required fields need to be password and email. And of course, that's not true anymore because name is also required. We run the test one more time, we are at 128 successes. Everything is fine now.
Final Review and Next Steps
Over in the app let's test our work. If I go in and register and try and use the name C, it's going to be too short, at least two characters. And if for some reason I try and put in a whole bunch of letters, it's going to say should be at most 100 characters.
Changesets are really powerful things, and their validators allow you to describe properties that result in error messages that you don't have to type out all the way. They can be localized, and they are validated while you are changing the form as well as when you submit them. So this isn't something that'll be bypassed just because somebody decides to go and submit a form by changing some of their HTML. The back end is in control here, sending information and feedback to the front end, which means that you only have to define things in one place.
In the project repo, I've added some more tests to make sure we're validating the length of a name properly. I also added a form to change the user's name in the setting page. If you want to see how that went, take a look at the pull request linked in the description below. But this video is getting long enough, and we haven't even started working in the app's core domain. At this point, we have a solid foundation on which we can build this app. Make sure you don't miss future updates by subscribing. There's a lot more building ahead of us.
This has been Code and Stuff, thanks for watching.
