From Project to Productionized with Python

Casey Faist: Hi, I'm Casey Faist the queen Pythonista at Heroku and this is From Project to Productionized on Heroku. Now, I wish we could be together today. I wish we could be swapping stories and coding together. I wish we could hear the groans from my deployable jokes. Now this is intended to be a hands-on workshop, so while video format presents some challenges for that, the good thing is that if at any point you need me to slow down or you want to go off on a rabbit hole, just pause the video. I'll be waiting for you right here. Now, this is intended as a hands-on workshop, so if you're going to follow along at home, first of all, high five, yeah. Second of all, we assume a couple things about your experience and there are a couple of prerequisites necessary for a good experience with this workshop. The first thing is that this is not a Django tutorial.

Casey Faist: If you're looking for intro to Django information, go to the official Django docs where they have some excellent tutorial information. Next, we assume a little bit of Git familiarity, you need Git installed on your machine that you're working on. We provide a starter repo for you to work along with that has instructions but it's available on GitHub, so you'll need to be able to Git clone that repo. To complete this workshop, you'll need four things. You will need to sign up for a Heroku account. This is free, you don't need to add a credit card. You will need to download the Heroku CLI. You will need to clone the workshop repo or use a project of your own, and you will need to be able to open that repo in a text editor or IDE of your choice. All right, have all that ready to go? Awesome. Let's get to it.

Casey Faist: So let me set the scene. You have an application, you've poured your blood, sweat and tears into this application. It's wonderful and it's finally ready. It's ready to go out there and be on the Internet. How do you do that? Luckily with Heroku, you've already done the hard part. See, Heroku is a platform-as a-service and unlike infrastructure-as-a-service models, there's a bunch of stuff that you just don't have to do. Stuff like infrastructure maintenance or building deploy pipelines, setting up tools to manage user access or system level security updates. You don't have to do it. It's not your problem. That is what Heroku is there to do. We're managing those security updates, providing tooling, all of those, plus metrics and more to make your life as a developer easier. It's a huge time save but there's a catch. A 12 factored catch.

Speaker 2: Boo.

Casey Faist: See Heroku works best with 12 factor applications. This is something that Heroku championed more than 10 years ago. It's the idea that you build an application with robust redeploys in mind. Things like config variables, using processes and more. If you'd like to learn more, you can check out the link here. But most of this workshop is actually not specific to Heroku. A lot of what we're going to do today is about taking a Django application and making it 12 factored. There's a lot of benefits to using 12 factor applications whether or not you're deploying to Heroku but it's especially important on Heroku because of something called ephemeral deploys. Ephemeral deploys or ephemeral environments are designed to spin up or down at a moment's notice with very little downtime but this means that you can't persist anything to the local file system.

Casey Faist: It either needs to be checked into Git, stored in a database or as an environment variable or in an attached resource like an S3 bucket that does persist. Now if you've never worked with 12 factor applications before, don't worry. We're going to do 10 steps together to take your application From Project to Productionized on Heroku. Once you've opened the project in the text editor of your choice, take a second to look around. Your editor and your terminal should be in the root directory of the Pycon 2020 project. So I type ls, you should see manage.py, requirements.txt, and workshop.md. You'll also see two Django projects. The first is hello, we won't do anything to this one, so don't worry about it but the other one is called getting started. We'll make a couple of changes to this one including updates to settings.py and the WSGI file. But for now, bring up the workshop.md file. Our first step is to update the gitignore file, copy the text here in the code block, then navigate to the gitignore file in the root of your project. Highlight and copy and paste.

Casey Faist: If you're working off of the raw markdown file, you want to make sure you don't transfer any quotes or back ticks as they'll mess with how this file works. I created a virtual env already, up here it's called venv, so I don't need to add this line, I can remove it. I don't want any pycache files stored. I also don't want to add any SQLite databases or sequel SQLite artifacts to source control. Any local data that I'm using for testing or to ensure my application works that should stay locally on my machine and not get pushed to Git. However important note, you absolutely do want to add your Django migrations to source control, so make sure that you do not add any of your Django migrations to this gitignore file. Okay. And now the last step, this app is not called your app, it's called, we're actually looking for getting started here. This is where our static files will generate. Heroku will automatically generate our static files for us, but we don't want to check our local ones into Git, we want Heroku to rebuild those on deploy.

Casey Faist: This looks good. So I am going to save the file. Then I'm going to come down to my terminal and I'm going to check this change into Git, so I can check the status real quick just by the git status command. Cool, I only have the one change I'm going to git add the gitignore file and then I'm going gitcommit -m for message. Step one add gitignore. Now to deploy to Heroku do we need a gitignore file? Technically no. You can deploy successfully without a gitignore file present but it's highly recommended and not just for Heroku. Never store passwords or credentials in a Git repo. Don't push them up, don't put them in to begin with. You want to keep those things private and safe. Adding those files to your gitignore is a great way to do that.

Casey Faist: Similarly, anything you don't want pushed up to Heroku is a great candidate for the gitignore file. Things like your local venv, which you don't want to deploy it to Heroku because Heroku sets you up and environment for Python automatically. Other things like files you don't want included or general cruft to keep your deploy clean goes in your gitignore file. Next up we want to modularize our Django settings. To do that we need to navigate to our getting started Django directory, it's right here and now that we're here we're going to add a new folder. You're going to call this folder settings and this does matter, so make sure that you do use the name settings and now we're going to grab our settings.py file and we're going to move it to our directory. Now, this naming will get confusing, so I'm going to rename this to base.py. I chose this name because this is going to serve as the base that all of our other settings configurations are going to pull from but if something like dev or local.py makes more sense to you, feel free to use that.

Casey Faist: Just change out base for what you name this file later on. By moving and renaming the settings file our Django application now has two broken references, let's fix them before we move on. The first is in the wsgi.py in your getting started folder. Open it up and on line 12 you'll see that it has a Django settings module default being set and what we're setting it to is gettingstarted.settings. This is what it used to be, but we just moved our settings file, so to fix this, all we have to do is add a dot and then since I named my file base, I'm just going to type out base. Perfect. Save that and then navigate one more directory up to the root level directory, find manage.py. On line six you'll see here the same default is being set for the Django settings module. Same thing here, we want to add .base to the end of this line. Then as always, save these changes and add them to Git.

Casey Faist: Local projects only have one environment to keep track of, your local machine but once you want to deploy to different places or stages, its important to keep track of what settings go where. Nesting our settings files this way it makes it easy for us to keep track of where those settings are as well as take advantage of Heroku's continuous delivery tool pipelines. So you have four stages you build in development stage. This is your local machine, Docker container, however you like to develop locally. Next in the review stage, you want to check if your test pass along with the full test suite of your code base. If that goes well, you merge to staging, which is where you probably have more realistic settings, maybe some test data available in order to more accurately predict how the change will impact your production settings. Lastly, if all that goes well, you push to production where the change is now live for your customers.

Casey Faist: Continuous delivery or CD workflows are designed to test your change in conditions progressively closer and closer to production and in more and more detail. If something breaks, this means it's easier to debug since you know exactly what settings it broke with and when you deploy, you can deploy it with confidence knowing that you've tested your change thoroughly and in conditions as close to your production settings as possible. Continuous delivery is a powerful workflow and can make all of the difference in your experience as a developer once you've productionized your application. Heroku can save you a lot of time here, we've already built the tools for you to have a continuous delivery workflow. From your dashboard on Heroku you can with the click of a button, set up a pipeline, add applications to staging and production as well as deploy. You can also, if needed, roll back.

Casey Faist: If you connect your GitHub repository, pipelines can also automatically deploy and test new PRs opened on your repo. By providing the tooling and automating these processes, Heroku's continuous delivery workflow is powerful enough to help you keep up with your development cycle. Modularizing your Django settings is a great way to take advantage of this continuous delivery workflow by splitting up your settings, whether you're deploying to Heroku or elsewhere but there's one more thing we have to do to base.py. Django static assets work best when you also use whitenoise to manage your static assets. It's really easy to add to your project. Start by copying this middleware line here, all of it, this time you want the quotes and the comma. Copy and now head back to base.py.

Casey Faist: We're looking for your middleware list, Django loads your middleware in the order that it's listed here, so you always want your security first but it's important to add whitenoise as the second step in this base file. In this project I have a handy reminder where it goes. Boop. Next, jump back to the workshop and grab this next code block, the static file storage setting. Copy it and in base.py we're going to scroll all the way down to the bottom. See we've loaded whitenoise as middleware but to actually use whitenoise compression, we need to set this variable and with that we're done with base.py, congratulations. Save your work and commit it to Git. Our base settings are complete but now we need our Heroku settings. If you come to the workshop.md, you can see I've already provided a template file that should work with most projects. You can see here at the top we are importing our base settings file. If you're using your own project, make sure that you update this line to match what your directory structure looks like. For now, grab this whole text block and copy it.

Casey Faist: Navigate to base.py and instead of changing that file, we're going to add a new file to the settings folder. I'm going to call it heroku.py and inside this file we're going to paste our template. You can see in this file the values that we're listing are the ones that we're overriding from our base settings, so these are the settings that will be different on Heroku. Now we're using one of my favorite packages to do this. It's Django-environ imported just as environ. This allows us to quickly and easily interface with the operating system environment without knowing much about it. It's got built-in type conversions such as this, it knows and will cast this environment variable to a list for us and in particular it's automatic database parsing. This is all we need in order to parse our Heroku Postgres database URL that we will be given. It's just really convenient, so I'm a big fan of this package. I recommend you check it out. But with that, save and our settings are complete.

Casey Faist: Awesome. That's all the work we need to do to get our application into 12 factored shape but there's three more files we need in order to deploy to Heroku. Requirements.txt is a standard Python dependency file. In addition to the packages your project already uses, there's a few you need to deploy to Heroku. If we take a look at the provided requirements.txt file, you can see these required packages here. Obviously we have Django but there are four other ones. We've already talked about Django-environ as well as whitenoise and we've already configured those but the other two are also important and needed for deploy. The first is Gunicorn or Gunacorn. This is the recommended WSGI server for Heroku. We'll take a look at configuring this in just a bit. The last one is psychopg2. This is a Postgres database adapter. You need it in your requirements.txt file to deploy but you don't need any code changes in order to activate it, we just need it installed in the environment.

Casey Faist: A quick note, I chose to keep things simple for the purpose of this demo but when you're ready to deploy your project to Heroku, consider freezing your dependencies. You can do this with pip freeze. This will make your build a little bit more predictable by locking your exact dependency versions into your Git repo. This is important because Heroku recycles your resources once a day. This is important for application health but if your dependencies aren't locked, you might find yourself deploying one version of Django one day and a new one the next. When Heroku cycles your application resources which are organized into containers called dynos, we're not just reinstalling your application. On every dyno cycle or dyno deploy, Heroku rebuilds your environment onto AWS resources, this doesn't just mean your Django app. This means all of the system packages that Heroku provides as well as your Python installation.

Casey Faist: Heroku will install a default Python version if you don't specify one, but if you want to pick your Python version, you'll need a runtime.txt file. You want to put it in the root level directory next your requirements, manage.py, gitignore and the rest. This file only needs one line formatted like this, a lowercase python, a dash, and then the major, minor and patch version that you want your application to run on. The last file we need to add is a file specific to Heroku. The Procfile. This is what we use to specify the processes our application should run. The processes specified in this file will automatically boot on deploy to Heroku. The Procfile also goes in the root level directory right next to your requirements and runtime file. Make sure to capitalize the P of Procfile otherwise Heroku might not recognize it.

Casey Faist: Now the first thing we're going to do is specify a release phase process. The release phase of deploy is the best place to run tasks like migrations or updates and that's what we're going to add here. So pythonmanage.py migrate. The other process we're going to add to this file is the web process, the most important process for any web application. The other process we're going to add today is a web process. This is a very important process for web application. This is where our Gunicorn config goes. You want to pass Gunicorn the same things you would if you were to run it locally. So we want to pass it our WSGI file, which is located in the getting started directory and then we're going to give it a bit of configuration. So we're going to pass it preload, this will ensure it can receive requests just a little bit faster and then we're going to specify that the log file should get routed to Heroku.

Casey Faist: Congratulations, you've updated your app and you are ready for deploy. Woo hoo! Take a second before moving on and just double check that you've saved and committed all of your changes to Git. Remember, we need those changes in the Git repo in order for them to successfully deploy. After that, let's get ready to make an app. The first step is to create our Heroku app. You can do this from the command line since we already installed the Heroku CLI at the beginning of this workshop. All you need to do is type in Heroku create. All right, time to make some magic. It is Heroku create time. Enigmatic mountain. Excellent. This has done a couple of things for us. First, you can see it's generated not only an app but a unique name, that's that "enigmatic mountain" there. Next, it's provided a unique URL, this is where our app will live on the web and lastly, it's added a Git remote to our Git repository.

Casey Faist: You can confirm by saying git remote and now you can see that Heroku is there, that's how we're going to deploy. Now, before we go anywhere, I'm just going to grab this URL. I'm going to copy it because we'll need it in a second. Remember when we created our heroku.py settings file? We used Django-environ to load environment variables into our settings config. Those settings config need to be present in our Heroku environment, so let's set those now. We'll be using the Heroku CLI for this but you can add and edit config variables from your dashboard directly. The Heroku CLI command we'll be using though is heroku config:set. This will take in your key-value pairs as argument and set in your Heroku runtime environment. First, let's configure our allowed hosts, so we want to type in a heroku config:set and allowedhosts= and then we're going to paste that URL we just got. Make sure to remove the slash on the end if you haven't already. Perfect.

Casey Faist: Next, let's set our Django settings module, so up arrow and Django settings module. This is what determines what settings configuration we use on this platform. So you will remember that we want to use the gettingstarted.settings and instead of base this time, we want the Heroku settings. Now we need to add the secret key as a config var to our application. We can use the same CLI command, heroku config:set to do this and make sure that you add a different secret key than you used locally. It can be anything. You may even want to check out the Python secrets module if you have specific secret key requirements. But the important thing is do not add your production secret key to Git ever and only share it with a password manager. Like Gandalf said, "Keep it secret, keep it safe." Now locally, Django is configured to use a SQLite database but we're productionizing. We need something a little bit more powerful, so it's time to provision our Postgres database.

Casey Faist: First, let's check if we have a database already. Heroku addons will tell us. Okay, so we do need one for our application. Makes sense, it's brand new. Now you can do this with the click of a button from your dashboard but I'm going to show you how to do it from your command line. heroku addon just like before and we're going to create a Heroku Postgres goal, Postgres sequel and we're going to use the hobby dev tier of database. We offer several tiers of Postgres database. This is the free tier, so you can play around with this without a credit card. Boom, Postgres database. And if you remember from the settings file, our application is expecting this database URL. Luckily, this command automatically adds this to our Heroku environment so we don't have to do anything more to connect our application to our Postgres database. It is time. Your code is ready, your Heroku app is configured, you are ready to deploy. This is the easy part.

Speaker 2: Yay.

Casey Faist: First, make sure your terminal is in the root level directory of your project. If you type ls on Mac or Linux, you should see that requirements.txt file and the runtime.txt file right there. Then just type out, git push heroku master and we take care of the rest. You'll see your build logs scrolling through your terminal. This will show you what we're installing on your behalf and where you are in the build process. You'll see the release phase as well that we specified earlier. Now, if you're not on the master branch, you made your own, no worries. You can still deploy your code to Heroku but you'll need to type out, git push heroku, your branch:master. The last step is to scale up our web process. This creates new dynos, however many you specify or copies of your code into our containers to handle more web traffic. You can do this from your dashboard or you can use the Heroku ps:scale command and we want to scale up our web process to one.

Casey Faist: Now to go freak out about your brand new website just type in heroku open and there you have it. Okay. If you hit some snags, don't worry, we have some tips that might help. Number one, are all of your changes saved and checked into Git? Number two, are your changes on master or are they on a different branch or a combination? Make sure that whatever you're deploying, all of your changes are in that Git branch. Number three, did you deploy from the root directory of your project? Did you also Heroku create from the root directory of your project? If not, this could absolutely cause a trip up. Number four, did you remove anything from the code in the provided demo that we didn't discuss? Number five, use your tools. What tools you ask? In addition to your build logs which will tell you whether your application successfully deployed or not, you have access to all logs produced by Heroku and by your application. You can get to them through a couple of ways but the quickest way is just to say, heroku logs tail.

Casey Faist: And this, let me make this screen a little bit bigger, is the live feedback from your application. If you switch over and click on something that'll get reflected here. You can see that we have our Heroku web one process, we have our app processes and then we have these router methods. You can see this one's a GET. All of this is here to help you debug any connection issues you may have. Another tool you have is the Heroku run bash command. This spins up a one time or one-off dyno container that you have direct access to from your console. So if I ls, you can see that this is my deployed application, it can be useful to check that what is up here matches what is locally on your machine. If not, you might see some issues. You can also check whether things work by spinning up a Python terminal and this is a full Python REPL, so anything you could do locally you can do here.

Casey Faist: Heroku has a wealth of technical documentation as well. Dev Center is where you'll find most of our technical how-to and supported technologies information and if you're having a technical issue, chances are someone else has asked the same question and been answered on our help docs. Use both of these resources to solve your problems as well as to learn about best practices when deploying to Heroku. And with that you have successfully productionized and deployed a Heroku app. Congratulations. I hope you had as much fun in this tutorial as I had putting it together. I have been Casey Faist, you have been amazing. Stay well and happy coding from your friends at Heroku.