In the ever-shifting landscape that is developing web applications, there are a few things that have remained relatively constant:
- It will always take longer to develop an application than anyone involved thinks it will.
- No matter when your deadline is, you will always run short on time towards the end.
- No matter what project management methodology you use, (or what software supports it) at crunch time, people will resort to sticking Post-it notes to your monitor. (…but that’s another show.)
- No piece of software will ever be perfect, but any “extra time” will generally be eaten by trying to make it such.
- Deployment (putting the application into production) will always be a rush job done at the last minute.
The first four things are more or less laws of nature as far as I am concerned. No matter how hard everyone works or how on the ball everyone is, software is hard to write and it’s even harder to say how long it will take. What isn’t a constant, though, is that last one, despite the fact that most people do it that way. In general, there is a fair amount of wiggle room in the area of what order things get done in.
Generally, when you’re building an app, the minute you have a loose idea of what’s getting built you want to build something and get it in front of people so they can start picking it apart. This idea, in fact, is at the core of a lot of very well-meaning project management methodologies. (I’m looking at you, Agile.) The sooner you can show something working, the better the people who sign the checks will feel about your project. (At least at the beginning, anyway.)
The flaw in this approach is that it gives everyone a false sense of security about the schedule. Even as work proceeds at a brisk pace, most people ignore that most crucial of steps: getting the thing to work in “real-life”. Deployment is left for the last possible second, when the project isn’t quite done, but must be pushed live. There’s generally a lot of hand-wringing and brow-mopping and a lot of impatient people at this stage. (A good PM might even build in “go live” time for this, but it will always be eaten up by last minute development.) In the worst case, there will be no actual deployment strategy and someone from the development team will end up dragging files from one machine to another, manually. Things that worked perfectly on all the developers machines will break catastrophically on the live servers. Someone will have forgotten to purchase a license for some critical piece of software the app needs to run. The firewall ports that the web server uses to talk to the database server won’t be open, and neither will the help desk of the hosting company. All of this is, of course, terrible. To paraphrase Scott Hanselman, “You get to drag files from your local machine to a server for deployment once. After that, if you’re not typing ‘deploy’ somewhere to push your application, YOU ARE STUPID.”
The second worst case is that an already under-the-gun developer will be pulled off some other very important task (you know, like, finishing the damned thing) to crap out some way of deploying the app to production. It will often end up being a brittle, undocumented kludge that some poor intern will have to sift through later. Everyone will hate it and it will reflect poorly on the team that spent six months developing an otherwise amazing application.
Why is deployment (and the setup therein) such a loathed task? I would say it’s because it’s a necessary but highly unglamorous task. It’s also one that very few people will get. Show your client the cool new feature you just added to the image gallery and they’ll buy you a pizza. Show them your one-button application deployment and they’ll give you a blank stare. For this reason, deployment has always been a last-minute rush job.
I would like to propose a better way. Deploy first. By “first”, I mean that as soon as you have checked in the very skeleton of your application into your version control system, figure out how and where and by what method you will put the thing into a production environment. Then do it. Do it a bunch of times. Get it so that it’s a completely natural, totally painless process that you don’t even have to think about. While it’s my opinion that a deployment-first strategy is good practice for any project, I think it’s absolutely critical to any project using a new or unfamiliar development platform. (Also, as an aside, if you do not have a version control system, please stop reading this. Clean out your desk, leave your ID card with security, walk out of the building and don’t look back. Seriously. Shame on you.)
To deploy your app, you will generally need to take care of all (or most) of the following:
- setup and configuration of production machines (be they real or virtual)
- installation of prerequisite software for your application (web servers, application platforms, RDBMSes, etc.)
- load balancing
- network configuration
- development of a deployment system
If you get this all squared away first, you can merrily write code up until the last possible second, because when you’re done, you push a button and the site goes live. Maybe there will be a speed bump or two in that process, but you won’t be starting from scratch.
Make no mistake. Writing software may be hard, but deployment is always harder. It also (generally) involves coordinating lots of different people, often at different companies, many of whom will not share your sense of urgency. Deploying first exposes a bunch of issues that you can square away long before they become emergencies.
That last item on my list there is often a source of confusion for a lot of people. Why would you “develop” a deployment system? There’s a lot of tools that do this already, right? Well, sure, but you have to set them up and configure them, and in most cases, write scripts that will tell them what to do. This is often a lot less trivial than the makers of those tools would have you believe. Take the time and figure out what you’re going to use and how your going to use it before you even really start developing.
As for how you go about building deployment systems, there are lots and lots of options. The best one for your project will largely depend on what you’re using. In most cases, you can get your build tool (Ant, Maven, NAnt, Capistrano, etc.) to do it for you. In a lot of cases this can get overly complicated, though. My personal favorite way is to write a short, project-specific shell script (though I recently started using Python for this) with good inline comments. Whatever you use, here’s a loose set of steps for any deployment script:
- Pull a copy of the latest code from the version control system (never deploy off of your current development tree)
- Increment the version number (I like using a text file called VERSION in the root of my project)
- tag (or whatever your VCS calls it) the revision with your version number
- copy required libraries into the build tree
- create compressed archive of your build
- transfer the archive to the production system
- unpack the archive into the right place (via remote, perhaps with passwordless SSH or Powershell. Some build tools have this functionality as well.)
- perform any required administration tasks that a new build would need (update database structure, restart web server, etc.)
- email a notification that a build was pushed out
You may need more steps, you may need fewer. Whatever you can reasonably automate, do it. Every step you make automatic will be one less manual headache at the 11th hour. Also, the more manual steps you remove, the more you reduce the likelihood of human error and dependency on a particular human to do deployment.
I hope this will inspire you to think about incorporating deployment-first development into your next project. I can tell you that doing things this way has greatly improved my life as a developer, especially at crunch time.
Unless you’ve been hiding under a rock, you probably know about Amazon S3 by now. As far as I’m concerned, it’s about as cheap and easy to use as cloud-based storage gets. If you’re writing your own application, there’s a very simple API for getting stuff into and out of it. If you’ve got a baked application, it’s a bit more complicated. Sometimes you want to take advantage of the cost savings and scalability of S3, but can’t (or don’t want to) modify the web application to use the s3 API directly.
Be Social