Issue

As the Nuve team became more comfortable releasing features in pieces, managing scope creep, and testing effectively as we went; our development pace and quality increased significantly. This meant that more developers were adding new productive changes to the codebase for every release.

This led to a subtle shift: almost every release included changes from multiple developers, even though we were still releasing one or two times per week. This meant that each release involved changes that were made by someone other than the person who was releasing the code.

Don’t get us wrong, this is a great problem to have, but we still needed a way to provide the person who was leading the release with a sense of comfort that they would release all changes without issue.

Solution

There were two questions that we needed to address:

1. Who runs the release when multiple people have made changes?
2. How does that person release with confidence?

Our solutions were simple and low tech.

Question 1: Who Runs the Release

The person who made the highest risk change within the release runs the release.

Generally this is the person who made the most changes, but not always. For example, if a person changed a whole bunch of marketing copy, they probably changed a lot of code. But those changes are very low risk to the to the release process. Changing text is no big deal.

Whereas if somebody changed a whole bunch of user data as a part of a release, that might involve very little code but could be very high risk. So that part of the release would have to run smoothly and the person who added those changes should run it.

Question 2: How Do They Release With Confidence?

Each developer needs to write release notes for changes that must happen outside of the normal release process.

A “normal” release process will vary team to team, but ours was essentially as follows . . .

First, release the code that runs on our servers:

• Change the back end code
• Run routine database updates
• Restart to ensure changes take effect

Second, release the code that runs in the browser:

• Change the front end code
• Restart to ensure changes take effect

So that was our starting point. If the release required anything outside of that process, say running a script or installing something new, that had to be added to the release notes. And it had to be added in a foolproof manner. Something like this:

# Release Notes

#######################
# Server Side Release
#######################
# After pulling code and making routine database updates, run
python3 .../scripts/post_release_script.py

#######################
# Browser Code Release
#######################
# After pulling code, before be sure to run
make build

These aren’t actual release notes, but they should get the point across. There should be literally zero judgment calls during the release process.

The best possible process is foolproof. Like, copy/paste this line and press enter, then copy/paste that line and press enter.

Results

This process led to peace of mind for the developer who ran the release. They could trust that they had all the steps to run the release documented clearly.

Though we used to have minor issues with releases in our early days as a team (”Oh no! I forgot to XYZ so something is broken. I’ll go fix that!”). We haven’t had an issue across dozens of releases after implementing this process.