Over the last couple of years we have been reading a lot about Continuous Delivery (CD). A fantastic concept that can really propel organisations forwards by enabling fast, incremental value add to products. There are a number of different concepts here that I’d like to address as they all interlink but I have yet to have read any article that links them all together. If there is one, please let me know in the comments section, below.
CD allows us to repeatedly and consistently execute the deployment and monitoring steps necessary to deliver new functionality to a system in production in an automated fashion. A more thorough definition can be found on Wikipedia. Generally speaking, CD is a concept and strategy that has had real world implications and benefits. What is important to note is that, like all strategies, they can be executed well or badly. So to give CD adoption the greatest chance of success, there are a number of prior important factors that must be in place. Let’s call these the foundations, which are:
- A comprehensive suite of automated acceptance and unit tests integrated with the build process for the service in question (try PiTest if you think you have good tests).
- Services that do one thing, and one thing only. The concept of Micro Service Architectures has been about for a while, some people even saying that 200 lines of code is the most any service should contain. In my opinion, the guiding principle is a service that has groups of logically related functions that don’t break the Single Responsibility Principle and is not too large to conceptualise. A great description is from 21 mins 41 secs in this presentation: Micro Services.
- Small stories and not Business Accepting those stories until they are in production makes rollback easier. This is particularly important when rolling out a CD implementation for the first time.
- Prepare the team for CD so that continually deploying to production is second nature.
It is this last point, I think, that is the most important and will make or break CD adoption. This is what I refer to as the missing piece of the puzzle.
In most organisations, deployments only happen several times per week. With the underpinning tools of CD, we are expecting to get to tens, if not hundreds, of deployments per day. Moving an engineering culture, both at a team and organisational level, to the mindset required for CD adoption can be challenging. So what is the best way to do it? At Shopzilla, we started with CD adoption in mind for the future, so we took steps to prepare the Inventory Engineering team for this expectation (note, depending on your organisation, you may already have some or all of these implemented):
- Removal of all restrictions for deployment to production so that the team that built the code deploys to production.
- Stories are not BA’d until they are in production – this has the knock on effect of ensuring that tests pass, are thorough and that the Engineer(s) that wrote the feature have a high degree of confidence that the feature will work in production: the team sense of ownership is strong. Of course, all the other benefits of a good test suite come part and parcel.
- Stories are kept small so that the life of a story is only a couple of days at most until it is deployed to production. Of course, not all features are like this because there are some that are low risk (feature additions, modifications in a single service) to high risk (e.g. database schema change that may affect more than one service), so strategise accordingly.
- The architecture supports CD (see this presentation by Ancestry.com as an example).
- Engineers deploy several times per day manually, normally as soon as a story has passed Quality Assurance – automated or manual depending on whether the feature is low or high risk.
- Product Owners get used to having software go live many times per day and work with the Engineers to break down features into deployable increments with feature toggles.
Doing the above over a period of time changes the expectation of a team, from Product Owners to Engineers, so that deploying to production several times per day becomes the norm. When this stage is reached, there is a fantastic opportunity to underpin the process with some quality tools that will provide automation.
At Shopzilla where I am currently, we have reached the stage where we have changed the Engineering mindset so that CD is seen as a valuable adoption pattern. Another blog post will follow recounting our experiences of this as we implement over the next few months.
So to conclude, tools do not enable CD adoption, it is the mindset of the team. Work on this first and the rest will follow.