Continuous Delivery Pipelines - How to Build Better Software Faster
“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software”
Continuous Delivery is founded upon three key ideas: • The reliable, repeatable production of high quality software. • The application of scientific principles, experimentation, feedback and learning. • The Deployment Pipeline as a mechanism to organise and automate the development process.
Discard Release Candidates on any failing test, and so reject changes that are not fit for production. • Carry out all necessary testing on all Release Candidates, so that there is no more work to do. • Complete this cycle multiple times a day, so that we gain timely, valuable insight into the quality of our work. • Generate a Releasable Outcome on successful completion, so that our software is ready to be delivered into the hands of its users.
Continuous Delivery teams work collaboratively, sharing responsibility for their development process. They do not divide up the responsibilities for designing code, writing code, testing code and deploying systems into production, and they do not ‘throw changes over a wall’ to the next person in the process.
The Deployment Pipeline is the ONLY route to production: all new code, and all changes to existing code, will be tested, audited, traced and recorded through the Pipeline.
Tags: #software
It encourages design for testability: testable code has the same properties as high-quality code.
The characteristics of testable code are: • Simple, efficient, and easy to read and maintain • More modular, more loosely coupled, and with better separation of concerns • Higher-cohesion and better abstraction • The code works!
TDD is less about testing and much more about good design.
The simplest approach to data migration is to apply it as an automated, integral part of deployment, so that every time the system is deployed, the migration will run.
Branching is about hiding information: dividing up the work, and working on it independently, in ways that team members cannot see each others’ work. Branching comes with the risk that one person’s work may be incompatible with the changes of their team-mates.
If branches exist at all, they should be tiny, few in number, and short-lived, i.e: merged at least once per day. When we are working well, we will merge every 10, or 15 minutes, to get fast feedback on our changes and optimise the Deployment Pipeline to be efficient and thorough.
The goal of the Commit Stage is to achieve a high level of confidence that our changes are good enough to proceed, and to achieve that confidence as quickly as we can.
Tags: #computer-science #efficient-software
When working well, the developer will be committing changes every 10-15 minutes. To achieve that level of productivity, to get fast feedback in under 5 minutes, and be able to quickly revert and correct any defects, means working on small pieces of code and working in small steps.
Making small, simple, frequent changes to our software is less risky than making big, complicated, infrequent changes. • Simpler changes are easier to understand and easier to revert if necessary. • Smaller steps mean smaller mistakes, faster feedback and the opportunity to learn and improve more quickly. • Smaller steps can be completed more quickly and help us to achieve our Continuous Delivery goal of our software always being in a releasable state.
If we commit a change that causes a test to fail, then, here comes another rule of thumb… Allow 10 minutes to commit a fix or revert the change.
Commit Stage tests should provide quality feedback to the developer in under 5 minutes. • If a test fails, we allow 10 minutes to commit a fix or revert the change. • If a team-mate introduces a failure, and is not around to fix the problem, we revert their change. • If the code passes the Commit Stage tests, the developer moves onto new useful work. • We work in small steps and aim to commit at least once per day. • Package the results of a successful Commit Stage to create a Release Candidate. • Release Candidates are stored in the Artifact Repository.
Effective Acceptance Tests: • Are written from the perspective of an external user of the system • Evaluate the system in life-like scenarios • Are evaluated in production-like test environments • Interact with the System Under Test (SUT) through public interfaces (no back-door access for tests) • Focus only on WHAT the system does, not HOW it does it • Are part of a systemic, strategic approach to testing and Continuous Delivery
The Acceptance Tests are business facing - written from the perspective of external consumers of the system. Acceptance Tests are designed to support programming and guide the development of code to meet the users’ need.
The primary characteristic of Manual Testing in Continuous Delivery is really that Manual Tests are one-off explorations of the system.
Performance Tests typically cover: • Usability: How slow is the system from the users’ point of view? Is it annoying to use? • Throughput: How much stuff can our system process? • Latency: How long does it take to get results back? • Soak-Testing: Long-running tests to find out if protracted use causes any unforeseen problems.
To prepare the environment ready for Data Migration Tests, we follow the same four-step process as we have for other test stages, i.e: 1. Configure Environment 2. Deploy Release Candidate 3. Smoke Test / Health Check 4. Run Data Migration Tests
The usual Continuous Delivery principles apply to management of data in the Deployment Pipeline, i.e: • Make changes in small steps • Test everything to make sure the changes are safe • Design to support change • Automate as much as possible, and • Version everything
Employ NO manual intervention in the configuration and deployment of data. • Automate tests to validate data, with respect to Data Migration. • Use Database Refactoring techniques. (The book “Refactoring Databases” by Scott Ambler, describes these in detail.) • Version schemas with monotonically increasing sequence numbers: each change in structure ticks the sequence. • Record every small change in a delta script identified by that sequence number. • Store the desired version of the schema, along with the application. • Store the current version of the schema along with the data.
Blue/Green Deployment We work with two different versions of the system: we update one while the other is in operation, and switch over when we are ready to deploy.
Canary Release We start by deploying into low risk: low volume environments and, once assured that the changes are safe, we progressively deploy into more and higher risk environments. Netflix use a combination of these strategies and deploy new features to locations around the world where it is the middle of the night and usage is at its lowest. This is monitored as usage increases through the day, so deployment can be stopped if there is a problem, before prime-time usage.
We are able to be experimental, to try out different versions of our system, discover what our customers make of our software and what they prefer. • We can release new features to our customers fast, frequently and safely, to be the best in our market. • The Deployment Pipeline is the primary tool for Digital Disruption.
The Deployment Pipeline is an organising mechanism around which we can build an effective, efficient, high-quality development approach.