Blue-Green and Canary Deployments on Cloud Infrastructure
Blue-green and canary releases both reduce deployment risk, but they cost different things and fail in different ways. Here is how to choose and how to run them.
Every deployment is a bet that the new version works. Blue-green and canary releases are two different ways of making that bet smaller. Blue-green reduces the time you are exposed to a bad version. Canary reduces the number of users exposed to it. They are not interchangeable, and teams that adopt one because they read about it rather than because it fits their failure mode usually end up with the cost of the pattern and none of the benefit.
This guide covers the mechanics of both on cloud infrastructure, the parts that do not switch cleanly, and how to decide which one your system actually needs.
The release problem these patterns solve
Deploying in place means stopping the old version and starting the new one. If the new version is broken, every request fails until you notice, build a fix or a revert, and deploy again. The outage lasts as long as your detection plus your redeploy time, and both of those are usually longer than anyone estimates.
Both patterns attack that number differently. Blue-green keeps the old version running, so recovery is a traffic switch rather than a deployment. Canary sends a slice of traffic to the new version, so a broken release harms a fraction of users while you decide. Note what neither one fixes: a release that corrupts data is not undone by routing traffic elsewhere.
How blue-green works on cloud infrastructure
You run two complete environments. One serves production traffic, the other sits idle at the current or next version. Deploy to the idle one, verify it, then move traffic across. The old environment stays up for a defined window so a rollback is a second traffic move rather than a rebuild.
On cloud infrastructure the switch usually happens at one of three layers. At the load balancer, by changing which target pool receives traffic, which is fast and precise. At DNS, by repointing a record, which is simple but subject to resolver caching you do not control, so the switch is gradual and incomplete in ways that are hard to observe. At the service mesh or ingress layer, which gives the most control and adds the most moving parts.
Prefer the load balancer. DNS based switching is tempting because it works on any provider without special features, but a client caching the old address for an hour after you thought you had rolled back is a genuinely bad afternoon.
The cost of a duplicate environment
Blue-green means paying for two production sized environments, at least during the deployment window. Some teams keep the idle side scaled to zero and spin it up for releases, which cuts the bill and lengthens the deployment. Others keep both warm permanently and accept roughly double the compute spend on that tier. Whichever you choose, model it before you commit, using the approach in building a cloud budget, because this is one of the few architectural decisions with a direct and predictable line item attached.
How canary works and what it costs
A canary release sends a small share of traffic to the new version while the rest continues to the old one. If the metrics on the canary look like the metrics on the baseline, you widen the share. If they do not, you take the canary out.
The hard part is not the traffic split. It is deciding what “looks fine” means. A canary carrying a small slice of traffic generates a small number of events, and small samples are noisy. Error rate on a thousand requests tells you very little about a failure that affects one request in two thousand. You need either enough traffic to make the comparison statistically meaningful, or enough patience to leave the canary running long enough to accumulate it.
Canary also demands that both versions run simultaneously and correctly against the same backing services. That constrains what a release can change. Anything that is not backward compatible with the version still serving traffic cannot ship as a canary without a preparatory release first.
Rolling updates and where they fit
Rolling updates replace instances in batches, so there is no duplicate environment and no explicit traffic split. They are the default in most orchestrators and they are fine for the majority of releases. The limitation is rollback: once a batch is replaced, rolling back is another rolling update, so recovery time is the same order as deployment time.
A reasonable position is rolling updates as the default, canary for changes you are unsure about, and blue-green reserved for releases where recovery speed is the dominant concern. Running three patterns is more machinery than most teams want, so pick two.
Databases: the part that does not switch cleanly
Traffic shifting assumes the two versions are interchangeable from the data layer’s point of view. Schema changes break that assumption immediately, and this is where most blue-green and canary implementations quietly fail.
The discipline that makes it work is expand and contract. Ship the schema change first, in a form both versions tolerate: add the new column, leave the old one, write to both. Then ship the code that uses it. Then, in a later release, remove the old column once nothing reads it. Three deployments instead of one, which feels slow, and which means you can roll back any of them without a data migration.
Destructive migrations have no safe rollback via traffic switching. Treat them as a separate class of change with their own review, their own backup verification, and their own window. The failure design thinking in designing for failure in the cloud applies here more than anywhere else in your release process.
Traffic shifting mechanics
Whichever pattern you choose, a few details decide whether the switch is actually clean:
- Connection draining. Give in flight requests time to finish before the old targets stop receiving traffic. Without it, a switch produces a burst of errors that looks like the new version failing.
- Health checks that mean something. A check that returns 200 whenever the process is alive will happily declare a version healthy that cannot reach its database. Check a dependency.
- Session affinity. If users are pinned to a backend, a canary slice is not a random sample, and widening the split moves whole cohorts at once.
- Warm up. Caches, connection pools and runtime compilation mean a freshly started instance is slower than a settled one. Send it some traffic before you judge its latency.
Rollback as a tested path
The entire value of these patterns is the rollback, which is exactly the path least likely to have been exercised. Roll back on purpose, on a quiet day, and time it. Confirm that someone who is not the author of the change can execute it from the runbook alone.
Decide the rollback trigger before the release rather than during it. “Error rate above baseline for five minutes” is a trigger. “If it looks bad” is a discussion, and discussions during an incident take longer than the outage budget allows. Wire the trigger into the signals you already trust, built the way monitoring and alerting describes, so that the deciding metric is one you look at every day rather than one invented for the release.
Picking a provider account to run this on
These patterns need a few specific capabilities: a load balancer that supports weighted targets or at least fast target group changes, an API you can drive from automation, and enough instance headroom to run duplicate capacity without hitting a quota.
Hyperscaler platforms offer the richest traffic shifting features, including weighted routing managed for you, which is why teams running canaries at fine granularity tend to be on an AWS account or Google Cloud account. Smaller platforms generally give you a load balancer with pool membership control, which is entirely adequate for blue-green and for coarse canaries. A Linode account or similar will do this without trouble, and the AWS and Linode comparison sets out where the extra platform depth is worth paying for.
Check your quotas before the first blue-green release rather than during it. Doubling an environment doubles instance count, address count and sometimes load balancer count, and a release that fails because the account hit an instance limit is a frustrating way to learn about service limits.
Start with the pattern that matches your actual failure history. If your bad releases are usually obvious within seconds, blue-green gives you the fastest exit. If they are subtle and only visible at volume, canary is the one that will catch them.