Why Netflix Took Seven Years to Move to the Cloud, and Rebuilt Instead of Copying: A Cloud Migration Case Study

Real-world case study · Cloud Migration

“Move to the cloud” is often treated as a single project. Netflix’s own account shows it was a seven-year redesign of how the company builds and runs software, and that the decision not to take the shortcut was the reason it worked.

At a glance

• Company: Netflix
• Industry: Streaming entertainment (DVD rental at the start of the migration)
• Country: United States, operating globally
• Period: August 2008 to January 2016
• Technology: Amazon Web Services, microservices, NoSQL databases, continuous delivery
• Source type: Mostly Netflix’s own published accounts

The challenge

Netflix describes the trigger plainly: in August 2008 a major database corruption meant it could not ship DVDs to members for three days. The lesson the company drew was that it needed to move away from vertically scaled systems with single points of failure towards horizontally scalable, distributed systems.

The situation before

Netflix ran a monolithic application in its own data centres. Its 2016 post describes “centralized release coordination and multi-week hardware provisioning cycles”. Growth was also hard to predict, which made it difficult to plan data-centre capacity in advance.

The solution

Netflix chose AWS. In a December 2010 post, engineer John Ciancutti gave four reasons: the need to re-architect created a moment to decide; using AWS for “undifferentiated heavy lifting” let engineers focus on the product; growth in members and devices was hard to forecast; and Netflix believed the industry would move to cloud platforms.

Crucially, it chose not to lift and shift. In the company’s words, moving systems unchanged means “you end up moving all the problems and limitations of the data center along with it.”

Implementation

• Rebuilt the application from a monolith into hundreds of microservices
• Moved data models to distributed NoSQL databases
• Replaced central release coordination with continuous delivery and DevOps practices, so teams could deploy independently
• Tested resilience by deliberately triggering failures in production with tools such as Chaos Monkey, as reported by InfoQ
• Migrated in stages over seven years; by late 2010 most member traffic already ran on AWS, and the last data-centre pieces of the streaming service were retired in early January 2016

Documented results

Netflix reported the following in its January 2016 post:

• Eight times as many members as in 2008
• Overall viewing up by three orders of magnitude over eight years
• Service availability approaching four nines (99.99%)
• Cloud costs per streaming start “a fraction of those in the data center”
• Ability to expand to more than 130 new countries on 6 January 2016

These results come from Netflix itself rather than an independent audit. Netflix also acknowledges that it “hit some inevitable rough patches in the cloud, especially in the earlier days of cloud migration”.

Why this matters

The cloud did not make Netflix reliable on its own. The redesign did: many small services that fail independently, automated deployment and teams that own their own services. A lift-and-shift would have been faster to start but would have carried the old single points of failure into the new environment.

For a growing business, the equivalent choice is whether to copy an old server and its software into the cloud, or to use the move to fix what is fragile.

What businesses can learn

• Name the problem you are solving. Netflix moved to remove single points of failure, not to follow a trend.
• Do not copy problems into the cloud. Review what is fragile before migrating it.
• Migrate in stages. Move and stabilise one service at a time.
• Change how you release software, too. Smaller, more frequent changes reduce risk.
• Plan for growth you cannot forecast. Elastic capacity is most valuable when demand is uncertain.

Raymond Cardoza Enterprises perspective

Most small and mid-sized businesses do not need Netflix’s architecture. They do need the same discipline: an inventory of what they run, a clear reason for moving each system, a decision on whether to move, replace or rebuild it, and backups and access controls in place before the switch. That assessment is part of our business solutions and IT outsourcing work, and of the SME digital transformation approach.

Sources and verification

Every figure in this case study comes from the sources below. Primary sources are listed first.

1. Netflix (Feb 2016): Completing the Netflix Cloud Migration
2. Netflix Technology Blog (Dec 2010): Four Reasons We Choose Amazon’s Cloud as Our Computing Platform
3. InfoQ (Mar 2016): Netflix Cloud Migration