I was sitting in a conference on enterprise infrastructure this week when I realized that the generational shift long promised by cloud advocates is finally, irreversibly underway.
That shift is away from “legacy” data centers built on x86 servers, VMware-managed hypervisors, SQL databases from Oracle, and storage hardware provided by EMC. Replacing all that are web-scale (or at least wannabe web-scale) technologies based on containers, commodity hardware, NoSQL databases of various kinds, and flash storage. The new infrastructure is cheaper, easier to scale up to large volumes of data and computation, and more flexible and agile.
But who really cares about that architecture, except the billion-dollar infrastructure companies that are about to take a giant hit in their valuations? And by that I mean Dell, HP, IBM, Cisco, Oracle, and, yes, EMC (which Dell is in the process of trying to buy). These companies might not quite be the walking dead, as Wired called them this week, but they are certainly headed for a world of hurt, which is why Dell is trying to buy EMC: It needs to shore up its legacy business.
Who cares about them, except their shareholders? Because it’s now possible to build a billion-dollar company without ever setting foot in a data center. You don’t have to care whether the datacenter is using HP and Dell hardware or some cheap commodity CPUs built to spec by the cheapest possible manufacturer. All you need are virtual servers you can spin up on a moment’s notice, the ability to deploy containerized apps into that environment, and support for the unstructured databases you need to handle the massive influx of bits you’re about to start collecting and will need to analyze.
Netflix shows what that looks like, and why — for now — Amazon owns such a big piece of that future.
Neil Hunt, the chief product officer and vice president of engineering for Netflix, was speaking at the Engineering Summit on Infrastructure, which had been organized by Engineering Capital, a small, enterprise-focused VC fund. Hunt talked about Netflix’s longstanding use of Amazon Web Services, the market and technology leader in cloud services. But it’s not just Netflix, Hunt said: Everyone is moving toward AWS.
“AWS is now the basic layer of compute services,” said Hunt.
Netflix is not just heavily reliant on AWS — it’s about to become completely dependent. Hunt plans to power down his company’s last data center this year, at which point Netflix will be running almost entirely on outsourced cloud infrastructures, mostly operated by Amazon. (It’ll still run its own content delivery network — CDN.)
Note that this timeline is new. Netflix originally said it would shut down its last datacenter in 2014, and then again this past summer, but the future sometimes comes a little slower than expected. That’s one aspect of enterprise infrastructure that will probably never change.
Still, Hunt would be happy to be less dependent on a single vendor: “That’s a somewhat uncomfortable place: To be dependent on a partner who is also competing with you,” Hunt said, referring to the fact that Amazon also sells a streaming video service.
But up to now, Hunt hasn’t found a single provider that matches Amazon in terms of its capabilities and scope.
“AWS is years ahead of Azure and a year or two ahead of Google in terms of the features and levels of abstraction they offer,” Hunt said.
“It’s getting closer, but Amazon keeps raising the bar in terms of AWS features.”
Gleb Budman, the CEO of Backblaze, which also recently began providing storage services that compete with AWS, asked Hunt if he’d consider using other cloud providers, even piecemeal. For the most part, Hunt said, the answer was no. Apart from a few tests here and there (Netflix is backing up data to Google, for instance), the company is almost entirely based on AWS.
So is the battle for the next generation over? Hardly. AWS has an enormous head start, but there is still no standardization of cloud services — something that Hunt believes will be necessary.
Hunt looks forward to a day when there is more standardization among computing components — and, by extension, more competition for AWS.
“Let’s get it right. Let’s make a standard toolkit that software engineers use when building software, just like hardware engineers use when building a bridge,” Hunt said.
“Then we’ll see an incredible increase in productivity.”
—
