Dylan Tweney
VentureBeat

3 takeaways from 3 big tech outages: NYSE, United, and WSJ

What a day! Yesterday morning, the New York Stock Exchange went offline, United Airlines grounded its entire fleet, and the Wall Street Journal website went offline. All three outages were related to technical glitches of one sort or another. And on top of that, the Chinese stock market tanked. That
Dylan Tweney 3 min read
Press here to panic.

Sign up for the weekly Dylan's Desk newsletters to get insights delivered right to your inbox.

What a day! Yesterday morning, the New York Stock Exchange went offline, United Airlines grounded its entire fleet, and the Wall Street Journal website went offline.

All three outages were related to technical glitches of one sort or another. And on top of that, the Chinese stock market tanked. That wasn’t a technical fault, but it didn’t stop people from lumping it together with the other three big news events of the day, adding in some horrendous commutes in New York and San Francisco and saying the sky is falling.

Or, maybe, don’t panic: These are all unrelated events, and coincidences happen.

Then again, maybe we should panic: These are symptoms of how badly software sucks, and it’s only going to get worse.

It’s tempting to join the chorus of extremist headlines, but I think it’s more productive to try to draw some useful lessons from the day.

First of all, the three technical outages were not related to hacks or cyberterrorism or any other malicious activity. The head of Homeland Security spoke with both United and the NYSE, and attested that the glitches were “not the result of any nefarious actor.” United blamed its hourlong outage on a “network connectivity error.” The WSJ similarly denied being hacked.

That said, cyberattacks do happen. Already in 2015 we’ve seen a few gigantic, disturbing hacks: The State Department was breached and lost control of millions of personnel records, including Social Security numbers; and Anthem Blue Cross got hacked and lost control of tens of millions of customer records. That’s following a horrendous 2014, in which Sony Pictures and JP Morgan got embarrassingly hacked, among many others. If you really want to scare yourself, check out Paul Sparrow’s Hackmageddon site, which collects timelines of all reported cyberattacks, month by month.

Lesson 1: You need to invest more in security. If the prospect of a technical glitch taking you offline for an hour or a day scares your CEO, just imagine how she’ll flip out if hackers abscond with millions of accounts, and the story gets written up in the WSJ — or VentureBeat.

Second, yesterday’s technical problems had no lasting effects. Although the NYSE is one of the world’s biggest stock exchanges, it only handles about 20 percent of the U.S. stock market. While it was down, other exchanges, like the NASDAQ, remained online. As a result, the nearly four-hour outage had almost no effect on the stock market.

United’s system crash caused a lot of grumbling among its customers, but the outage didn’t last very long and most of those customers probably reached their final destinations with only slight delays. The brand damage would have been much less if United didn’t already have a reputation among many people for indifferent customer service.

As for the WSJ, its outage came at a bad time. I can tell you from personal experience that journalists start tearing their hair out if their own site has technical problems at the same time they’re trying to cover a big news story. I can only imagine the yelling that was happening in the Journal‘s newsroom as the team discovered their site was borked at exactly the moment they were trying to cover United and NYSE. Fortunately the company quickly put up an alternate homepage, and got things rolling again in less than an hour. Today, things are back to normal on the site, and I’m sure the WSJ’s traffic suffered only a temporary dip yesterday.

Lesson 2: Big outages are survivable — if you’re prepared. Put in a disaster recovery plan and keep it up to date.

Third, don’t believe everything you read in the press. (Yes, I realize the irony of a journalist saying this.) But know that you may have to deal with the press and the public, even if their perceptions are wildly off base. Consider yesterday’s outages as a case study of what happens during an outage. Headline writers get excited, Twitter blows up, and suddenly you’re dealing with angry hashtags and aggressive inquiries from financial and business reporters.

Lesson 3: Make sure your disaster planning includes communicating with your customers (and the press) as quickly and openly as possible. Be on Twitter. Be on Facebook. Respond as quickly and as humanely as you can to angry customers and confused journalists, and you may be able to contain the damage.

The bottom line: Don’t panic. Don’t read too much into three big outages happening in a single day.

But do take these outages as a serious warning to take care of technical debt, invest in security, and make sure you have backup servers ready in case something really goes sideways.

Originally published on VentureBeat

Share
More from Dylan Tweney
VentureBeat

Can the Kindle Fire disrupt the tablet market? Not so fast

The arrival of Amazon’s Kindle Fire, a $199 tablet, in a market dominated by $500 models looks like an obvious case of price disruption. Not so fast, says Horace Dediu, an analyst at Asymco: Amazon’s margins are too thin to allow it to compete on the tablet’s core technology. It’s an interesting cou
Dylan Tweney 1 min read
VentureBeat

First iPhone in space to launch with last shuttle mission

When the final space shuttle mission launches later this year, two iPhone 4s will be on board. The iPhones will be running an experimental app called SpaceLab for iOS, designed by Odyssey Space Research. Once the space shuttle Atlantis docks with the International Space Station, crew members will us
Dylan Tweney 1 min read

Dylan Tweney

If you're bored, you're not paying attention

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to Dylan Tweney.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.