How to Beat the High Cost of Storage

In an ancient Chinese legend, the emperor offers a reward to the man who has just invented the game of chess. The inventor modestly asks for one grain of rice on the chessboard’s first square, two grains on the next square, four grains on the third, and so on, doubling every square. The emperor, thinking this a swell bargain, grants his request — only to realize later that the 64th square would have enough grains of rice to cover the entire earth. He angrily orders the inventor beheaded.

A similar scenario is being played out inside your company — not with grains of rice (or beheadings, one hopes), but with bits of data. According to research conducted last year by the University of California at Berkeley, the amount of digital information produced in the world is doubling as often as every two years. At many companies the growth is even faster, with data doubling each year, thanks to the reams of information generated by the Web. All those e-business tools — your databases, customer-relationship management systems, and personalization servers, not to mention your website’s content, graphics, and audio and video files — are kicking up a veritable tsunami of data.

Data storage is fast becoming the biggest expense in large companies’ IT budgets, gobbling up as much as 30 percent of capital expenditures. Which means that if you need to hold down IT costs, storage should be the first place you look. It may not be the sexiest topic, but when you consider that you might be able to cut 10 to 30 percent of your budget by outsourcing your storage needs, you may begin to see the allure.

With the cost of the traditional means of storage — hard discs attached to file servers — dropping, you might wonder where the problem lies. After all, you can now buy PC hard discs for $7 to $8 per gigabyte, about half of last year’s price. Shouldn’t the falling cost balance out the rising need for storage capacity?

In a word, no. And the reason is that hard discs alone can’t efficiently handle all the data that companies are churning out these days. As a result, many companies are deploying more complicated solutions such as network-attached storage (NAS) and storage-area networks (SANs), both of which pool data so that it can be centrally managed and shared by all the servers. On the low end, NAS devices are high-speed servers dedicated solely to storing data. They can be plugged into the network wherever they are needed, enabling you to add storage capacity quickly. SANs offer an even bigger, more efficient data vault, with high-end storage devices plugged into a special high-speed network that’s separate from your regular local-area network. SANs essentially operate as their own mini-network of data servers.

Both of these technologies improve the performance of your network by delivering data more efficiently and quickly. But at $50 or more per gigabyte for NAS devices and as much as $300 per gigabyte for SANs, these solutions don’t come cheap. On top of all that, you have to pay for maintenance, backups, recovery of data when a hard drive crashes, and even electric power to keep all the drives spinning. When you factor in all of the expenses, the purchase price amounts to just 20 percent of the total cost of owning your own storage system.

What’s a budget-conscious company to do? My vote: Outsource the whole mess to a storage service provider (SSP) such as IBM Global Services, EDS, Compaq, or newcomer StorageNetworks. SSPs provide storage capacity and manage it for you, either in your own data center, in the co-location facility where your site is hosted, or at the SSP’s own data center. In return you pay only for what you use — typically $20 to $70 per gigabyte per month, depending on how much data you have, what kind of backups you need, and how much bandwidth you connect to your data.

Outsourcing data storage is a relatively new option, but the industry is growing fast: This year, according to research firm IDC, the storage-services market in the United States is expected to grow to a healthy $500 million, up from $140 million last year. The reason is that outsourcing storage is faster to deploy, easier to manage, and often much cheaper than building your own storage infrastructure. When Web portal Terra Lycos needed to move 5.5 terabytes of data — spread across 30 network-attached storage devices — from its Tripod subsidiary in Jersey City, N.J., to company headquarters in Waltham, Mass., CTO/CIO Tim Wright knew that his team couldn’t handle it alone. The difficulty and expense of buying hardware, deploying it, and then transferring all the data was, as he puts it, “daunting.” Instead, he turned the whole project over to StorageNetworks, which built a new storage infrastructure for Tripod and moved the data in just three hours.

Wright has found that the outsourced storage system is indeed faster and cheaper than the old system. The Terra Lycos data handled by StorageNetworks has since swelled to 20 terabytes, yet Wright estimates that he has shaved 10 percent off his storage costs while improving the overall performance of Terra Lycos sites. Even bigger savings are possible: Merrill Lynch claims to have cut its storage costs in half by outsourcing its needs to StorageNetworks.

Storage services can also be a good way to minimize up-front capital outlay on new ventures. Take ChartOne, a company in Dallas that digitizes medical records for hospitals. When it started a new online version of its service, outsourcing storage to IBM wasn’t much of a decision — it was more like the only available option, since building data centers would have been prohibitively expensive. “Even with a couple million bucks,” says Michael Sanderson, executive vice president at ChartOne, “we would only have been able to do about 50 percent of what we needed.”

Bear in mind, however, that outsourcing storage isn’t always the cheapest option. It doesn’t make sense to outsource unless you have a huge amount of data to deal with, at which point volume discounts will bring your monthly cost per gigabyte down to less than the cost of doing it yourself. Right now, that minimum level is about 8 terabytes, reckons StorageNetworks co-founder and CTO Bill Miller, although it may drop as the cost of outsourced storage does.

If you do decide to outsource, pick your partner carefully. There is stiff competition among SSPs — especially now that heavyweights like IBM have entered the game — so consolidation is inevitable. “We anticipate that most pure-play storage service providers are going to go out of business,” says Gartner Group senior analyst Adam Couture. If you don’t want to explain to the boss what happened to all that data when your SSP went bust, avoid the startups unless you can be absolutely certain of their continued viability. One notable exception is StorageNetworks, which is publicly held and financially stable.

Of course, competition has its benefits too. “SSP prices can only come down,” predicts IDC program manager Doug Chandler. That’s good news for e-business: There may be a place to put all that data after all. Which means that e-business managers, unlike the hapless chess master, just might get to keep their heads.

Dylan Tweney ( is a contributing writer for eCompany Now. Sign up for e-mail delivery of his weekly Web column at

Link: How to Beat the High Cost of Storage

Link broken? Try the Wayback Machine.

How to Beat the High Cost of Storage

Your Employees Love IM. Should You Worry?

Psssst. A word of warning: Your company is being invaded by a seemingly innocuous consumer technology that could compromise your deepest corporate secrets, render you vulnerable to lawsuits, and adversely affect employee productivity.

The name of this Trojan horse? Instant messaging. And it’s your employees who are bringing it inside the firewall. These text-based, consumer chat programs, a longtime fetish of teenagers and computer geeks, have gone mainstream and invaded corporate America in recent months. IM’s spread to the business scene poses a variety of problems, but also underscores its potential as a communications tool no less powerful than e-mail or the old-fashioned phone call.

The free, consumer programs – including market leaders AOL Instant Messenger (AIM), AOL-owned ICQ, Yahoo Messenger, and Microsoft’s MSN Messenger — are simple Internet tools intended for keeping friends and family in touch with each other. Each instant-messaging program features some form of “buddy list,” a window showing which of your friends and colleagues are currently online. Click on a buddy’s name and you can “talk” with that person by typing on your keyboard. After you type each line and click Send, the message immediately appears on your correspondent’s screen (for more on how instant messaging works, see “Getting the (Instant) Message“). It is, without a doubt, a very effective way to stay in touch, ask quick questions, or gossip.

Chances are, your employees already use instant messaging of some form. According to research firm IDC, 5.5 million people now send instant messages at work. Further, Jupiter Media Metrix’s February study of 60,000 U.S. Internet users found that workers with AOL Instant Messenger spent an average of 6 hours and 20 minutes per month actively messaging while on the job. That’s significantly more than the four hours per month AIM users spent chatting at home. The Jupiter report does not indicate how much of that office time on AIM is used for business as opposed to personal matters, but it’s reasonable to guess that most folks at work use it to exchange some notes with friends and family. Typically, employees download and install the free tools from the Web without any supervision from their employers. While there are commercial IM programs designed for corporations, such as Lotus SameTime and Microsoft Exchange 2000 (more on those later), they have far fewer users, according to Jupiter.

So what’s the problem with free IM programs? Productivity, for starters. Why are your employees spending six hours a month on instant messaging at work? Are they goofing off and gossiping with their friends? Or are they using IM as a new means of business communication? It’s probably a bit of both, and ironically, it’s the latter of these possibilities that’s especially worrisome.

If your employees are conducting business via instant messages, they should know that the conversations are vulnerable to interception. That’s because the most popular chat tools lack any encryption technology to encode communications. Worse yet, IM programs can record your chats (ICQ saves messages on your PC by default; other programs don’t keep continuous logs, but do allow users to save conversations). If those recorded chats fall into the wrong hands, you could find your company secrets posted all over the Internet. In other words, consumer instant messaging can easily undercut all that money your company spends to ensure that corporate e-mail is secure. Sam Jain, CEO of Internet startup eFront, based in Costa Mesa, Calif., learned that lesson the hard way. In March, Jain’s ICQ logs, including sensitive contract negotiations, were stolen and published on the Web, embarrassing Jain and the company. eFront is still investigating the incident but suspects it was an inside job.

Litigators are also wise to the potential of IM logs. Just like e-mail, the logs can be subpoenaed in the event of litigation. “Attorneys are getting more savvy about instant messaging,” says Michael R. Overly, a partner at law firm Foley & Lardner. “If a deal blows up, or if an employee gets terminated, attorneys will ask for IM records, along with e-mail records, faxes, and other communications.” The bottom line: Anything you say in an IM chat may be used against you.

Perhaps less crucial, but still important to anyone interested in using consumer IM for business communication: It’s not reliable. Most IM tools depend on a central server to act as a switchboard, routing messages and keeping track of who’s online. As with any other Internet application, if that server goes off-line, so does your instant-messaging capability. Outages are a frequent occurrence. In March, for instance, AOL Instant Messenger went out twice for several hours. If the idea of relying on AOL, MSN, or Yahoo for a critical piece of your company’s communications infrastructure doesn’t make you nervous, I don’t know what will.

Still, using IM at the office does have significant advantages, so long as it is a planned part of your infrastructure. For brief, immediate communications, it beats a phone call or e-mail. At IBM, the company’s more than 300,000 employees have used SameTime, a corporate IM program from IBM’s Lotus subsidiary, for several years to conduct virtual meetings, augment telephone conversations, and communicate quickly with remote employees. “If we were to shut down instant messaging at IBM, I think we would have mutiny,” says John Patrick, VP for Internet technology at IBM. “I don’t know how to put a dollars-and-cents figure on it, but the impact on productivity has been profound.”

At application service provider Vobix, based in Louisville, Ky., MSN Messenger has turned into a critical communications tool for the company’s entire 75-person staff — in many cases supplanting phone conversations. Managers often use IM to ask their employees for additional information while in the middle of executive meetings, for instance. Think of it as a high-tech version of passing notes under the table. “We use IM religiously,” says Vobix spokesman Jon Reischel. “It plays a pretty big role in having us all keep in touch with each other.”

There is little question that IM will be a major part of corporate communications in the years ahead. But if you’re going to use it, heed the Defogger’s advice and steer clear of the kinds of free programs that your employees are probably already using. In addition to their lack of security and manageability, these tools generally cannot communicate with one another across platforms, thanks to protracted “IM wars” between AOL, Microsoft, and Yahoo. Efforts to come up with a single IM standard, such as IMUnified and, have not made much headway so far.

Furthermore, AOL, MSN, and Yahoo are not pursuing the enterprise market for IM, and have no plans to support corporate users (though, in my opinion, this is a huge mistake). Microsoft, through its Hailstorm initiative, is working on building IM capabilities into future business products, but those applications are at least a year away.

So where should you turn? Your best bets are corporate tools like Lotus SameTime and Microsoft Exchange 2000. Both programs run on a company’s own servers and are relatively inexpensive to support; Exchange isn’t compatible with consumer IM programs, but SameTime works with AIM and also adds encryption capabilities. The software runs $27 and up per user and requires server setup and maintenance comparable to that of an e-mail system. For example, the 400 lawyers at Shaw Pittman, a law firm based in Washington, D.C., use SameTime at a total cost to the company of a few thousand dollars annually, says CIO Nicole Picciotta. (By the way, if chat is to be used strictly for customer support, check out the Web-based solutions offered by Kana, eGain, and others. See “Who Needs Online Customer Service? You Do,” July 2000.)

Whatever program you choose, the key to reaping gains like those of Shaw Pittman, IBM, and Vobix is to set guidelines for usage and train your troops well. This may take some time, says Picciotta, but the results are “absolutely worth the effort. There’s no question.”

Sign up for e-mail delivery of Dylan Tweney’s weekly column at

Want to learn more about IM solutions for businesses? Check out our Web File.

Link: Your Employees Love IM. Should You Worry?

Link broken? Try the Wayback Machine.

Your Employees Love IM. Should You Worry?

High Price of Search Technology

Having a search engine on your website is critical, because no matter how intuitive your site’s interface is, there will always be people who can’t find their way around. A search engine acts like a lifeline for these customers, connecting them to the pages they want before they drift away from your site forever.

The problem is, it’s difficult and expensive to get a search engine working properly. In a perfect world, customers could just type their questions into the search box and get coherent answers back. Too bad that ideal is so hard to achieve. The experience of one online powerhouse — stockbroker Charles Schwab — shows that while search technologies are getting better, implementation is still a bear.

This month Schwab’s website quietly debuted a new search engine that comes a lot closer to that ideal of functioning like a conversation partner. With the new engine, Schwab customers can type questions such as “Show me the P/E ratios and market caps for IBM, Cisco, and Microsoft” and receive a table showing the three companies’ price-earnings ratios and market values. You can even ask follow-up questions, such as “How about revenue?” and the site adds a revenue column to the table.

The slick new Schwab search tool (which works quite well for investment research as well as site navigation) is provided by iPhrase Technologies. The iPhrase software parses search questions and automatically builds responses, such as the dynamically generated table of P/E ratios in the above example. But technology alone isn’t the whole story.

Schwab senior vice president Robert Seidman says a lot of work went into building, optimizing, and testing the engine. That’s typical for most websites, which require that you spend a lot of time tuning their search engines, linking specific query words to corresponding webpages. Depending on the complexity of your site and the kinds of questions your customers ask, this can be a very laborious process.

Seidman says that iPhrase’s product needed less such tuning than other search engine technologies, but some was still required. What’s more, the company spent almost eight months deploying the software and then putting it through an extended beta test. Seidman says it was critically important to avoid bad search results, because they are so off-putting to customers.

All that for a feature that Seidman estimates will be used by at most 30 percent of Schwab’s customers. Still, he doesn’t underestimate the importance of a good site search engine. “If you don’t know what you’re looking for, or if you know what you’re looking for but it doesn’t fit into the top levels of our [website’s] navigation, we need to give you a way to find that,” Seidman says.

Schwab won’t disclose how much it’s spending for the search software, but iPhrase says it typically charges in the six- to seven-figure range. Other vendors of “smart” search software, such as Autonomy, Google, and Ask Jeeves, also charge steep prices for their wares.

Unfortunately, Schwab’s case is typical. Search engines are expensive, time-consuming, and recalcitrant technologies, particularly if you want to get them working well. Even the best search engines will probably be used by only a minority of your site’s visitors. The problem is that you might lose those customers if they can’t find what they want on your site — so effective searching is essential for customer retention.

What’s a company to do? If you have Schwab-like deep pockets, then invest in a high-tech search tool and take the time to set it up right (or pay the vendor to do so). If you can’t afford to spend hundreds of thousands of dollars, buy a lower-end search tool and do some tuning so that you are sure to capture the 50 to 100 most common queries. No money or time at all? Put a free search engine on your site, make sure it’s prominently featured on your homepage, and hope for the best. For customers who are trying to find something on your site, a poorly configured search engine is better than nothing at all.

Never miss The Defogger. Sign up to have it delivered daily to your e-mail box. It’s for your own well-being.

Link: High Price of Search Technology

Link broken? Try the Wayback Machine.

High Price of Search Technology

Round Two Is Coming for the Net

Face it, the Internet we know today is slow and clunky. It’s getting better, but it’s still not the lightning-fast medium everyone wants, and the functionality is limited by the network pipelines. But all that could change with Internet2 — yes, that’s right, the Internet has a sequel.

A coalition of 170 universities, plus a few dozen corporations, is pooling its resources to build the Internet2 framework, which researchers are now using to develop a variety of futuristic, high-bandwidth applications. All told, spending on Internet2 by the universities and their corporate partners totals only about $80 million per year. Nowadays, that’s hardly enough money to get a medium-size network communications company off the ground, let alone build the broadband network infrastructure of tomorrow. But with lots of eager undergraduates and computer science grad students providing free or cheap labor, universities can do an awful lot of advanced development on relatively shoestring budgets.

Internet2 is built on a foundation of several networks capable of carrying data at dizzying speeds. For example, one of the Internet2 backbone networks, called Abilene, is composed of connections as fast as 2.5 gigabits per second. (For comparison, that’s equivalent to about 1,666 T-1 lines, a data rate 45,000 times faster than your 56-kbps modem.)

In addition to raw speed, Internet2 also includes support for Internet protocol version 6 (see last week’s Defogger column, “Ante Up: Why the Web Needs an Upgrade“), the ability to broadcast information efficiently to multiple recipients, and tools for guaranteeing that data is transferred at a certain quality of service (known as QoS to network engineers). That makes Internet2 an ideal test bed for broadband applications such as videoconferencing, real-time medical image transmission, and the like.

For example, research teams at Stanford University are using Internet2 to develop a variety of long-distance applications: a robotic helicopter that can be remotely controlled by spoken commands, a system for transmitting three-dimensional models of brain activity to remote locations, several videoconferencing courses involving students on multiple continents, and tools to allow surgeons to collaborate on operations happening thousands of miles away.

Other applications are in the works, some of them commercial. A startup company called Teleportec has developed a broadband conferencing system that produces the illusion of a three-dimensional hologram, so you can use the Net to project a ghostly image of yourself, Princess Leia-like, onto a specially equipped lectern. The governor of Texas, Rick Perry, recently used the system to address a crowd in Dallas without ever leaving his office in Austin, and British prime minister Tony Blair plans to use it to appear virtually at political rallies this summer. Because Teleportec’s system (which costs $70,000) is data-intensive, requiring 384 to 768 kbps of bandwidth, the Texas governor’s address last month was transmitted over the Internet2 network.

Will Internet2 supersede the current Internet? Not really. The purpose of Internet2 is somewhat like that of the space program: to produce lots of indirect benefits by funding primary research. So rather than actually building the next version of the Internet, Internet2 researchers hope to develop technologies and techniques that can later be applied to the public network by private enterprise and the government. If companies like Teleportec take that ball and run with it, the results may be truly amazing.

For more information and news on Internet2, see the Web Guide and

Never miss a Defogger column. Sign up to have it delivered daily to your e-mail box. It’s for your own well-being.

Link: Round Two Is Coming for the Net

Link broken? Try the Wayback Machine.

Round Two Is Coming for the Net

Ante Up: Why the Web Needs an Upgrade

There are about 6 billion people on Earth. Currently, only a fraction are online — maybe half a billion of us. But that number is expanding rapidly — and will balloon even more quickly when Internet-enabled mobile phones become the Internet-access device of choice in many developing countries. By themselves, such phones are expected to add another 1 billion souls to the Internet’s population.

Great — the Internet has room for everyone, right? Not exactly. In fact, the Net is running out of addresses for all those people — not to mention their computers, phones, Net-connected cars, and (someday) their Internet-enabled toasters, refrigerators, thermostats, and lightbulbs. Making room for everyone will require a massive, worldwide investment of capital — and could require your company to replace all of its Internet equipment in the next few years.

Here’s the problem, in a nutshell. The Internet’s basic communications are made possible by a system called IP (for Internet protocol), which requires every Net-connected computer or device to have a digital address, called an IP address. It’s this address that allows computers to find one another on the Net. (Most of the time, this happens under the hood — you type in a domain name, such as, and it’s automatically converted into an IP address, like

The current version of this protocol, IP version 4 (or IPv4), which has been used for about 20 years, has room for only about 4 billion addresses in all (or 2 to the 32nd power, for mathematically inclined readers). That might have seemed like plenty in 1981, but you can see how that number is starting to look a little small now, given the Net’s growth and the globe’s population.

Fortunately, the computer scientists who developed the Internet and preside over its basic standards foresaw the problem. About 10 years ago, they developed a new version of the Internet protocol, IP version 6. IPv6 has room for many, many more addresses — 2 to the 128th power, to be precise. (In nontechnical terms, that’s many billion, billion, billion times the number of addresses IPv4 supports.)

IPv6 also provides some additional features that will better accommodate broadband services like video and audio. For instance, it allows you to prioritize data — so that, for instance, you could give streaming video data a higher priority than e-mail or website traffic. That way, the video won’t get all herky-jerky when Internet traffic gets heavy. IPv6 also offers better support for mobile devices like cell phones and handheld computers, and it would likely allow data to be routed more efficiently from place to place, increasing the Internet’s overall speed.

So if IPv6 has been around for a decade, why aren’t we all using it right now? The problem is that switching from IPv4 to IPv6 requires an overhaul of the Internet’s entire infrastructure, and you can’t exactly take the Net off-line for a day to make the upgrade. IPv6 does include features that make it compatible with IPv4, to smooth the transition. But making the change would still be a little bit like converting a standard automobile with four wheels into an armored tank with caterpillar treads — while driving 60 miles per hour.

IPv6 will also be expensive. To use IPv6, you need routers — the Internet equivalent of traffic cops — capable of supporting it. With high-end routers costing $100,000 or more, no one is exactly rushing out to replace them, as long as the old ones are still working. Your company probably has several of these machines; your Internet service provider, or carrier, may have dozens or hundreds. (See eCompany Now’s e-Business Parts List for more on carriers and routers.) You also need operating systems and applications capable of supporting IPv6 — adding to the expense and hassle of upgrading.

Eventually, however, the move to IPv6 will be unavoidable. The explosion in Net-connected mobile phones alone will put pressure on telecoms to upgrade to IPv6 so they can give addresses to all of their customers. That will happen soonest in areas where mobile phone adoption is high, such as Asia (where several ISPs have already started upgrading to IPv6) and Europe (where the European Commission recently assigned a task force to investigate making the transition to IPv6).

In the United States, IPv6 won’t be very widespread for a while — as long as four or five years, according to some estimates. So don’t sweat it yet; you probably won’t even have to consider IPv6 in your technology budgets for several years.

But sooner or later, we’ll run out of addresses for all of our Internet devices. And when that happens, ISPs and businesses alike will be forced to make the upgrade, regardless of the expense.

Coming up: IPv6 is just the beginning. Next week the Defogger will take a look at Internet2, the next generation of the Internet.

Link: Ante Up: Why the Web Needs an Upgrade

Link broken? Try the Wayback Machine.

Ante Up: Why the Web Needs an Upgrade