Having a search engine on your website is critical, because no matter how intuitive your site’s interface is, there will always be people who can’t find their way around. A search engine acts like a lifeline for these customers, connecting them to the pages they want before they drift away from your site forever.

The problem is, it’s difficult and expensive to get a search engine working properly. In a perfect world, customers could just type their questions into the search box and get coherent answers back. Too bad that ideal is so hard to achieve. The experience of one online powerhouse — stockbroker Charles Schwab — shows that while search technologies are getting better, implementation is still a bear.

This month Schwab’s website quietly debuted a new search engine that comes a lot closer to that ideal of functioning like a conversation partner. With the new engine, Schwab customers can type questions such as “Show me the P/E ratios and market caps for IBM, Cisco, and Microsoft” and receive a table showing the three companies’ price-earnings ratios and market values. You can even ask follow-up questions, such as “How about revenue?” and the site adds a revenue column to the table.

The slick new Schwab search tool (which works quite well for investment research as well as site navigation) is provided by iPhrase Technologies. The iPhrase software parses search questions and automatically builds responses, such as the dynamically generated table of P/E ratios in the above example. But technology alone isn’t the whole story.

Schwab senior vice president Robert Seidman says a lot of work went into building, optimizing, and testing the engine. That’s typical for most websites, which require that you spend a lot of time tuning their search engines, linking specific query words to corresponding webpages. Depending on the complexity of your site and the kinds of questions your customers ask, this can be a very laborious process.

Seidman says that iPhrase’s product needed less such tuning than other search engine technologies, but some was still required. What’s more, the company spent almost eight months deploying the software and then putting it through an extended beta test. Seidman says it was critically important to avoid bad search results, because they are so off-putting to customers.

All that for a feature that Seidman estimates will be used by at most 30 percent of Schwab’s customers. Still, he doesn’t underestimate the importance of a good site search engine. “If you don’t know what you’re looking for, or if you know what you’re looking for but it doesn’t fit into the top levels of our [website’s] navigation, we need to give you a way to find that,” Seidman says.

Schwab won’t disclose how much it’s spending for the search software, but iPhrase says it typically charges in the six- to seven-figure range. Other vendors of “smart” search software, such as Autonomy, Google, and Ask Jeeves, also charge steep prices for their wares.

Unfortunately, Schwab’s case is typical. Search engines are expensive, time-consuming, and recalcitrant technologies, particularly if you want to get them working well. Even the best search engines will probably be used by only a minority of your site’s visitors. The problem is that you might lose those customers if they can’t find what they want on your site — so effective searching is essential for customer retention.

What’s a company to do? If you have Schwab-like deep pockets, then invest in a high-tech search tool and take the time to set it up right (or pay the vendor to do so). If you can’t afford to spend hundreds of thousands of dollars, buy a lower-end search tool and do some tuning so that you are sure to capture the 50 to 100 most common queries. No money or time at all? Put a free search engine on your site, make sure it’s prominently featured on your homepage, and hope for the best. For customers who are trying to find something on your site, a poorly configured search engine is better than nothing at all.

Never miss The Defogger. Sign up to have it delivered daily to your e-mail box. It’s for your own well-being.

Link: High Price of Search Technology

Link broken? Try the Wayback Machine.