Most of the words in the English language are greatly underutilized. Words like fracas and sanguine and superlative don’t pop up nearly as often as simpler words like book or car or the. This simple observation has a name: It’s called Zipf’s Law, named for an obscure linguistics professor, George Kingsley Zipf, who noticed this pattern around the turn of the century. [Author’s note: Zipf’s Law is closely related to the “Pareto Distribution” in economics. -dft]
Fast-forward to today. Azer Bestavros, a professor of computer science at Boston University and chief scientific adviser to Allaire, is currently exporting Zipf’s linguistic principle into a retail setting. Bestavros’s work and ideas might also offer a breakthrough in how we use personalization technologies both online and in traditional stores. Zipf saw that our vocabularies are woefully understocked; Bestravros says that, likewise, most products in any given category are undersold.
Take books as an example. Few books are as popular as Harry Potter and the Goblet of Fire. If you make a graph of all the books sold, you’ll find that the number one title sells about twice as many copies as the number two title, three times as many as number three, and so on. Unlike most exponential curves, however, this graph doesn’t swoop rapidly down to zero. Instead, Bestavros argues, there is a lot of value on the right side of the graph — especially since profit margins tend to be higher on the less popular products. If booksellers ignore the less popular books and concentrates only on best-sellers, they’re leaving a lot of money on the table.
This brings us to the personalization problem: Most personalization technologies, says Bestavros, are based on popularity. For instance, Amazon.com might recommend Johnny Cash CDs to people who buy Willie Nelson’s latest album, because Cash tends to be popular among Nelson fans. That kind of personalization just ends up pushing more and more of the best-sellers. The consumers don’t benefit, because they often already know about the popular option that’s being foisted upon them. And the retailer has missed an opportunity to push a potentially attractive product that the consumer would have been happy to learn more about.
The solution, says Bestavros, is a personalization technology that can tout offbeat products. Instead of relying on popularity as the determining factor for the technology, Bestavros prefers “attribute-based personalization,” which makes recommendations to customers based on who they are and what they like. In other words, the retailer creates a profile of each customer, then uses that profile to match customer attributes with the those of various products.
The downside to this method is that it requires merchants to compile a list of attributes for each product in their catalogs or on their shelves. Of course, such lists aren’t just lying around waiting to be plunked into a database, so there’s a bit of labor involved. In addition, customers need to be willing to complete surveys and fill out their own profiles.
Later this year retailers will have a chance to see if Bestavros’s ideas add up, when Allaire releases a personalization product that puts his theories into practice. If consumers can see real value in filling out these forms, expect the new product to be a hit. Retailers also won’t mind the extra work if they find that it makes inventory move faster.
Perhaps if Bestavros solves the personalization problem, he can turn his technology toward improving our vocabularies. Now wouldn’t that be superlative.
Link: Personalization Without Popularity
Link broken? Try the Wayback Machine.