Invasive Justice.

The FBI has been telling some journalists to turn over their notes, email messages, and sources, secretly, as part of an investigation into the hacker Adrian Lamo, according to Mark Rasch, a computer security expert and former head of the FBI’s computer crimes division. “… the FBI has threatened to put these reporters in jail unless they agree to preserve all of these records while they obtain a subpoena for them under provisions amended by the USA-PATRIOT Act,” Rasch writes, with justifiable outrage: The First Amendment is supposed to protect journalists from subpoenas of this nature, with certain specified exceptions.

This just proves how rotten the Patriot Act really is. Rather than an emergency measure needed to fight terrorism, it’s being used as a tool for vastly increasing the power and scope of government investigations, wiretaps, and asset seizures, as the New York Times reported Sunday. (alternate link to story: SFGate) We have, I’m afraid, traded a whole bunch of freedom for a scrap of false comfort.

Invasive Justice.

Tablet toddler test.

I took home a Gateway Tablet PC this weekend to test it out. With its built-in WiFi connection, the tablet makes a pretty decent Web-browsing and reading machine: I could stretch out on the couch, for instance, and read the latest WSJ while holding the tablet on my lap at any comfortable angle. Or on the front porch. Nice.

But the real killer app for the Tablet PC, as far as my daughter is concerned, is Sesame Street. Age two and a half, Clara is already very familiar with this Web site and its many, Flash-based interactive games. (I’ve had “Chicken Dance Elmo” drilled into my brain from seemingly infinite repetition — and don’t even get me started on the SoCal surrealism of “Make-A-Monster.”) With an ordinary computer, she needs to sit at a desk with a grownup who can work the mouse for her, since she doesn’t yet understand the mouse-cursor connection — a fairly complicated piece of hand-eye coordination.

With a tablet, though, she can click on things herself, just by tapping on the screen with the stylus. She mastered this in short order. Many of the games on the Sesame Street site ask her to click on various things, and there are immediate results when she does — things make noise, the monsters respond by talking, colors change, etc. What’s more, we could sit on the couch instead of at a desk. This is a plus since she frequently wants to hop down onto the floor to dance along with the music from the site.

The only potential disaster happened when she grabbed a bottle of cleaning fluid her grandfather was using and started to spray it onto the screen. Yikes! We stopped her in time, fortunately.

The Gateway tablet is a test unit from the Mobile PC labs. Would I pay $2300 for one of my own? No way. But bring the price down, and make it a bit more durable, and you’ve got a pretty good toddler computer.

Tablet toddler test.

Camera-phone sales boom.

According to a research firm called Strategy Analytics, sales of digital camera-equipped cell phones have, for the first time, exceeded sales of standalone digital cameras: Mobile Commerce World.com

“Mobile phone makers shipped 25 million handsets with built-in cameras worldwide in the first half of the year. This number is compared with four million in the year-earlier period.”

Camera-phone sales boom.

Spamfighters

One sad side effect of online life is that the longer you stick around, the more spam you get. This is particularly true if your email address appears online in any clear, unobscured form [Why Am I Getting All This Spam? (3/2003 report by CDT)].

After being online at more or less the same address for about five years, the amount of junk mail I was receiving finally reached ridiculous proportions earlier this summer. I was getting hundreds of spam messages per day, and just deleting these messages, easy as it was, took too much time. It was time to put a stop to it. Over the past few months, I tried four spam-fighting strategies, with varying success.

One: Spam filtering with Opera

The web browser Opera 7 [Opera Software] includes an email client, called M2. It’s got built-in spam filtering features, so I decided to test that first, since I was already using the browser and had been impressed by its speed.

The spam filtering in Opera is extremely easy to set up: All you need to do is specify, on a drop-down box, whether you want the internal filter set to strong, medium, or off. Opera takes it from there, redirecting any messages that it deems suspicious to a “spam” folder. Since I was getting hundreds of junk messages per day, I set this filter to “strong” and smiled smugly as I watched the folder fill up with dozens of messages.

My happiness was short-lived, however – turns out that strong filtering in Opera really is strong, and it had shunted nearly all of my mail, including a good number of legitimate messages, into the spam category. I reset the filter to “medium” and got somewhat better results, but I still had to comb through the spam folder every day, looking for messages that were inadvertently tagged as suspicious. Also, whether set to strong or medium, the filter let many spam messages through, so I was still deleting junk from my inbox — though not nearly so much as before.

This points out the biggest problem with spam avoidance schemes that are based on filtering. No matter how good the filter, you’re going to have some false positives and some false negatives. That means you still have to dirty your hands dealing with spam, deleting stray junkmail from your inbox and rescuing mis-categorized messages from your “spam” folder. This is not much of an improvement.

To top it off, Opera’s mail-filtering software isn’t very smart, and it doesn’t learn from its mistakes. If you find a legitimate message in the spam folder, you can push a button to tag it as “not spam” but all this does is add the sender to your list of contacts. It does nothing to increase the efficiency of the filter itself.

Two: Training Popfile

I decided to take another approach, by installing filtering software that you can “train” so its efficiency improves over time. I downloaded Popfile, a free, open-source email filtering program that does just this. [Popfile]

Popfile acts as a proxy mail server, running in the background on your own computer. You need to change the settings within your mail client so that it points to the Popfile program. From then on, whenever you check for mail, Popfile retrieves them from your ISP and then hands them off to your mail client. In the process, Popfile examines the content of each message, guesses how to categorize it, and adds a flag to the message’s headers indicating the appropriate categorization. You then set up filters within your mail client to file each incoming message according to the headers that have been added by Popfile.

As installed, Popfile has zero filtering intelligence – you need to train it by first downloading a few messages, then telling it how those messages should be categorized. Popfile analyzes the words contained in each message and figures out the statistical correlation between each word and its likely categorization – for instance, if you put a few messages containing the word “viagra” into the spam category, Popfile figures that any message containing this word is more likely to be spam. This calculation extends to almost every word in almost every message, resulting in fairly sophisticated automated classification.

You can set up any number of “buckets” into which Popfile should put messages, but I used just two – “inbox” and “spam.” At first, it threw everything into the inbox. I went into the program’s Web-based interface and told it how to categorize each of several dozen messages, then downloaded the next batch of incoming mail from my server. This time, Popfile’s accuracy rose to more than 90%. After two days, it was categorizing spam messages with a pretty remarkable 98% accuracy.

However, like Opera, Popfile’s filtering doesn’t eliminate the need for me to look at spam. I’ve still got to pick legitimate messages out of my spam box, and vice versa. Worse, I need to take the additional step of going into the Popfile interface, finding the miscategorized messages, and telling Popfile what categories they really belong to. In the end, Popfile is only a marginal time-saver – though it certainly does cut down significantly on the stress of seeing a couple hundred spam messages in my inbox.

Three: Learning to love Oddpost

Oddpost is a web-based email service that costs $30 per year [Oddpost]. It’s got a much more Windows-like interface than most Web applications I’ve seen, and works quite well for basic email. Because it’s web-based, it’s useful to anyone who needs to read and send email from more than one location (such as work and home). It has a built-in RSS newsfeed reader, making it a convenient tool for keeping up with news and weblogs.

Oddpost also has built-in spam protection. No configuration required here – it automatically files suspicious messages right into the spam folder. If a junk message sneaks into your inbox, you can “nuke” it by clicking on a button labeled with a little mushroom cloud. To rescue legitimate messages from the spam folder, you click on a peace symbol button; this moves them back into your inbox.

The spam filter here is remarkably effective, and best of all, it has very few false positives – that is, legitimate messages almost never get miscategorized as spam. That’s especially nice, because you can ignore the spam folder for days at a time without fear of missing something important – something you cannot do when using Popfile or Opera’s spam filter.

In addition to being a webmail service, Oddpost also provides IMAP access to its servers. If you have an IMAP-compatible email client, you can use it to connect to your Oddpost account – and the spam filtering continues to work. Very nice.

Four: Changing my address

Based on my informal experiments, I have a few conclusions.

First, for content filtering systems, false positives (mistakenly tagging messages as spam) are much worse than false negatives (mistakenly letting spam through the filter). If a spam filter catches 98% of the spam messages without mistakenly trashing a single legitimate message, it has saved the user a lot of time. On the other hand, if it mistakenly trashes just 1% of the legitimate messages, you might as well have no filtering at all, because the user has to comb through all the spam that was filtered out, looking for legitimate mail embedded among the trash.

Second, spam filtering has to be tightly integrated into the email client so that users can trash spam messages – and train the filtering algorithms, if possible – with a single click. Anything else is just too much of a hassle.

Finally, spam control based on content filtering is ultimately a losing battle. Spammers are getting more and more clever about making the content of their messages look like legitimate mail. Besides, even baldfaced advertising will slip past if it contains content that is new to the filter, before the filter has been trained or adjusted to catch it. Over the long term, this means that antispam content filtering will always remain an arms race with the spammers, with no decisive victories to be won by either side.

Ultimately, I suspect, the only way to get rid of spam entirely is to use a “white list” system, in which the only people who can send messages directly to you are those you’ve already identified as legitimate senders. Senders who are not on your white list will have to validate themselves before their messages can go through, either by replying to an automated message or by doing something only humans can do, such as reading partially obscured text in a GIF image. Spam tools such as Matador use exactly this approach, and although it’s annoying – and doesn’t work perfectly [Spamfighters

Link broken? Try the Wayback Machine.

Spamfighters

Seaport insecurity.

Threat Matrix is a TV drama about the Department of Homeland Security and their anti-terrorism efforts. Last night’s episode centered on some terrorists sneaking into the country via shipping containers.

The show is a mediocre and jingoistic ripoff of 24. And the terrorists were supposedly sneaking into the Port of San Francisco, “one of the busiest ports in the world.” Maybe in 1930 … perhaps they meant Oakland? But the writers got one thing right: It’s very easy to get anything you want into the U.S. through shipping ports. A four-month investigation by Baseline magazine culminated in this long, detailed, and disturbing report: Why Our Seaports Aren’t Safe.

Seaport insecurity.

Banjo history.

This summer, NPR ran a fine series on the extensive sharing and crosslinking between the blues, country music, and white and black musicians. But they inexplicably overlooked the banjo — which pretty much epitomizes the rich, intertwined, and often troubling musical history of America. Today they rectified that omission, with a very nice segment featuring banjo player and music historian Stephen Wade, who discusses oldtime banjo music and plays a few tunes on various banjos: NPR : Stephen Wade and the Banjo (audio)

Banjo history.

E-Mail on the Cheap

SOUTHWEST AIRLINES WANTED to give e-mail accounts to each of its pilots, flight attendants and ground-crew workers—critical employees who needed to be in the corporate loop but didn’t even have computers. The problem: It would have been prohibitively expensive to give all 30,000 of them accounts on the corporate mail system, Novell GroupWise. It wasn’t just the license fees. Shannon Kessner, manager of Intel core services at Southwest, says that the company would have needed to buy—and manage—at least 30 new servers.
Instead, Southwest chose a more lightweight e-mail system, Novell’s NetMail. In early 2002, the company provided e-mail accounts to all 30,000 “deskless” employees using a fault-tolerant array of just three servers. Those employees check their mail using Web browsers, usually from home or from PCs installed in airport terminals. Meanwhile, the 8,000 corporate employees with desks still use GroupWise, which includes the collaborative features that they need. Southwest saves money on software licensing: Fees for NetMail are typically $12 to $15 per user, says Novell, compared with about $70 per user for GroupWise. And the airline also saves on administration costs because the new system is simple, stable and requires little maintenance. “We’ve had very few problems with it at all,” says Kessner.

Like Southwest, many companies are discovering that corporate e-mail systems don’t have to be expensive to be effective. In many cases, simple, stripped-down mail servers fit the bill quite nicely. But that doesn’t mean you should rip out your Exchange or Domino servers tomorrow.

Expensive E-Mail
Microsoft Exchange and IBM Domino/Notes dominate the corporate e-mail world. Together, the packages own nearly 90 percent of the Global 2000 e-mail market, and that dominance will continue through 2007, according to Meta

Together, IBM and Microsoft own nearly 90 percent of the Global 2000 e-mail market, and that dominance will continue through 2007, according to Meta Group.

Group. However, these products are expensive. Mix in costs for maintenance, administration, upgrades, training and downtime, and the average cost of providing e-mail during a three-year period tops $18.45 per user per month for Exchange and $12.55 for Domino, according to The Radicati Group, a consulting and market research company. Add in the platforms and network infrastructure required to run these systems, and the fully loaded, monthly per-user cost soars to $36.56 for Exchange and $33.88 for Domino.

Fortunately, fully functioning corporate e-mail systems can be had for far less. According to Radicati, Oracle Collaboration Suite averages $5.40 per user per month ($16.25 including the infrastructure costs) and Sun One Messaging Server costs $8.04 ($17.80 including infrastructure).

If you’re willing to forego the more advanced features offered by those high-end products, the cost drops to the floor. Sendmail recently announced a partnership with Hewlett-Packard and Intel to provide corporate e-mail (called Workforce Mail) for a total cost of $1 to $2 per user per month, while IBM claims that its new, low-cost Lotus Workplace Messaging can do the same for less than $1.

Switching Costs
So why would anyone pay high prices when they could deliver e-mail for one-sixth to one-tenth the cost of Exchange?

One reason is that corporate knowledge workers—those whose job it is to discover, create and manage information—actually do use the more complicated collaborative features (document sharing, scheduling and the like) built into Notes and Exchange. In some cases, the low-cost systems lack basic features, such as spellcheckers and mail-filtering rules. And you can’t switch from a full-featured e-mail system to a less capable one without angering at least some end users. “Once you’ve convinced people to use a fork, you don’t want to take that away and convince them to use a spoon,” says Mark Levitt, research vice president for collaborative computing at IDC (a sister company to CIO‘s publisher).

What’s more, everything you’ve already spent on e-mail to date is a sunk cost. You’re not getting that money back, even if you switch. “Although commodity e-mail systems may look cheap on paper, the ongoing maintenance of your existing e-mail system may not be as expensive as switching,” says Matt Cain, senior vice president at Meta Group. Finally, switching e-mail platforms requires your IT staff to install and support a new system (a major retraining headache) and to migrate user accounts and data.

The bottom line? “I don’t think there is such a thing as cheap e-mail, particularly if it’s got the capabilities everybody wants,” says Robert Moon, CIO and vice president of information services for ViewSonic, whose e-mail system is based on Oracle Collaboration Suite.

Lower Cost of Ownership
Commodity e-mail systems do, however, offer some powerful advantages that lend themselves to situations where basic e-mail is all you need—such as providing e-mail to deskless workers or to users who are not computer savvy.

First and foremost is the lower cost of ownership. Commodity mail systems are based on robust, standard, open Internet mail protocols, such as SMTP, POP3 and IMAP. They run on standard hardware and may use the same back-end data stores as the rest of your enterprise. They can deliver e-mail to end users via Web interfaces (much like Hotmail or Yahoo mail), to standard POP clients such as Eudora or in some cases even to Microsoft Outlook, all of which may simplify client maintenance headaches.

Lotus Workplace Messaging, for example, uses industry-standard J2EE code running on IBM’s WebSphere, stores its data in a DB2 database and delivers mail via webpages. If you’re already running those systems for, say, your Web applications, you can get significant economies of scale by running e-mail on the same platform. (By contrast, Domino uses a proprietary data store, has its own programming language, and generally requires the bulky and idiosyncratic Notes client.)

Keep It Simple
When your mail system is based on a standard platform, maintenance is simpler because your IT staff can apply skills it has already learned in managing other IT resources. That’s one reason Jim Bobo, systems administrator and chief programmer at Courtesy Insurance Agency, switched his company’s 100 users from Exchange to Stalker Software’s CommuniGate Pro. “You don’t have to have a degree in Microsoftese to use the thing,” says Bobo.

Beyond cost, some companies are finding that fewer features can actually be beneficial to the end users. For instance,

Beyond cost, some companies are finding that fewer features can actually be beneficial to the end users.

ManuLife Financial, a Canadian financial services company with extensive business in Asia, used Lotus Workplace Messaging to deliver Web-based e-mail to 3,600 independent insurance agents in Japan. Because most of those agents are not computer savvy, the company wanted an easy-to-use solution. “A high degree of functionality would be a bad thing because you’ve got novices who have never used a computer before,” says Rob Salerno, a partner at MetaLogic Consulting, which installed the Lotus system. “You need something that has an easy-to-use interface and performs well.”

Others agree. For many workers who don’t use PCs every day—factory workers, retail employees and the like—simpler is better. “Low maintenance and low total cost of ownership make a lot of sense for those workers who don’t need a lot of high-end features,” says Dana Gardner, a senior analyst for The Yankee Group.

A Mature Market
Such employees—about one-third of the corporate workforce, according to estimates by Radicati and Ferris Research—are a tempting market for e-mail vendors. With the rest of the corporate world already sewn up by IBM and Microsoft, vendors are looking for growth where they can get it. Therefore, the recent push toward low-cost mail solutions may be driven more by vendors’ marketing desires than it is by customer needs.

There’s also an underlying technical reason for vendors to push standards-based e-mail. Putting e-mail servers on a common foundation gets vendors in line with the overall industry trend toward open Internet standards. “The business design we have, which is based on industry standards, leverages the industry’s investment, which is why we can get the licensing costs down,” says Ambuj Goyal, general manager of IBM’s Lotus Software. In other words: With millions of developers working on Java applications, IBM doesn’t have to devote its own resources to building a robust platform from scratch. IBM’s goal is to have all of its messaging and collaboration products migrate to an open platform over time, says Goyal, while preserving support for current Notes clients.

Naturally, Microsoft isn’t taking this threat lying down. The new Exchange Server 2003 includes a per-device licensing option that makes it more economical for companies that want to provide kiosk-based Web access to large numbers of deskless workers. Instead of paying for each user, companies can pay a license fee for each device, with an unlimited number of users. And Microsoft has announced plans to move Exchange toward a SQL-based data store, although that is probably several years from fruition.

How to Save Money
In addition to deskless workers in large corporations, commodity mail vendors may find some traction among small and midsize businesses, where the allure of added features may not be enough to overcome high costs. While commodity mail is not going to unseat Exchange and Domino from their thrones, says Yankee Group’s Gardner, it may contribute to a gradual erosion of their market share over time.

In the short term, however, most companies are moving cautiously. “I want to pick a platform that’s going to be around for a long time,” says Len Pagon, president and CEO of technology consultancy Brulant. Salerno agrees: “A lot of companies are playing wait and see—they’re waiting for a success story.”

“Commodity mail is nothing new. It’s been out there for the past decade, and it has yet to take hold in corporate America,” says Meta’s Cain. Instead of switching mail systems, Cain recommends looking for ways of reducing costs in your existing mail setup: Consolidating servers, centralizing and managing storage more effectively, and adding Web mail access to eliminate client maintenance and training headaches.

Still, if you’re adding large numbers of new users to your e-mail system, if you’re expanding your mail system to groups of employees that haven’t had mail accounts, or if you’re a small company with a tight IT budget, you should take a look at low-cost mail alternatives. If nothing else, those systems are providing IBM and Microsoft with some much-needed competition. And they just might deliver what you need at a fraction of the cost.

Sidebar: Economical E-mail Servers

Anticipated benefit Provide basic e-mail services at lower cost than full-fledged e-mail systems.

Hurdles Lack of collaboration features. Switching costs. Reluctance to use smaller vendors.

Primary markets Large enterprises with “deskless” workers. Small and midsize businesses.

Estimated cost 50 cents to $2 per user per month.

Vendors
Gordano
www.gordano.com: GMS Mail for Unix and Windows platforms.

IBM/Lotus Software
www.lotus.com: Workplace messaging and Notes e-mail.

Ipswitch
www.ipswitch.com: IMail Server for Windows 2000 and Windows NT.

Microsoft
www.microsoft.com: Exchange messaging and collaboration.

Mirapoint
www.mirapoint.com: Message Server appliances.

Novell
www.novell.com: NetMail e-mail and calendaring system.

Oracle
www.oracle.com: Collaboration Suite messaging and collaboration system.

Sendmail
www.sendmail.com: Workforce Mail server.

Stalker Software
www.stalker.com: CommuniGate Pro mail server.

Sun
www.sun.com: Sun One Messaging Server corporate messaging platform.


Dylan Tweney is a freelance writer based near San Francisco. He can be reached at dft at tweney.com.

Link: E-Mail on the Cheap

Link broken? Try the Wayback Machine.

E-Mail on the Cheap

Keeping an eye on Big Brother.

The Government Information Awareness site is trying to provide a total picture of our government, the individuals within it, and their connections to various interest groups. In other words, it’s aiming to do for government officials what John Poindexter’s Total Information Awareness program would do for individuals — put them under the microscope, and use information technology to assemble a comprehensive view.

The site’s motto is a quote from James Madison which I also recently spotted, and underlined, in Howard Rheingold’s book Smart Mobs:

“Knowledge will forever govern ignorance, and a people who mean to be their own governors, must arm themselves with the power knowledge gives.” — James Madison

Keeping an eye on Big Brother.

My new gig.

I’m psyched: Starting this week I am executive editor for Mobile PC magazine, focusing on mobile technology: notebooks, PDAs, handhelds, mobile phones, etc. Our first issue will be out in January. I’ll be editing features and the news/trends section. Got a mobile product, a story idea, or a trend you want me to know about? Write me: dtweney at mobilepcmag.com

My new gig.