Eeky Geeky: Weekly Peeky at the Freaky – 4/29/04

Carl Brooks

Part 3: Taking one last stab at actual informative reporting, we tell you if Google is still the best search engine there is. Note that the fabled Google IPO revs up the rumor mill this week. Thursday is their financial disclosure filing deadline, and thousands of tiny little underwear analysts are frothing their way into early graves right now, screaming, “1. Buy Google. 2. ???? 3. Profit!”

And suspicious geeks, alarmed by what they see as Google prying open the lids to their darkened, whiffy bedrooms, are yelling about the information nightmare Google will unleash. But the really interesting question is whether Google remains the best.

More after this: EGWPATF reports an actual, real live threat to your life, liberty and civil rights, in the same woeful manner as poor, sweet Louis Tully hammering on the restaurant glass just before his excorporeation by Vince Klortho. The FBI raided the administrative offices of the Deer Valley School District outside Phoenix, AZ, under a sealed warrant. They were looking for unauthorized media files. Read that again slowly and then tell me how worried you are about tailored advertisements.

For the uninitiated, a search engine has an algorithm, or a repeatable, logical method for deciding how relevant a web page is to the search query. Arguably, some are better than others, and some search engines simply shoot the highest bidder to the top of the list, skewing the results in favor of commerce rather than usefulness.

So, if you, proud internet seeker, sit at home and search for “Dirty Monkey Love,” the search engine, quivering with disgust, hunts through its collection of billions of web pages for those words, as a title, in metatags, a complete phrase, or single occurrences. From the millions of correct answers, it has to decide which one you see at the top of the list.

Google uses something called PageRank, which, simply, awards a web page points for relevance based on links to other pages. There are other mind-numbing details, but that’s basically it. It is objective, non-pecuniary, and super speedy.

However, since a business lives or dies based on how far down the page it appears on Google, an entire industry of rapacious “search engine optimizers” has sprung up to dick around with witch-doctor cures to get business X to the top of the list, and they have skewed the results far, far away from actual objectivity.

They’ve done it for all the other search engines, too-this is known as “gaming” a search engine-but has Google, top of the heap, become less than the best simply through the social pressures that it generates? Does Google affect its own search results through its own mammoth influence? Yeppers.

I did an informal study. I collected a smattering of search engines and ran the same query through each of them, in what will henceforth be known as “The Google Test.” Using AllTheWeb, Yahoo!Search MSN, HotBot, Overture, Dogpile, and Google (all of which run different search processes), I searched for a variety of things: anti-matter, my 7th grade math teacher, a wicked spiffy new cell phone (Kyocera SE47) overheating Maxtor hard drives, and free DVD player software for XP, in precisely those terms. I based my results on how informative and useful the search results were.

The bottom line? Google still wins, but only because of Google Groups, which archive USENET, and the layout, which is still the fastest and easiest to read. Google has no currency in informative pages that aren’t bettered or equaled by the others; 95% of all search results with a saleable product name turn up commercial sites, but Yahoo did an excellent job bumping real information to the top of its objective results, half again as good as Google, for example.

Google’s web results are gamed so hard that they do not produce the best results for web searches. Better information is available at Yahoo!, Dogpile, and Hotbot, but it is a marginal enough victory that Google will remain my homepage, if only for Groups and layout. For more efficient searches, you are better off using more specific, and less influential, engines, which remain slightly less polluted by virtue of their lack of success. The future lies in subject specific search engines; the geeky has spoken.

Read more about the study at eekygeeky.hostrocket.com.