Thursday, February 22, 2007

Search Engine Shoot-Out

In an attempt to gauge the performance of one search service versus another, I tried a series of searches across the top three search sites (Google, Yahoo, and Microsoft Live) and then attempted to gauge the relevance of the search results. The rating will be simple: if the result is relevant to the goal of my search than I will give it one point, non-relative gets negative one point, and so-so gets zero points. In each case I only looked at the first ten results.
As you can see the results were fairly similar. My guess is that the choice of search terms was too simple. My biggest surprise was with the low relative performance of Google, my preferred search. However, some observations on usability did emerge. Yahoo seemed to have much more relevant results towards the top of the list for definitions. Live did a better job with images by prividing a "bottomless list" of images. Try it out and you'll see what I mean. The results follow, if you're interested...

The 1st search term will be “definition of net present value.”
Google: 3, Yahoo: 6, Live: 7
The 2nd search term is “definition of CRM system”
Google: 10, Yahoo: 9, Live: 6
The 3rd search term is “definition of current ratio”
Google: 8, Yahoo: 8, Live: 8

The 1st search will be “dollar sign”
Google: 10, Yahoo: 10, Live: 10
The 2nd search will be “Kanji”
Google: 10, Yahoo: 10, Live: 9
The 3rd search will be “Boston University seal”
Google: 3, Yahoo: 7, Live: 0

No comments: