Google vs. Bing — a Quasi-Empirical Study 356
eko3 writes "SearchEngineLand.com is featuring an article that compares Google's result query relevance performance to Microsoft's Bing. Through the author's methodology and very small sampling, he argues Bing returns slightly more relevant results than Google. The article suggests that Google is riding its current market success based on its legacy namesake when internet search used to be a lot more painful than it is today."
O No (Score:5, Insightful)
Through the author's methodology and very small sampling,
Science Fault Detected! Engaging TL;DR.
Re: (Score:2, Funny)
Bing returns slightly more relevant results than Google.
This just means his sample was porn searches.
Re: (Score:3, Funny)
Really? So Bing is better for pr0n? Time to invest in Microsoft. :)
Re:O No (Score:5, Informative)
Following the "science fault" route:
* what does the article's author do for a living? Falsifying of search return.
* does the site that published this study have ties to the "winner"? It's among their "sponsors and partners" page.
Somehow, nearly every time you find an "independent" study giving sensational results, it is sponsored by someone with a vested interest in those results.
Re:O No (Score:4, Interesting)
Of the three searches he discusses in the article as examples of Bing doing better than Google, I get the same top two results on both search engines for one, and for another he is giving Bing credit for putting two Linkedin pages at the top (not a good thing IMAO) and for the third he thinks that having a keyword stuffed spammy affiliate site, that does not actually have the tickets searched for available, as the first result is better than having an out of date news story.
If I was scoring it then Google would have won.
I actually tried using the three major search engines for a few days, using blind search [fejus.com] , at the time Bing came out, and Google was the clear winner then.
Re:O No (Score:4, Informative)
http://www.linkedin.com/in/conradsaam [linkedin.com]
On this page you will find that Microsoft is one of Conrad Saam's clients. Google, oddly enough, is not mentioned:
"My experience includes numerous awarding-winning interactive projects for clients including AOL, Disney, Ford, General Motors, Kraft Foods, Lego, Macromedia, Mattel, McDonalds, Microsoft, Napster, Nickelodeon, The United Nations and WeightWatchers."
I will take his anecdotal research with a very large grain of sodium chloride.
quasi-empirical (Score:3)
Science Fault Detected! Engaging TL;DR.
Indeed, and I'd like to know what precisely is "quasi" about the "empirical"ness of it? Seems to me it is completely empirical and that neither "quasi", "empirical" nor "quasi-empirical" would have very much to do with scientific worth.
What about AltaVista? (Score:5, Funny)
As I sit here surfing the web on my Digital Equipment Corp. VAX 4000, I wonder... why is there no comparisons to AltaVista... the king of search engines.
Re:What about AltaVista? (Score:5, Funny)
Because there IS no comparison to AltaVista. Good or Bad!
Re:What about AltaVista? (Score:5, Funny)
You lucky, lucky bastard. I only WISH I could afford a sweet chunk of iron like the DEC VAX 4K! I'm on a Commodore Vic 20 connected to CompuServe and I can't search shit! In my day we'd have to use our HP programmable calculator connected to a dodgy barcode reader the size of a small aircraft to parse through the pages of a phone book, and we LIKED it that way. Darn, whippersnappers on my lawn, gotta get the rake...
Re: (Score:2)
I'm on a Commodore Vic 20
What's the weather like in Afghanistan these days?
Re: (Score:2, Funny)
I'm on a Commodore Vic 20
What's the weather like in Afghanistan these days?
I imagine it's windy where you are, with a notable wind shear about 5-6 feet off the ground...
Re: (Score:3)
Re: (Score:2)
Hey, I have a 4100 right over here. 3 feet away. I don't surf with it.
And hey, Altavista worked great, I had no complaints about it. Only when it was obviously on the way down did I start using google.
Brett
Re: (Score:2)
I want it back. (Score:3)
AltaVista respected punctuation. If you're searching for something with a dash in it, for example, or for the use of some operator in a programming language, AltaVista found it for you.
Google strips your punctuation right away, making it largely useless for such searches.
Re: (Score:3)
Re:What about AltaVista? (Score:5, Insightful)
Re: (Score:2)
Ahh for want of mod points, +1 Frickin Funny.
Re:What about AltaVista? (Score:4, Informative)
Did you know Excite [excite.com] is still around? I had no idea.
This list is pretty amazing for some nostalgic perusal.
http://en.wikipedia.org/wiki/List_of_search_engines [wikipedia.org]
(Now as for that VAX... No! Bad!)
Re: (Score:2)
AltaVista? WebCrawler is much better, it has a mascot!
And AltaVista Personal? (Score:4, Interesting)
I guess not many people used it, but AltaVista Personal did an amazing job of indexing and searching local and network files. Faster than any of the "modern" OS integrated offerings I've seen. And without sucking up resources. If there were a version for XP/Vista/Win7 I'd use it in a heartbeat.
Re: (Score:3)
why is there no comparisons to AltaVista...
AltaVista returns slightly more relevant results than Lycos.
Bah, just use metacrawler and search them all.
Re: (Score:2)
why is there no comparisons to AltaVista...
AltaVista returns slightly more relevant results than Lycos.
Bah, just use metacrawler and search them all.
Metacrawler? Dogpile is so much more efficient. And it has a cuter mascot!
Re: (Score:2)
Re: (Score:3)
Good ol' Lycos. That was actually my search engine of choice in the pre-Google days.
I don't remember it being THAT bad, but I think it may be a matter of adjustment. I DO remember that back then my bookmarks file meant a lot more to me. If I found something interesting I bookmarked it because otherwise I might not find it again. Nowadays - I rarely care about my bookmarks. Anything I need to find again I can usually find FASTER by Googling than by searching through a long list of bookmarks.
I guess that
The market will decide (Score:3, Insightful)
Microsoft's primary business function is documents and the like, though they've attempted to diversify with search.
There's a very low barrier to individual users to choose between them for either (given that MS has put its document processing online for free, last I heard) so, in the end, it's likely that the superior product (whether marketed better or actually better) will triumph in marketshare.
Bring this back up in 18 months, and we'll likely see some clear differential if there really is an actual difference in the applicability of either one's functions.
Re:The market will decide (Score:5, Insightful)
Google's primary business function is 'global hegemony'.
Microsoft's primary business function is 'global hegemony'.
FTFY.
Re: (Score:2)
There's a very low barrier to individual users to choose between them for either
Not as low as you might think. Network effects can skew results for people that intend to share or collaborate with their documents. Just like the justification for a lot of people who choose MS Windows or MS Office, what their friends and/or business associates uses plays a big role here.
Re:The market will decide (Score:5, Insightful)
Less motivation? On the surface, I agree with you. But the US government and Microsoft have something of a strange yet cooperative relationship. I get a feeling that Microsoft does a bit more data collection than we know. But speculation aside, Microsoft has far more potential to collect information than Google. And if requested, I have little doubt that Microsoft would comply with anything the government "or its partners" asks.
Re: (Score:2)
I like your initial comparison... it's why I at least don't start with Google.
I've also found that if I want concise results I use Bing, if I want tons of links (and sometimes I do) I use Google.
Bing is new yet, so this might not work for me two years from now, but so far so good.
Re: (Score:2)
Check out GoogleSharing [googlesharing.net]. It's a proxy service that anonymizes your queries, but also utilizes Google's encryption service. So GoogleSharing doesn't know what your queries are, and Google doesn't know where they came from.
Re:The market will decide (Score:5, Informative)
Actually Google doesn't sell your data to advertisers. They use your data to determine which ad to show you.
Microsoft conversely filed for a patent specifically to govern a method of how best to auction your private data to third-parties.
Re: (Score:3)
No, it's the other way around. The ad buyer chooses which demographic to show it to.
Re: (Score:2)
You know all other things being equal, MS has less motivation to violate my privacy than Google does, as selling my data to advertisers isn't their primary source of revenue. For that reason alone, I look more favourably on Bing than Google search.
"Despite the fact that they have yet to do so, I don't trust Google to not sell my data to evil companies, such as convicted monopolist Microsoft. That's why I use Bing."
Um, what?
Re: (Score:3)
Google pays Mozilla in exchange for integration with Firefox.
Re: (Score:2)
Did you translate this post into English with Google Translate?
Small sample is right (Score:5, Informative)
A single person's subjective analysis of 20 search terms is a small sample indeed! I will say, Bing has come a long way in producing search results I feel are useful, but I still find myself frequently forgetting Bing is the default search, coming up with bizarrely useless results, switching to Google, and saying to myself, ah yes, these are the results I was expecting.
Perhaps I've just learned to produce search results in Google that meet my needs and haven't developed that skill in Bing. A more thorough, less subjective analysis comparing the two search engines would be very interesting. Sadly, I think this writer's personal conclusion is just going to spark a nerd-war over Google vs. Microsoft filled with subjective opinion (like mine) and little empircal evidence.
Re:Small sample is right (Score:4, Insightful)
To be fair, he's not trying to get this published in a journal, just point out that Google is no longer streets ahead of everyone else. I think that is a fair assessment.
Re:Small sample is right (Score:5, Insightful)
Indeed. The study's methodology was, to put it mildly, badly flawed. A far better methodology would have had twenty other people do the searching, and have THEM rank the results. That would still have been flawed, too, but less subjective than just having one guy decide how relevant the searches were.
Google is still #1 because people tried Bing and found it wanting. I did, the first day it was out.
Re: (Score:3, Interesting)
Google is still #1 because people tried Bing and found it wanting. I did, the first day it was out.
Isn't that the point being made? When Bing was first launched Google may have had better results, but now Bing is catching up with Google and (maybe?) surpassing it in terms of relevancy. Google is slow on adjusting its algorithm since some/most/all people have the perception that it is better than Bing and since those people never go back to try Bing again, Google has little need to adapt as quickly.
Re:Small sample is right (Score:5, Interesting)
I try Bing about once a month for a day. I'm constantly changing back to Google to find the results I'm looking for. It isn't for lack of trying, but the result is that I can't stand Bing. I've even begun to suggest that BING stands for "Bing Is Not Google".
Re:Small sample is right (Score:5, Informative)
Re:Small sample is right (Score:5, Informative)
Replication? (Score:2)
Anyone care to try to replicate the results? You could probably just use his list, or create a list of your own if you really want to. I'd do it myself but I'm supposed to be working.
Re: (Score:3)
Anyone care to try to replicate the results? You could probably just use his list, or create a list of your own if you really want to. I'd do it myself but I'm supposed to be working.
Duh, just post your actual work to an "ask slashdot" post, then come back and work on this instead. I mean come on, what are you, new here?
Re: (Score:2)
I use DuckDuckGo as my main search engine, and it almost always gives me the page that I want early on. If I can't find the result there, I go to Google. So far, I've not come across a single instance where Google returns a useful result but DDG doesn't. The main difference is that DDG admits when it can't find anything relevant, but Google gives you 10 pages of irrelevant ad-filled pages that you might like to look at.
When I started using Google, around 2000, this would not have been the case - Google
Bing is great for non-techies (Score:5, Interesting)
I agree with the notion that Google is riding it's legacy of taking search from something that was literally an impossible problem to solve to something that was instant. It earned every bit of that, but search has entered a new era.
Bing is now competing at the forefront, which is taking search from finding results in an index to finding answers to questions and solving problems. "Decision engine" is a bit overhyped, but it's the right direction to move in, in my opinion. This is a good thing, because Bing and Google will push each other.
I generally refer friends and people I know to Bing because they tend to treat search engines like a natural language processor, or as a companion that can help them answer questions and solve problems.
Google is still (much) more effective if your Google-fu is powerful, but if it's not, Bing can be a bit friendlier and better at getting you to what you want to see.
Re:Bing is great for non-techies (Score:4, Informative)
I'm sorry, you must be using a different Bing than I do. Your statements regarding Bing's performance do NOT match up to my experience with it in the slightest.
Perhaps you're confusing bing.com with google.com/bing ? =)
Re: (Score:2)
Oh the number of times I still see people type "what is the population of equador?" or something formatted that way into the search bar. I suppose we have jeeves to thank for that.
Re:Bing is great for non-techies (Score:4, Informative)
Re: (Score:2)
Well, yes, that IS how it should be, despite you trying to sarcastically make it sound like its ruining the human condition.
If you are using a search engine, you are searching for something. You enter in what you are searching for. Naturally, if the engine is optimal, it will return exactly what you are looking for, and no more searching could be required. Bringing up a page of 20 things for you to sift through, is sometimes not as helpful as if it had just brought up the most relevant example.
If I look up
Re: (Score:2)
No the best would present with a list of said recipes as a search engine has no way to evaluate good.
Re: (Score:2)
Search engines do have a way to evaluate good. Based on user reviews, recommendations, etc etc, even page hits can be considered some value.
Granted, these aren't perfect, but they are better than nothing.
Re: (Score:2)
And there's nothing to say you can't go "Next" and have it bring up the next page in its list, but it shouldn't require you to search through them manually like it does now.
Though you can also keep things the way they are now, with the common "skim over the first line it brings back" and then decide that way - but ultimately thats a process that a computer should be able to do anyways.
Re: (Score:2)
What I would really prefer is a mode that gives me exactly what I search for. Search engines used to be better but now they cater to the lowest common denominator too much.
Re: (Score:2)
Search engines do have a way to evaluate good. Based on user reviews, recommendations, etc etc, even page hits can be considered some value.
Granted, these aren't perfect, but they are better than nothing.
I thought the google engine looked at popularity in deciding which pages to give you.
OK, I took a shot at it, (Score:5, Interesting)
But I still don't know how to change the water filter [google.com] on a Frigidaire Professional Series [bing.com].
For some reason, they gave Bing 7 points for that query.
But the first result merely regurgitates the question, then has an ad link for Fixya.com.
Re: (Score:3)
Re: (Score:2)
Re: (Score:2)
I guess it likes you and hates me?
Re: (Score:2)
The bing link just sells you replacement filters, it doesn't tell you how to install them. Throw "manual" into the query and you'll get better results (bing and google).
Re:OK, I took a shot at it, (Score:4, Informative)
Perhaps there's no existing webpage that answers this brutally obvious question. Here, Google and Bing. Crawl this:
How to change the water filter on a Frigidaire Professional Series:
- Push the button labelled "eject" on the old water filter
- Remove the old water filter
- Insert the new water filter
I've reluctantly moved to Bing (Score:4, Insightful)
Google just returns too many garbage marketing links. Bing isn't vastly better, just slightly. And, I imagine that if people start to migrate, they'll take on the same ad ratios as Google.
I'll see your small data set and raise an anecdote (Score:2)
After trying to put up with Bing (being the annoying default in IE 7/8, and on my smartphone) it just doesn't hit the right notes with the kinds of searching I do. It's probably that it doesn't prioritize Wikipedia results high enough, though.
Re: (Score:2)
Re: (Score:2)
Insignificant Result (Score:5, Informative)
20 searches, 15% margin, 100% subjective.
It doesn't matter, google won. (Score:4, Insightful)
Generally speaking, to dethrone the entrenched standard (in any industry, not just search engines) you have to be substantially better to get people to switch to something they aren't used to. Marginally better just won't cut it. Cost is a moot point, because outside of MS paying me a check every month to use bing, you can't beat the price of free.
Humans are generally animals of habit, and unless you give them a good reason to, they won't change.
"Better" didn't help Yahoo. (Score:5, Interesting)
For about the first half of 2008, Yahoo search was better than Google search.
Yahoo introduced specialized subengines - stocks, weather, movies, celebrities - which were triggered by matching queries. Each subengine had a special case for that class of information. Yahoo had about fifty such subengines.
Nobody noticed. Yahoo's market share didn't move. I only knew about this because I went to a talk by the head of Yahoo R&D at the time.
Bing's strategy seems to be mostly to follow Google. Google put Google Places into web search (a big mistake [sitetruth.com], because Places is so easy to spam), and Bing followed within days.
This week, everybody from Techdirt to CNN is dumping on Google for their spam problem. Even Paul Krugman at the New York Times mentioned it. There's much blog talk of "human powered search" or "curated search" to stop the spam but the failure of Wikia Search, and the lack of interest in ChaCha, Swicki, and Rollyo, indicates that's a dead end. (Mahalo started as human-powered search and ended up as a content farm, which is a hint that "human powered" doesn't equate to "better". No complaints from search users about that, though.)
(Note: I have a position in this; I run SiteTruth [sitetruth.com]. There, we try to find the business behind the web site, and rate that, using data from the SEC, BBB, D&B, and other hard data sources about businesses. This works well at eliminating spam. Too well for some sites; we get complaints about our hard-ass "when in doubt, rate it down" approach.)
Re: (Score:2)
(Oops, the line "No complaints from search users about that, though." was supposed to go in the last paragraph, not the one about Mahalo.)
Re: (Score:2)
Mod parent up. Fascinating paper linked deserves to be be Slashdotted in it's own right.
not for my searches (Score:5, Interesting)
I use about six languages on a daily basis and IMHO bing sucks at everything that isn't English.
Self searches (Score:3)
I get strange looks when I tell someone I Binged myself.
You won't get me to use Bing any time soon ... (Score:3, Informative)
Google isn't paying attention to searching (Score:5, Insightful)
It has been my experience that as Google has gotten bigger they seem to return at the top of their results pages that are nothing more than aggregating websites (most contain LOTS of google adverts too, which piques my thoughts on why they do show up at the very top of Googles searches). This is VERY annoying. As a result, I, previously a great supporter and user of Google, have been looking for a search engine that doesn't return websites that do nothing but hand me links to other websites. If i find one, that loads quickly, I will dump Google.
If Google is listening, it should be very easy to stop the aggrigation websites (sites that have NO CONTENT but just contain links to other sites) from reaching the top of your results.
Re: (Score:3)
Give DuckDuckGo [duckduckgo.com] a try. I've been pretty satisfied with the results so far (I've been using it for a couple weeks after getting totally sick of the link farms on Google). They seem to be pretty strong on privacy too, if you're into that.
Re: (Score:3)
So if Bing is so fantastic, and knows that B921XF is a light bulb, then why don't any of the subsites it links to know that B921XF is a light bulb? And how
ORLY? Dig a little deeper on this one..... (Score:5, Informative)
Of course Bing! is better than Google. Shenanigans! Or at the very least, suspect.
Comment removed (Score:5, Insightful)
Re: (Score:3)
Microsoft is notorious for working hard until they get it right and then steadily eating into their competition.
So is Google.
And Google's known for doing it more recently than Microsoft, and faster.
Or Bing is being temporarily accurate.... (Score:2)
If Bing, the search engine created by the massive for-profit Microsoft corporation, is returning better results than Google and is still struggling to retain major market share, could it not be that Bing is allowing itself to be artificially more accurate just to gain ground? Once the market share is locked down, they will likely allow in more advertised results.
Attorney Tom Brady (Score:5, Interesting)
Bing scored 62 and google 53. Google lost 5 points because it didn't find an attorney named Tom Brady and Bing gained 5 points because they found it. Remove this one query and google actually wins by a point.
But what google does really well is get current results. Search for "attorney tom brady" now and you will find TFA on google, but not on bing.
Re:Attorney Tom Brady (Score:5, Interesting)
That "current bias" on Google is, imho, more of a liability than an advantage.
Once any term becomes at least somewhat popular, it also becomes "self-sustaining" on Google - which means that any attempts to look for truly relevant information bring up only more and more recent "meta-discussions".
This also means that finding anything that hasn't happened recently on Google becomes more and more difficult. Their time-based index is severely broken (showing recent results as if they are from the past etc).
Questions not Skills (Score:5, Insightful)
I think the author's assumption that people would search for "When are the Patriots playing next year?" rather than "patriots game schedule" is flat out wrong. People know they are using computers, and not talking to a person, and they compensate accordingly. Google therefore, also compensates accordingly, by finding every page on the internet with "patriots", "game", and "schedule" in some close proximity. They may (and probably do) do more, but Google's approach has always been index everything you possibly can, and NLP has always taken a back seat. The Bing folks on the other hand have explicitly tried to optimize for NLP cases. However which engine is better isn't a matter of can you ask it questions in English, but can someone find what they are looking for. Given that most people know that "Googling" is not the same as asking a question, it is not fair to only test NLP queries.
Don't know if it matters (even if true) (Score:2)
When I do searches, Google works very well for me. I can't think of the last time I was frustrated when searching for something. So, let's say it's true that Bing is slightly better (and I'm not granting that; it's just for the sake of argument) - what's my motivation for switching to Bing if Google is already working just fine for me? My search needs are already being met.
Gaming search engines... (Score:2)
Newer and less widely used search engines often have better results, because there are thousands of spammers out there trying to game the bigger search engines.
Google's lead was simplicity, and it's dying (Score:2)
Google was my choice because of its simplicity. They're systematicly destroying what made me favor them in the first place. They Bing-ified their image search just recently, and it sucks. It crashes IE if you load a 2nd page of results. Maybe they're trying to foist their own version of lockin on us.
Google needs to get back to its roots, or somebody will come along with something better. The real Google killer might even be FaceBook-based search, where real human beings in your network (or extended netw
I Don't Google Much These Days (Score:2)
Laws of reciprocity (Score:2)
Don't forget a lot of people hate Microsoft generally for being douches. Don't really care how good Bing has gotten will never ever use it. Microsoft is a terrible company and would just as soon see them go bankrupt. What goes around come around.
Google indexes a lot faster than Bing (Score:2)
If I open a topic at a large forum and Google for it 15 mins later, the result is shown on the first page. I imagine the same goes for new stories in Slashdot: you can find them using Google almost instantly after they're published.
I don't think Bing comes even close to what Google is doing.
Re: (Score:2)
So a priest, a rabbi, and Marc Andreessen walk into a search bar ...
Google no longer the Default ? (Score:2)
...Google [h]as reached "Default" status ...
But is it losing the default status? My understanding is that in China people are starting to use "baidu" as a verb. "I'll baidu that" as someone in the west may now be saying "I'll google that".
:-). For a while people used "xerox" as
A search engine is a pretty simple thing to replace. I don't think many users care who provides search, they just want decent results. Google will have to work hard to maintain their position or they may for the most part become a verb, well except on Android based systems
Re: (Score:3)
Re: (Score:3)
Re: (Score:3)
Re: (Score:2)
I wonder what percentage of techies are like that?
Growing steadily smaller even as I write this. MS, it has turned out, has become less arrogant than Apple and less evil than Google. Who'd a thought?
Re: (Score:3)
Huh? I'm forced to run IE8 at work.. and we have Google as the search engine default...
Go Tools, Internet Options, then click Settings under "Search"... can change it in there.
Re: (Score:2)
Re: (Score:2)
Oh wait. I hate to respond to my own message, but Googling Google and Binging Bing makes the conclusion obvious.