Forgot your password?
typodupeerror
Google Microsoft The Internet Technology

Google vs. Bing — a Quasi-Empirical Study 356

Posted by timothy
from the unbearable-lightness-of-bing dept.
eko3 writes "SearchEngineLand.com is featuring an article that compares Google's result query relevance performance to Microsoft's Bing. Through the author's methodology and very small sampling, he argues Bing returns slightly more relevant results than Google. The article suggests that Google is riding its current market success based on its legacy namesake when internet search used to be a lot more painful than it is today."
This discussion has been archived. No new comments can be posted.

Google vs. Bing — a Quasi-Empirical Study

Comments Filter:
  • O No (Score:5, Insightful)

    by ae1294 (1547521) on Thursday January 13, 2011 @04:22PM (#34867274) Journal

    Through the author's methodology and very small sampling,

    Science Fault Detected! Engaging TL;DR.

    • Re: (Score:2, Funny)

      by Anonymous Coward

      Bing returns slightly more relevant results than Google.

      This just means his sample was porn searches.

    • Re:O No (Score:5, Informative)

      by KiloByte (825081) on Thursday January 13, 2011 @05:23PM (#34868368)

      Following the "science fault" route:
      * what does the article's author do for a living? Falsifying of search return.
      * does the site that published this study have ties to the "winner"? It's among their "sponsors and partners" page.

      Somehow, nearly every time you find an "independent" study giving sensational results, it is sponsored by someone with a vested interest in those results.

      • Re:O No (Score:4, Interesting)

        by the_womble (580291) on Thursday January 13, 2011 @11:27PM (#34873226) Homepage Journal

        Of the three searches he discusses in the article as examples of Bing doing better than Google, I get the same top two results on both search engines for one, and for another he is giving Bing credit for putting two Linkedin pages at the top (not a good thing IMAO) and for the third he thinks that having a keyword stuffed spammy affiliate site, that does not actually have the tickets searched for available, as the first result is better than having an out of date news story.

        If I was scoring it then Google would have won.

        I actually tried using the three major search engines for a few days, using blind search [fejus.com] , at the time Bing came out, and Google was the clear winner then.

    • Science Fault Detected! Engaging TL;DR.

      Indeed, and I'd like to know what precisely is "quasi" about the "empirical"ness of it? Seems to me it is completely empirical and that neither "quasi", "empirical" nor "quasi-empirical" would have very much to do with scientific worth.

  • by thomasdz (178114) on Thursday January 13, 2011 @04:24PM (#34867304)

    As I sit here surfing the web on my Digital Equipment Corp. VAX 4000, I wonder... why is there no comparisons to AltaVista... the king of search engines.

    • by Monkeedude1212 (1560403) on Thursday January 13, 2011 @04:35PM (#34867516) Journal

      Because there IS no comparison to AltaVista. Good or Bad!

    • by countSudoku() (1047544) on Thursday January 13, 2011 @04:39PM (#34867576) Homepage

      You lucky, lucky bastard. I only WISH I could afford a sweet chunk of iron like the DEC VAX 4K! I'm on a Commodore Vic 20 connected to CompuServe and I can't search shit! In my day we'd have to use our HP programmable calculator connected to a dodgy barcode reader the size of a small aircraft to parse through the pages of a phone book, and we LIKED it that way. Darn, whippersnappers on my lawn, gotta get the rake...

    • Hey, I have a 4100 right over here. 3 feet away. I don't surf with it.

          And hey, Altavista worked great, I had no complaints about it. Only when it was obviously on the way down did I start using google.

            Brett

      • by afidel (530433)
        Actually until they switched to using Yahoo's engine I continued to use it for some searches because they had the near operator which Google has yet to implement.
      • AltaVista respected punctuation. If you're searching for something with a dash in it, for example, or for the use of some operator in a programming language, AltaVista found it for you.

        Google strips your punctuation right away, making it largely useless for such searches.

    • by martas (1439879)
      There could be, in a much more scientific way, as follows: create a wrapper search engine that randomly chooses one of any number of search engines (google, bing, altavista, lycos, and heck, even yahoo) without telling the user what it chose, performs the user query, returns results, and asks the user to rate relevance. The results would be perfectly unbiased; the only drawback is the possibility of a huge lawsuit from all the search engines you'd essentially be ripping off (no ad revenue!).
    • by Toe, The (545098) on Thursday January 13, 2011 @04:50PM (#34867766)

      Did you know Excite [excite.com] is still around? I had no idea.

      This list is pretty amazing for some nostalgic perusal.
      http://en.wikipedia.org/wiki/List_of_search_engines [wikipedia.org]

      (Now as for that VAX... No! Bad!)

    • by Yvan256 (722131)

      AltaVista? WebCrawler is much better, it has a mascot!

    • by jbeaupre (752124) on Thursday January 13, 2011 @04:58PM (#34867896)

      I guess not many people used it, but AltaVista Personal did an amazing job of indexing and searching local and network files. Faster than any of the "modern" OS integrated offerings I've seen. And without sucking up resources. If there were a version for XP/Vista/Win7 I'd use it in a heartbeat.

  • by KublaiKhan (522918) on Thursday January 13, 2011 @04:24PM (#34867306) Homepage Journal
    Google's primary business function is 'search', though they've attempted to diversify with documents and the like.

    Microsoft's primary business function is documents and the like, though they've attempted to diversify with search.

    There's a very low barrier to individual users to choose between them for either (given that MS has put its document processing online for free, last I heard) so, in the end, it's likely that the superior product (whether marketed better or actually better) will triumph in marketshare.

    Bring this back up in 18 months, and we'll likely see some clear differential if there really is an actual difference in the applicability of either one's functions.
    • by Anonymous Coward on Thursday January 13, 2011 @04:38PM (#34867554)

      Google's primary business function is 'search', though they've attempted to diversify with documents and the like.
      Microsoft's primary business function is documents and the like, though they've attempted to diversify with search.

      Google's primary business function is 'global hegemony'.
      Microsoft's primary business function is 'global hegemony'.

      FTFY.

    • by node 3 (115640)

      There's a very low barrier to individual users to choose between them for either

      Not as low as you might think. Network effects can skew results for people that intend to share or collaborate with their documents. Just like the justification for a lot of people who choose MS Windows or MS Office, what their friends and/or business associates uses plays a big role here.

  • by KnownIssues (1612961) on Thursday January 13, 2011 @04:27PM (#34867358)

    A single person's subjective analysis of 20 search terms is a small sample indeed! I will say, Bing has come a long way in producing search results I feel are useful, but I still find myself frequently forgetting Bing is the default search, coming up with bizarrely useless results, switching to Google, and saying to myself, ah yes, these are the results I was expecting.

    Perhaps I've just learned to produce search results in Google that meet my needs and haven't developed that skill in Bing. A more thorough, less subjective analysis comparing the two search engines would be very interesting. Sadly, I think this writer's personal conclusion is just going to spark a nerd-war over Google vs. Microsoft filled with subjective opinion (like mine) and little empircal evidence.

    • by jayme0227 (1558821) on Thursday January 13, 2011 @04:35PM (#34867508) Journal

      To be fair, he's not trying to get this published in a journal, just point out that Google is no longer streets ahead of everyone else. I think that is a fair assessment.

    • by mcgrew (92797) * on Thursday January 13, 2011 @04:39PM (#34867566) Homepage Journal

      Indeed. The study's methodology was, to put it mildly, badly flawed. A far better methodology would have had twenty other people do the searching, and have THEM rank the results. That would still have been flawed, too, but less subjective than just having one guy decide how relevant the searches were.

      Google is still #1 because people tried Bing and found it wanting. I did, the first day it was out.

      • Re: (Score:3, Interesting)

        by mordero (1118893)

        Google is still #1 because people tried Bing and found it wanting. I did, the first day it was out.

        Isn't that the point being made? When Bing was first launched Google may have had better results, but now Bing is catching up with Google and (maybe?) surpassing it in terms of relevancy. Google is slow on adjusting its algorithm since some/most/all people have the perception that it is better than Bing and since those people never go back to try Bing again, Google has little need to adapt as quickly.

      • by afidel (530433) on Thursday January 13, 2011 @05:41PM (#34868636)
        I just did a quick check to see if Bing had actually improved since I last used it, I type "vmware vma password saver" without the quotes into both bing and google since I had read a blog post today about a cool feature but couldn't remember the exact command, googles first result was the vma release notes that talk about the feature, bing had no relevant results in the entire first page. Doesn't look to me like googles in any danger of losing my eyeballs.
    • by Totenglocke (1291680) on Thursday January 13, 2011 @04:46PM (#34867710)
      Yea.............I don't buy this guys results at all. I've used Bing and Google plenty of times - I stopped using Bing due to it rarely giving me what I wanted and mostly just giving links to MS products as search results.
  • Anyone care to try to replicate the results? You could probably just use his list, or create a list of your own if you really want to. I'd do it myself but I'm supposed to be working.

    • by jeffmeden (135043)

      Anyone care to try to replicate the results? You could probably just use his list, or create a list of your own if you really want to. I'd do it myself but I'm supposed to be working.

      Duh, just post your actual work to an "ask slashdot" post, then come back and work on this instead. I mean come on, what are you, new here?

    • I use DuckDuckGo as my main search engine, and it almost always gives me the page that I want early on. If I can't find the result there, I go to Google. So far, I've not come across a single instance where Google returns a useful result but DDG doesn't. The main difference is that DDG admits when it can't find anything relevant, but Google gives you 10 pages of irrelevant ad-filled pages that you might like to look at.

      When I started using Google, around 2000, this would not have been the case - Google

  • by nlawalker (804108) on Thursday January 13, 2011 @04:28PM (#34867376)

    I agree with the notion that Google is riding it's legacy of taking search from something that was literally an impossible problem to solve to something that was instant. It earned every bit of that, but search has entered a new era.

    Bing is now competing at the forefront, which is taking search from finding results in an index to finding answers to questions and solving problems. "Decision engine" is a bit overhyped, but it's the right direction to move in, in my opinion. This is a good thing, because Bing and Google will push each other.

    I generally refer friends and people I know to Bing because they tend to treat search engines like a natural language processor, or as a companion that can help them answer questions and solve problems.

    Google is still (much) more effective if your Google-fu is powerful, but if it's not, Bing can be a bit friendlier and better at getting you to what you want to see.

    • by Totenglocke (1291680) on Thursday January 13, 2011 @04:50PM (#34867768)

      I'm sorry, you must be using a different Bing than I do. Your statements regarding Bing's performance do NOT match up to my experience with it in the slightest.

      Perhaps you're confusing bing.com with google.com/bing ? =)

    • by I8TheWorm (645702) *

      Oh the number of times I still see people type "what is the population of equador?" or something formatted that way into the search bar. I suppose we have jeeves to thank for that.

  • by Compaqt (1758360) on Thursday January 13, 2011 @04:29PM (#34867396) Homepage

    But I still don't know how to change the water filter [google.com] on a Frigidaire Professional Series [bing.com].

    For some reason, they gave Bing 7 points for that query.

    But the first result merely regurgitates the question, then has an ad link for Fixya.com.

    • Interesting. My first result on Bing was: this [frigidaire.com]. Google, however, was useless.
      • by bberens (965711)
        Yeah, the first two results in Bing were included in the first 3 results in Google for me. *shrug*
      • by Ardaen (1099611)
        Interesting, Bing gave me ad-laden junk for the first 4 results, the fifth was the www.frigidaire.com main page. I didn't get anything like your first result.

        I guess it likes you and hates me?
      • The bing link just sells you replacement filters, it doesn't tell you how to install them. Throw "manual" into the query and you'll get better results (bing and google).

    • by BForrester (946915) on Thursday January 13, 2011 @05:10PM (#34868120)

      Perhaps there's no existing webpage that answers this brutally obvious question. Here, Google and Bing. Crawl this:

      How to change the water filter on a Frigidaire Professional Series:

        - Push the button labelled "eject" on the old water filter
        - Remove the old water filter
        - Insert the new water filter

  • by GodfatherofSoul (174979) on Thursday January 13, 2011 @04:32PM (#34867454)

    Google just returns too many garbage marketing links. Bing isn't vastly better, just slightly. And, I imagine that if people start to migrate, they'll take on the same ad ratios as Google.

  • After trying to put up with Bing (being the annoying default in IE 7/8, and on my smartphone) it just doesn't hit the right notes with the kinds of searching I do. It's probably that it doesn't prioritize Wikipedia results high enough, though.

    • by Zumbs (1241138)
      Another amusing anecdote: When I started working at my current workplace, my work computer were a freshly installed W7 with IE8 using Bing. At my workplace we develop exclusively for a Windows platform, so I did a lot of searches for functions in the WinAPI. However, Bing consistently did not find the MSDN documentation - or even anything remotely similar - on the first page. When I changed the search engine to Google, the documentation was consistently the first or the second link.
    • by bberens (965711)
      I found this to be true. Some of that is likely due to the fact that I have, subconsciously, tuned my queries to produce meaningful results in Google. If I were for some reason forced to use Bing for a while I suspect I would be able to produce meaningful results from it as well.
  • Insignificant Result (Score:5, Informative)

    by phantomcircuit (938963) on Thursday January 13, 2011 @04:33PM (#34867490) Homepage

    20 searches, 15% margin, 100% subjective.

  • by ewhenn (647989) on Thursday January 13, 2011 @04:36PM (#34867528)
    It doesn't matter, google won.

    Generally speaking, to dethrone the entrenched standard (in any industry, not just search engines) you have to be substantially better to get people to switch to something they aren't used to. Marginally better just won't cut it. Cost is a moot point, because outside of MS paying me a check every month to use bing, you can't beat the price of free.

    Humans are generally animals of habit, and unless you give them a good reason to, they won't change.
  • by Animats (122034) on Thursday January 13, 2011 @04:37PM (#34867540) Homepage

    For about the first half of 2008, Yahoo search was better than Google search.

    Yahoo introduced specialized subengines - stocks, weather, movies, celebrities - which were triggered by matching queries. Each subengine had a special case for that class of information. Yahoo had about fifty such subengines.

    Nobody noticed. Yahoo's market share didn't move. I only knew about this because I went to a talk by the head of Yahoo R&D at the time.

    Bing's strategy seems to be mostly to follow Google. Google put Google Places into web search (a big mistake [sitetruth.com], because Places is so easy to spam), and Bing followed within days.

    This week, everybody from Techdirt to CNN is dumping on Google for their spam problem. Even Paul Krugman at the New York Times mentioned it. There's much blog talk of "human powered search" or "curated search" to stop the spam but the failure of Wikia Search, and the lack of interest in ChaCha, Swicki, and Rollyo, indicates that's a dead end. (Mahalo started as human-powered search and ended up as a content farm, which is a hint that "human powered" doesn't equate to "better". No complaints from search users about that, though.)

    (Note: I have a position in this; I run SiteTruth [sitetruth.com]. There, we try to find the business behind the web site, and rate that, using data from the SEC, BBB, D&B, and other hard data sources about businesses. This works well at eliminating spam. Too well for some sites; we get complaints about our hard-ass "when in doubt, rate it down" approach.)

    • by Animats (122034)

      (Oops, the line "No complaints from search users about that, though." was supposed to go in the last paragraph, not the one about Mahalo.)

    • Mod parent up. Fascinating paper linked deserves to be be Slashdotted in it's own right.

  • not for my searches (Score:5, Interesting)

    by siddesu (698447) on Thursday January 13, 2011 @04:38PM (#34867550)

    I use about six languages on a daily basis and IMHO bing sucks at everything that isn't English.

  • by Cro Magnon (467622) on Thursday January 13, 2011 @04:38PM (#34867564) Homepage Journal

    I get strange looks when I tell someone I Binged myself.

  • by spafbi (324017) on Thursday January 13, 2011 @04:42PM (#34867632)
    One avenue companies utilize in trying to get you to use their products and services is through TV advertisements. While I have seldom been swayed to use products or services because of a TV ad, I often go out of my way to NOT use products or services from advertisers with either annoying ads or ads which go out of their way to insult the viewers' intelligence. Given Bing's current 'search overload' annoyvertisement (yeah, I'm coining a new word here), and regardless of Bing's competence in producing useful search results, I'll use the more-than-adequate Google search results which are easily customized using a few easy to remember search operators (http://www.google.com/help/cheatsheet.html [google.com]).
  • by Anonymous Coward on Thursday January 13, 2011 @04:43PM (#34867664)

    It has been my experience that as Google has gotten bigger they seem to return at the top of their results pages that are nothing more than aggregating websites (most contain LOTS of google adverts too, which piques my thoughts on why they do show up at the very top of Googles searches). This is VERY annoying. As a result, I, previously a great supporter and user of Google, have been looking for a search engine that doesn't return websites that do nothing but hand me links to other websites. If i find one, that loads quickly, I will dump Google.

    If Google is listening, it should be very easy to stop the aggrigation websites (sites that have NO CONTENT but just contain links to other sites) from reaching the top of your results.

    • Give DuckDuckGo [duckduckgo.com] a try. I've been pretty satisfied with the results so far (I've been using it for a couple weeks after getting totally sick of the link farms on Google). They seem to be pretty strong on privacy too, if you're into that.

  • by i_want_you_to_throw_ (559379) on Thursday January 13, 2011 @04:46PM (#34867706) Homepage Journal
    You'll note that the story says "Sponsored by In-House SEO Exchange@SMX West". A quick visit to that site [searchmarketingexpo.com] shows that Bing is a Premier sponsor of SMX West.

    Of course Bing! is better than Google. Shenanigans! Or at the very least, suspect.
  • by MikeRT (947531) on Thursday January 13, 2011 @04:46PM (#34867712) Homepage

    Microsoft is notorious for working hard until they get it right and then steadily eating into their competition. They've practically torn Sony limb-from-limb in the video game market, something which would have been unthinkable in the early millennium when Sony was an unstoppable juggernaut that was able to destroy Sega just on "sheer perceived awesomeness" alone. From the initial reports about Windows 8, it sounds like they've fully grasped the OS X/iOS lesson and are moving toward a similar unified Windows product base.

    It's really amusing to me whenever I see people dismiss Microsoft as a dinosaur that is thrashing in a tar pit. They act like its collapse is "inevitable" like Microsoft is some sort of corporate Soviet Union. In the late 90s/early millennium, everyone was saying that Linux or this or that would kill them. Guess what? Windows 7 probably put the nail in the coffin for desktop Linux among mainstream users in the US and much of Europe.

    People mistake the fact that the market is competitive with Microsoft dying. It's more realistic to say that Microsoft is being forced to adapt and compete. If Windows 7 is their first real volley in that respect, I'd be cautious if I were one of their main competitors because it's obvious that Microsoft is taking these threats very seriously now.

    • Microsoft is notorious for working hard until they get it right and then steadily eating into their competition.

      So is Google.

      And Google's known for doing it more recently than Microsoft, and faster.

  • If Bing, the search engine created by the massive for-profit Microsoft corporation, is returning better results than Google and is still struggling to retain major market share, could it not be that Bing is allowing itself to be artificially more accurate just to gain ground? Once the market share is locked down, they will likely allow in more advertised results.

  • Attorney Tom Brady (Score:5, Interesting)

    by Col. Klink (retired) (11632) on Thursday January 13, 2011 @04:53PM (#34867806)

    Bing scored 62 and google 53. Google lost 5 points because it didn't find an attorney named Tom Brady and Bing gained 5 points because they found it. Remove this one query and google actually wins by a point.

    But what google does really well is get current results. Search for "attorney tom brady" now and you will find TFA on google, but not on bing.

    • by ugen (93902) on Thursday January 13, 2011 @05:26PM (#34868418)

      That "current bias" on Google is, imho, more of a liability than an advantage.
      Once any term becomes at least somewhat popular, it also becomes "self-sustaining" on Google - which means that any attempts to look for truly relevant information bring up only more and more recent "meta-discussions".
      This also means that finding anything that hasn't happened recently on Google becomes more and more difficult. Their time-based index is severely broken (showing recent results as if they are from the past etc).

  • by necro351 (593591) on Thursday January 13, 2011 @04:57PM (#34867882) Journal

    I think the author's assumption that people would search for "When are the Patriots playing next year?" rather than "patriots game schedule" is flat out wrong. People know they are using computers, and not talking to a person, and they compensate accordingly. Google therefore, also compensates accordingly, by finding every page on the internet with "patriots", "game", and "schedule" in some close proximity. They may (and probably do) do more, but Google's approach has always been index everything you possibly can, and NLP has always taken a back seat. The Bing folks on the other hand have explicitly tried to optimize for NLP cases. However which engine is better isn't a matter of can you ask it questions in English, but can someone find what they are looking for. Given that most people know that "Googling" is not the same as asking a question, it is not fair to only test NLP queries.

  • When I do searches, Google works very well for me. I can't think of the last time I was frustrated when searching for something. So, let's say it's true that Bing is slightly better (and I'm not granting that; it's just for the sake of argument) - what's my motivation for switching to Bing if Google is already working just fine for me? My search needs are already being met.

  • Newer and less widely used search engines often have better results, because there are thousands of spammers out there trying to game the bigger search engines.

  • Google was my choice because of its simplicity. They're systematicly destroying what made me favor them in the first place. They Bing-ified their image search just recently, and it sucks. It crashes IE if you load a 2nd page of results. Maybe they're trying to foist their own version of lockin on us.

    Google needs to get back to its roots, or somebody will come along with something better. The real Google killer might even be FaceBook-based search, where real human beings in your network (or extended netw

  • I hardly use Google as my search engine these days. 90% or so of my search engine queries go to DuckDuckGo [duckduckgo.com]. I only resort to Google if I am looking for music files or something so obscure that I know Google won't provide me a list of 1000+ potential hits, half of which are marketing or shopping sites. I never use Bing, but that's because I have a general hatred for Microsoft and am trying, vigorously, to cut them out of my personal computer use entirely. I also find using some specific protocol search engin
  • Don't forget a lot of people hate Microsoft generally for being douches. Don't really care how good Bing has gotten will never ever use it. Microsoft is a terrible company and would just as soon see them go bankrupt. What goes around come around.

  • If I open a topic at a large forum and Google for it 15 mins later, the result is shown on the first page. I imagine the same goes for new stories in Slashdot: you can find them using Google almost instantly after they're published.

    I don't think Bing comes even close to what Google is doing.

Whenever a system becomes completely defined, some damn fool discovers something which either abolishes the system or expands it beyond recognition.

Working...