To Break Google's Monopoly On Search, Make Its Index Public (bloomberg.com) 135
Robert Epstein, an American psychologist, professor, author and journalist critical of Google, argues that Google's monopoly on search can be broken by making its index public. An anonymous reader shares an excerpt from the report via Bloomberg: Different tech companies pose different kinds of threats. I'm focused here on Google, which I've been studying for more than six years through both experimental research and monitoring projects. (Google is well aware of my work and not entirely happy with me. The company did not respond to requests for comment.) Google is especially worrisome because it has maintained an unopposed monopoly on search worldwide for nearly a decade. It controls 92 percent of search, with the next largest competitor, Microsoft's Bing, drawing only 2.5%. Fortunately, there is a simple way to end the company's monopoly without breaking up its search engine, and that is to turn its "index" -- the mammoth and ever-growing database it maintains of internet content -- into a kind of public commons.
Doesn't Google already share its index with everyone in the world? Yes, but only for single searches. I'm talking about requiring Google to share its entire index with outside entities -- businesses, nonprofit organizations, even individuals -- through what programmers call an application programming interface, or API. Google already allows this kind of sharing with a chosen few, most notably a small but ingenious company called Startpage, which is based in the Netherlands. In 2009, Google granted Startpage access to its index in return for fees generated by ads placed near Startpage search results. With access to Google's index -- the most extensive in the world, by far -- Startpage gives you great search results, but with a difference. Google tracks your searches and also monitors you in other ways, so it gives you personalized results. Startpage doesn't track you -- it respects and guarantees your privacy -- so it gives you generic results. Some people like customized results; others treasure their privacy. In closing, Epstein writes that dozens of Startpage variants would turn up within months of opening up access to Google's index. "Many would target niche audiences -- some small, perhaps, like high-end shoppers, and some huge, like all the world's women, and most of these platforms would do a better job of serving their constituencies than Google ever could," he writes.
"These aren't just alternatives to Google, they are competitors -- thousands of search platforms, each with its special focus and emphasis, each drawing on different subsets of information from Google's ever-expanding index, and each using different rules to decide how to organize the search results they display. Different platforms would likely have different business models, too, and business models that have never been tried before would quickly be tested."
Doesn't Google already share its index with everyone in the world? Yes, but only for single searches. I'm talking about requiring Google to share its entire index with outside entities -- businesses, nonprofit organizations, even individuals -- through what programmers call an application programming interface, or API. Google already allows this kind of sharing with a chosen few, most notably a small but ingenious company called Startpage, which is based in the Netherlands. In 2009, Google granted Startpage access to its index in return for fees generated by ads placed near Startpage search results. With access to Google's index -- the most extensive in the world, by far -- Startpage gives you great search results, but with a difference. Google tracks your searches and also monitors you in other ways, so it gives you personalized results. Startpage doesn't track you -- it respects and guarantees your privacy -- so it gives you generic results. Some people like customized results; others treasure their privacy. In closing, Epstein writes that dozens of Startpage variants would turn up within months of opening up access to Google's index. "Many would target niche audiences -- some small, perhaps, like high-end shoppers, and some huge, like all the world's women, and most of these platforms would do a better job of serving their constituencies than Google ever could," he writes.
"These aren't just alternatives to Google, they are competitors -- thousands of search platforms, each with its special focus and emphasis, each drawing on different subsets of information from Google's ever-expanding index, and each using different rules to decide how to organize the search results they display. Different platforms would likely have different business models, too, and business models that have never been tried before would quickly be tested."
Startpage (Score:5, Informative)
https://www.startpage.com [startpage.com]
I'm Google and this is your brain on drugs (Score:5, Insightful)
Re: I'm Google and this is your brain on drugs (Score:2, Funny)
Socialism, Google! Get with the program. Your effort is our free gain! What is wrong with you?
Re: (Score:1)
I'll share my index when all cable companies and telcos in the US share their wires with anybody that wants to start a cable company or telco.
Happened in germany. Telekom wasn't a company before ca. 1990 all the cables were owned by the country. But the government created Telekom and sold all their cables to them. A few years later when other cable companies started to emerge, courts ordered Telekom to share their cables for a minimal fee, because these other cable companies said they coudn't compete otherwise.
Re: (Score:2)
Would sharing it even be much of a cost saving for other companies? Either they would have to pay for a massive data feed and then build their own high performance distributed search system, or they would have to pay to use Google's.
Collecting the data isn't the hard part, it's building the search engine that is.
Anyway, search is a saturated, mature market. Even Bing mostly gives you the right answer most of the time, there just isn't that much more anyone can do beyond niche search engines like Shodan whic
Re: (Score:2)
...or Ecosia:
https://www.ecosia.org/ [ecosia.org]
API (Score:1)
Anyone can scrape the contents of Google searches. There are companies out there that do it for you.
https://serpapi.com/ [serpapi.com]
Re: (Score:2, Insightful)
That's not even remotely the same thing. An API would come with all sorts of additional data about each entry, and actually contain each entry... unlike a search for 'clothes' which stops a few hundred pages deep (eg, a few thousand urls).
Do you really think the word 'clothes' only appears 3000 times on the entire web?!
Your assertion is bizarre.
Better alternative to breaking up Google outright! (Score:1)
Google might not like the idea of being forced to make their index public, but given the recent pushes to outright break-up some of these giant near-monoplies, they might like it a lot better than being broken-up completely!
Car Analogy (Score:1)
To break Ford's monopoly on cars, make Ford give Chevy it's engines and frames.
Re: Car Analogy (Score:2, Insightful)
You're missing something there. A better car analogy would be:
Make Ford form an agreement with Chevy which requires Ford to manufacture components for Chevy, using designs provided by Chevy but also including proprietary Ford components. Chevy must pay for this service at a rate determined by the government.
I Dislike Google... (Score:1)
...as much as most. But just for the sake of argument suppose that the index was made public, anyone could access it against an API and that this spawned hundreds, even thousands of competitors in the search space. Who would pay to maintain the index? If Google were not making money off of the index, why would they continue to pay for it? I would be opposed to governments controlling it for the same reasons I don't like Google.
We would. With taxes. B-b (Score:3)
Who would pay to maintain the index?
We would. With taxes.
Forcing Google to give competitors access to its index would constitute a "taking" under the Fifth Amendment. So Google could demand the full value and the fed would end up paying.
And that means the taxpayers would pay, whether they use the engine or not, with the government taking a hefty cut.
They might pay in taxes. They might pay more - with additional accumulated interest plus effects on other pricing - if it's deficit financed. They might pay
Re: We would. With taxes. B-b (Score:2, Insightful)
Holy crap- you realize their revenue is in excess of $30B for that part of their business, right? And that plenty of "internet" companies are valued by markets well in excess of 10x revenue?
I think adding 3% to our national debt should be reserved for very serious shit, not just "we don't like google."
Re: (Score:1)
Why should google give up it's index? It is not a monopoly. If they were a monopoly then none of the other search engines would exist. People have chosen to use google because the provide a worthwhile service that is not riddled with obnoxious ads. Or have you forgotten what search engines were like before google came into existence?
If you want web search index data, go build your own web crawling bot and gather it yourself the same way google does it.
Why? (Score:1)
Before we jump to breaking a monopoly first we need to define what it is to be a monopoly and how it is being used improperly. Google has many monopolies, a monopoly on advertising, a monopoly on the mobile app space, and both of those are actively being abused.
However Search engines are completely fungible. People are free to use whatever search engine they want. Google represents 90% of the Search market, but it is not the default option on any desktop OS. It is only the default option on a subset of brow
Re: (Score:2)
Re: Why? (Score:1)
iPhone isnâ(TM)t an industry, itâ(TM)s a product. Everyone has a monopoly on their own product.
Re: (Score:2)
THEY have a ONE HUNDRED PERCENT MONOPOLY on the entire iPhone industry
No they don't because there is no market or industry around iPhone. Having one hundred percent control over your own product doesn't make you a monopoly, it just makes you a seller of a product. If iOS ran on other devices, then you could start talking about monopolies, but at present the only space Apple competes in with phones is one where they have too low of a market share to be considered a monopoly.
Now the courts are still working on the app space within the iPhone. But it is likely that case will be
Re: (Score:2)
How? Apple doesn't get a penny from the sale of a used iPhone. The problem is, the newest iPhones don't have new stuff that people feel it's worth the price to upgrade. Mature tech is mature tech. "Good-enough" is doing the same thing as happened in other tech devices, like laptops and desktops.
My 5-year-old laptop has 8 gig of RAM - I don't even need half that (wipes Windows). It's a quad core with 2 threads per core - again, still overkill. The 500-gig hard drive more than 400 gigs free. The battery wi
Re: (Score:2)
Google is the default search engine on FireFox and Chrome and Safari, so it probably is the default on a lot of desktop OSes. Being something *desired* (your emphasis) by customers doesn't make something not a monopoly.
Re: (Score:2)
I didn't say being desired doesn't make it not a monopoly. Being 100% fungible with no barrier to entry makes it not a monopoly. Google is doing nothing to prevent you from typing www.bing.com into your browser. But you won't largely because it's a choice you make. That is quite different from e.g. not being able to access that site from within Chrome.
Re: (Score:2)
Being 100% fungible with no barrier to entry makes it not a natural monopoly. It still can be a monopoly. Standard Oil sold "shit we dug up from the ground" that was identical to other oil (to the extent we cared at the time).
Also, I would say there are a ton of barriers to entry. One is the huge fixed cost of building an index. A cost that the article is saying we should remove by forcing Google to open it's index. It's literally exactly what we are talking about.
Robert Epstein goes googli again (Score:4, Informative)
Just about anything might originate from Robert Epstein. The guy is all over the map.
Here's Epstein on the joy of rubbing two sticks together, long enough:
You might eventually get enough of spark to ignite a little tuft of desiccated moss, and then with lots of careful blowing, a tiny lick of flame.
In some ways this is actually true: the worst thing for a teenaged mind is another teenaged mind. Only evolution doesn't see it that way, and they're all obsessed with each other.
This is the same man, it seems, who now wants to offer economic advice to a near trillion-dollar enterprise.
My infection, your problem. Your index, my gigolo.
Re: (Score:2, Insightful)
Then there is the confusion of ideas that is:
I'm talking about requiring Google to share its entire index with outside entities -- businesses, nonprofit organizations, even individuals -- through what programmers call an application programming interface, or API. Google already allows this kind of sharing with a chosen few, most notably a small but ingenious company called Startpage, which is based in the Netherlands.
It isn't that the API is not available to anyone, it's that at this scale it is stupid expensive.
Startpage uses the exact same API, but has a different contract to pay a different amount.
They take your search terms, anonymize it, rip out any identifying info, then as a middleman proxy they perform the search lookup on your behalf, relaying the results.
In essence Google only sees the startpage servers making tons of search queries themselves, so ideally has no inf
Very Premature (Score:2)
Epstein "argues that Google's monopoly on search can be broken" as if the decision to break up Google has already been made. He doesn't even provide any basis for breaking Google in the first place other than his own dislike of the company.
Why even discuss how to do it when nobody is even close to mandating that it should be done at all?
Re: (Score:2)
Or Hosts File Obsessive-Compulsive Disorder? An obsession becomes a disorder when it interferes with the ability to lead a normal life - 25 years without a real job so you can fill forums with self-agrandizi
what an idiot (Score:2, Insightful)
to "break" googles "monopoly" (which it really doesn't have) you just need to make a better search engine. Google earned their place. Stealing from them is not the answer.
Begs the question (Score:2)
What problem are you trying to solve? Your jealousy over Google's wealth? Monopolies aren't bad. Is Google abusing its monopoly? Hit them with antitrust. Otherwise, stop trying to tear down everything successful just because you can't cut it in the competition.
What? (Score:1)
Google is especially worrisome because it has maintained an unopposed monopoly on search worldwide for nearly a decade
Protip: Duck Duck Go, Bing, Yahoo, and others all currently exist and do work. Duck Duck Go in particular is gaining marketshare.
Next time use the internet before you add more reasons to consider journalists as being dumber than burger flippers.
At least a burger-flipper gives me a sandwich. What value do journos provide?
Wait a minute (Score:1)
How is this man making criticisms of Google while being indicted for prostitution?
Robert Epstein is an IDIOT! (Score:5, Insightful)
Wait, that is exactly what Google does! Imagine that.
As for Google making their index public. I'm sure they would be fine with that. The actual index is just data that anyone else can gather. Trust me, it isn't that hard. I've worked on a project with a small team that made one of these targeted search engines. The index is extremely easy to build, if you have the storage space. Even storage space isn't as much of an issues as it was back in the '90s.
Where Google shines is its interpretation of this index. What it displays, how it ranks everything, how it interprets what you are actually looking for. That is the secret sauce that makes Google Search work so well. Not the web site data they collect, but how they process it. If you want to use their secret sauce, then it is going to track you, because guess what? It is the tracking information that makes it work how it works. It get to know you, it gets to know people in your city, in your state/province, in your country. I tries to tailor the information to you.
Scary? Maybe, but Epstein fails to understand that the two pieces are not separate, and the index by its self is not very useful. The secret sauce works so well because it tracks you.
Re: (Score:2)
Re: (Score:2)
I agree that the index isn't what's valuable, but I'd wager two decades worth of (trillions of?) search results aggregated would still make them nearly impossible to compete with, even if you made them delete all the person-specific data and open-sourced the index.
Punish Success (Score:1)
Step 1: Private company develops profitable method to do [whatever]
Step 2: Competitors complain they can't compete because [whatever whine here]
Step 3: Competitors hire lobbyists who buy politicians who claim Private Company should have to share their [whatever]
And here we are.
The underwear gnomes would be proud.
Any supporter of capitalism, a free market, supply and demand, and freedom of choice would not.
Way to punish success.
E
SEO (yes, I know it's all just bullshit anyway) (Score:1)
"... each using different rules to decide how to organize the search results they display ..."
And you think the joke that is SEO now is bad. As a web developer/host, I already want to murder all these asshat SEO companies that just run a script, generate a billion errors, and fool our stupid customers into believing their bullshit. This will just make it so much worse.
What Monopoly? (Score:3, Insightful)
Last I checked, there's bing, duckduckgo, yahoo, and a myriad of small fry search engines.
A monopoly is where there's ONLY ONE provider of a service. This is not a monopoly. This is just a good search engine that most people like. There's plenty of other choices out there. Feel free to use any or all of them.
A monopoly would mean you don't have a choice. Google or go home. But that's not how it is. Please, learn to use your words properly.
Re: (Score:2)
Google is a de-facto monopoly
Maybe. BIG MAYBE. They are quite dominate, but I wouldn't go as far to say as their competitors 'may as well not exist,' as your definition suggests what a de-facto monopoly is. I'm sure Microsoft and whoever runs Duckduckgo would disagree about their relevance in the marketplace of search engines.
Some foreign search engines might also disagree. We're more than welcomed to use those foreign search engines, you know. If you want to see what a REAL monopoly looks like, look no further than Bell Systems p
Comment removed (Score:4, Insightful)
Re: (Score:2)
Actually they don't. No lawsuit has been brought against Google's Search Engine and determined that they have a monopoly in the Search space. They have monopolies in many other spaces though and those lawsuits rightfully have seen them slapped on the knuckles a few times.
Re: (Score:2)
Several lawsuits destined to fail disagree with you.
FTFY. Cuz if they're trying to lawsuit that Google is a monopoly, they gunna get laughed out of court, like I'm laughing now.
Pro-freedom division of the google (Score:2)
I think it would be better to make separate copies of everything and give one to each of the daughter competing companies when you cut it up. From one no-choice option, everyone would suddenly have REAL choices and meaningful freedom in choosing their search engines. Each of the daughter clones would be free to change things going forward, and I would pick the daughter that does the best job of protecting my personal information.
Initially no one would have to make a choice. The old google URLs would simple
Re: (Score:2)
Re: (Score:2)
Determine your competitors greatest assets.
Figure out which legislation you can lobby to attack those assets.
Use those assets for your own good.
Profit ($)
Google is not a monopolist in search! It's adwords (Score:4, Insightful)
What Google is is a monopolist in is advertising. There is a huge desire for there to be one market where people selling web ads and people seeking places to advertise can come together. Google owns this market. The barriers to entry for this are now almost insurmountable. Gmail, android, search, google docs, etc., these are just a moat around the castle that is adwords (now google ads). Maybe at one point all these defenses were necessary but now google could stop all of them over night and adwords would live on.
Not gonna happen. (Score:2)
This crackpot notion would be the biggest eminent domain case in the history of history. Anyone actually think Google would give up its index without compensation? Not a chance. They'll fight this effort tooth and nail. Google is valued at the better part of a trillion dollars. And most of that value comes from its search engine. GCP and G-suite are mere gravy in comparison. And do they have anything else that makes money?
Try to take away their search index, and you can take it to the bank that they'
what happened to the google SOAP API? (Score:4, Interesting)
https://www.google.co.uk/searc... [google.co.uk]
i used this in the past, it was *really* effective. i used it to do a search-on-search: a "diff-search" if you will. basically you could put in a search criteria, and the code would go download *every* one of the pages on the search, every day (or hour).
then - and this is the key bit - a report could be generated on:
* what pages had been ADDED since the last search
* what pages had been REMOVED since the last search
* what pages had CHANGED since the last search.
this is actually far more useful for many people than just a "plain search". they have some keywords, there are 50,000 hits, they're never going to read them all: they want to know what *NEW* pages have been added (which over a 24 hr period will be a manageable number). they want to know if people CHANGED a page (not what it contained).
the API was taken down in 2006.
Monopoly? How does my search work without Google? (Score:2)
Been using DuckDuckGo pretty much exclusively for about a year now as my search engine.
Monopoly.
You keep using that word. I do not think it means what you think it means.
Let's steal from a company I don't like (Score:2, Insightful)
Re: (Score:2)
Re: (Score:2)
While you have a point ( though stealing is the wrong word, forced technology sharing is probably more apt), I am ambivalent to what extent we should _try_ to create competition in areas where apparently not much exists. Shared indexing actually seems like not much utility, (besides the fact that google's crawlers are more likely to not be blocked than your up-and-coming-indexer). Perhaps more of the patent model where search results (and subsequent interactions) are proprietary for 5 years then must be op
Entitled much? (Score:2)
What gives this guy the idea that he, or anyone else, should take anything away from Google? Anyone else on this planet could create a search engine as good or better than what Google has created. With Google's rescinding of "Don't be evil", there is a huge market wide open for a Not Evil search engine.
Taking anything away from Google is evil and unethical. If you want something, then make it yourself. Don't take it from someone who already created it. They earned what they have.
OR (Score:1)
Links People (Score:2)
Uh, I love Google (Score:2)
Google's index? It's more complicated than that. (Score:2)
It's not like a database index, where you can just tweak the list of fields being indexed. Google's search is so good because, in part, they have many employees, from programmers to data analysts, working to improve search results. Are we going to ask Google to employ these people and give away the fruit of their labor?
Re: (Score:1, Insightful)
"Let us steal others work and make them pay for success. " = actually, unfettered capitalism...
Re: Yay (Score:1)
What you really mean is, Google accumulated years worth of data on everyone which they use to profile you. So actually, in a way that data is yours and you have paid for it. Remember, you are not the customer, you are the product in this case.
Re: (Score:3, Interesting)
You're right, Bloomberg media is never going to be put "into power."
But they're happy to clickbait this weird communist crap. Nothing "socialist" about this, thas would be if the government went and built ab index and let all the companies use it. This is just a communist expropriation proposal.
You're safe from this sort of proposal as long as the Democrats control at least one part of government, because American liberals are also strongly libertarian.
Re: (Score:1)
Again I go away for a few hours and you accuse others of being me.
Think of it - so I think that either sex would have anything to do with you when it comes to sex? I know you're desperate, you being an incel and all, but odds are you will be a virgin for a LONG LONG time.
It's for the same reason you haven't been able to get a real job for 25 years - you live in a tiny dead-end town where everyone is dependent on government payments, either directly or one step removed. There just isn't enough of a popu
Re: (Score:2)
Re: (Score:2)
It was stupid of letting that slip out and not expecting me to pick up on it. You don't have any relevant work experience for today's world. You're not just obsessed with me, you're still obsolete and incompetent. Get help.