An anonymous reader quotes a report from MIT Technology Review: In 1998 a couple of Stanford graduate students published a paper describing a new kind of search engine: "In this paper, we present Google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext. Google is designed to crawl and index the Web efficiently and produce much more satisfying search results than existing systems." The key innovation was an algorithm called PageRank, which ranked search results by calculating how relevant they were to a user's query on the basis of their links to other pages on the web. On the back of PageRank, Google became the gateway to the internet, and Sergey Brin and Larry Page built one of the biggest companies in the world. Now a team of Google researchers has published a proposal for a radical redesign that throws out the ranking approach and replaces it with a single large AI language model, such as BERT or GPT-3 -- or a future version of them. The idea is that instead of searching for information in a vast list of web pages, users would ask questions and have a language model trained on those pages answer them directly. The approach could change not only how search engines work, but what they do -- and how we interact with them.

[Donald Metzler and his colleagues at Google Research] are interested in a search engine that behaves like a human expert. It should produce answers in natural language, synthesized from more than one document, and back up its answers with references to supporting evidence, as Wikipedia articles aim to do. Large language models get us part of the way there. Trained on most of the web and hundreds of books, GPT-3 draws information from multiple sources to answer questions in natural language. The problem is that it does not keep track of those sources and cannot provide evidence for its answers. There's no way to tell if GPT-3 is parroting trustworthy information or disinformation -- or simply spewing nonsense of its own making.

Metzler and his colleagues call language models dilettantes -- "They are perceived to know a lot but their knowledge is skin deep." The solution, they claim, is to build and train future BERTs and GPT-3s to retain records of where their words come from. No such models are yet able to do this, but it is possible in principle, and there is early work in that direction. There have been decades of progress on different areas of search, from answering queries to summarizing documents to structuring information, says Ziqi Zhang at the University of Sheffield, UK, who studies information retrieval on the web. But none of these technologies overhauled search because they each address specific problems and are not generalizable. The exciting premise of this paper is that large language models are able to do all these things at the same time, he says.

The developers behind Niagara Launcher, a popular third-party home screen replacement app, have found new evidence in the Android 12 preview documentation, which suggests that Google is adding a new device search API in Android 12 that will let third-party launchers offer a similar universal search feature. XDA Developers reports: [T]he feature will give third-party launchers "access to the centralized AppSearch index maintained by the system." It further highlights that the AppSearch index is a search library for managing structured data featuring: A fully offline on-device solution; A set of APIs for applications to index documents and retrieve them via full-text search; APIs for applications to allow the System to display their content on the system UI surfaces; and Similarly, APIs for applications to allow the System to share their content with other specified applications. This feature will essentially provide a native alternative to universal search apps like Sesame, giving users the option to search for almost anything on their device in an instant.

AmiMoJo shares a report: In August of 2019, I published research from now-defunct clickstream data provider, Jumpshot, showing that 50.33% of all Google searches ended without a click to any web property in the results. Today, thanks to new data from SimilarWeb, I've got a substantive update to that analysis. From January to December, 2020, 64.82% of searches on Google (desktop and mobile combined) ended in the search results without clicking to another web property. That number is likely undercounting some mobile and nearly all voice searches, and thus it's probable that more than 2/3rds of all Google searches are what I've been calling "zero-click searches." Some folks have pointed out that "zero-click" is slightly misleading terminology, as a search ending with a click within the Google SERP itself (for example, clicking on the animal sounds here or clicking a phone number to dial a local business in the maps box) falls into this grouping. The terminology seems to have stuck, so instead I'm making the distinction clear.

[...] Here are the headline statistics from the data:
SimilarWeb analyzed ~5.1 trillion Google searches in 2020
These searches took place on the 100M+ panel of mobile and desktop devices from which SimilarWeb collects clickstream data
Of those 5.1T searches, 33.59% resulted in clicks on organic search results
1.59% resulted in clicks on paid search results
The remaining 64.82% completed a search without a direct, follow-up click to another web property
Searches resulting in a click are much higher on desktop devices (50.75% organic CTR, 2.78% paid CTR)
Zero-click searches are much higher on mobile devices (77.22%)

According to a Wall Street Journal investigation, hospitals are blocking confidential prices from web searches with special coding embedded on their websites. It's problematic because pricing information for hospital services must be disclosed under a new federal price transparency rule that went into effect on Jan. 1. Becker's Hospital Review reports: The code prevents pages from appearing in searches, such as a hospital's name and prices, computer experts told the Journal. While the prices are still there, it requires clicking through multiple layers of pages to find them. "It's technically there, but good luck finding it," Chirag Shah, an associate computer professor at the University of Washington, told the Journal. "It's one thing not to optimize your site for searchability, it's another thing to tag it so it can't be searched. It's a clear indication of intentionality."

Hospitals burying their pricing data include those owned by HCA Healthcare and Universal Health Services as well as the University of Pennsylvania Health System, NYU Langone Health, Beaumont Health and Novant Health, according to the Journal. Penn Medicine, NYU Langone Health and Novant Health told the publication they used the blocking code to direct patients first to information they "considered more useful than raw pricing data," for which they included web links. UHS uses the blocking code to ensure consumers acknowledge a disclosure statement before viewing prices and is making no effort to hide information, a hospital spokesperson told the Journal.

After the Journal reached out to hospitals about its discovery, the search-blocking code was removed from sites including those of HCA, Penn Medicine, Beaumont, Avera Health, Ballad Health and Northern Light Health. An HCA spokesperson told the publication the search blocker was "a legacy code that we removed," and Avera, Ballad, Beaumont and Northern Light said the code had been left on their websites by mistake.

Two years after publicly launching a privacy-focussed browser, Brave, founded by former Mozilla executive Brendan Eich, is taking on Google's search business, too. From a report: The announcement of Brave Search puts the upstart in the rare position of taking on both Google's browser and search dominance. Eich says that Brave Search, which has opened a waitlist and will launch in the first half of this year, won't track or profile people who use it. "Brave already has a default anonymous user model with no data collection at all," he says adding this will continue in its search engine. No IP addresses will be collected and the company is exploring how it can create both a paid, ad-free search engine and one that comes with ads.

But building a search engine isn't straightforward. [...] Eich says Brave isn't starting its search engine or index from scratch and won't be using indexes from Bing or other tech firms. Instead Brave has purchased Tailcat, an offshoot of German search engine Cliqz, which was owned by Hubert Burda Media and closed down last year. The purchase includes an index of the web that's been created by Tailcat and the technology that powers it. Eich says that some users will be given the ability to opt-in to anonymous data collection to help fine-tune search results. "What Tailcat does is it looks at a query log and a click log anonymously," Eich says. "These allow it to build an index, which Tailcat has done and already did at Cliqz, and it's getting bigger." He admits that the index will not be anywhere near as deep as Google's but that the top results it surfaces are largely the same.

Google says it's refusing to ditch planned changes to its cookie policy that attracted regulatory scrutiny and a wave of opposition from ad-tech companies and publishers. From a report: The Alphabet unit upended the advertising industry with its decision last year to phase out third-party cookies that help advertisers pinpoint customers with ads for websites they previously visited and monitor which ads convinced them to buy. "We're making explicit that once third-party cookies are phased out, we will not build alternate identifiers to track individuals as they browse across the web, nor will we use them in our products," David Temkin, Google's director of product management, ads privacy and user trust, said in a blog post on Wednesday. Google said last year that its so-called privacy sandbox initiative aims to tackle concerns people have about privacy and how their personal identity is used.

Google has threatened to disable its search engine in Australia if it's forced to pay local publishers for news, a dramatic escalation of a months-long standoff with the government. From a report: The proposed law, intended to compensate publishers for the value their stories generate for the company, is "unworkable," Mel Silva, managing director for Australia and New Zealand, told a parliamentary hearing Friday. She specifically opposed the requirement that Google pay media companies for displaying snippets of articles in search results.

The threat is Google's most potent yet as the digital giant tries to stem a flow of regulatory action worldwide. At least 94% of online searches in Australia go through the Alphabet unit, according to the local competition regulator. "We don't respond to threats," Australia Prime Minister Scott Morrison said Friday. "Australia makes our rules for things you can do in Australia. That's done in our parliament. It's done by our government. And that's how things work here in Australia."

Google is testing a new feature that will surface Instagram and TikTok videos in their own dedicated carousel in the Google app for mobile devices -- a move that could help the company retain users in search of social video entertainment from fully leaving Google's platform. From a report: The feature itself expands on a test launched earlier this year, where Google had first introduced a carousel of "Short Videos" within Google Discover -- the personalized feed found in the Google mobile app and to the left of the home screen on some Android devices. To be clear, this "Short Videos" carousel is different from Google's Stories, which rolled out in October 2020 to the Google Search app for iOS and Android. Those "Stories" -- previously known as "AMP Stories" -- consist of short-form video content created by Google's online publishing partners like Forbes, USA Today, Vice, Now This, Bustle, Thrillist and others. Meanwhile, the "Short Videos" carousel had been focused on aggregating social video from other platforms, including Google's own short-form video project Tangi, Indian TikTok competitor Trell, as well as Google's own video platform, YouTube -- which has also been experimenting with short-form content as of late.

Four years after offering special placement in a "top stories carousel" in search results to entice publishers to use a format it created for mobile pages, called AMP, Google announced last week that it will end that preferential treatment in the spring. "We will prioritize pages with great page experience, whether implemented using AMP or any other web technology, as we rank the results," Google said in a blog post. From a report: The company had indicated in 2018 that it would drop the preference eventually. Last week's announcement of a concrete timeline comes less than a month after the Department of Justice called Google a "monopoly gatekeeper to the internet" in a lawsuit alleging antitrust violations and as pressure mounts on officials in the European Union, which has already fined Google more than $9 billion for antitrust violations. "I did always think AMP posed antitrust concerns," said Sally Hubbard, author of the book "Monopolies Suck" and an antitrust expert with the Open Markets Institute. "It's, 'If you want to show up on the top of the search results, you have to play by our rules, you have to use AMP.'" Google spokesperson Meghann Farnsworth did not address the timing of the change but said AMP is not dead, saying the company is "fully committed to AMP as a technology." She said AMP continues to be required for certain features that "are not technically possible" without it, such as "swipe to visit" in Google Images, and that it's "preferred" in the "for you" feed in Google's news reading app, Google Discover.

Apple is stepping up efforts to develop its own search technology as US antitrust authorities threaten multibillion-dollar payments that Google makes to secure prime placement of its engine on the iPhone, Financial Times reported Wednesday [Editor's note: the link may be paywalled; alternative source]. From the report: In a little-noticed change to the latest version of the iPhone operating system, iOS 14, Apple has begun to show its own search results and link directly to websites when users type queries from its home screen. That web search capability marks an important advance in Apple's in-house development and could form the foundation of a fuller attack on Google, according to several people in the industry. The Silicon Valley company is notoriously secretive about its internal projects, but the move adds to growing evidence that it is working to build a rival to Google's search engine. Two and a half years ago, Apple poached Google's head of search, John Giannandrea. The hire was ostensibly to boost its artificial intelligence capabilities and its Siri virtual assistant, but also brought eight years of experience running the world's most popular search engine. The company's growing in-house search capability gives it an alternative if regulators block its lucrative partnership with Google. When the US Department of Justice launched a case last week, over payments that Google makes to Apple to be the iPhone's default search tool, urgency was added to the initiative.

A group of search engines from around the world are banding together to demand European Union regulators address Google's dominance in the online search market. They are also urging the EU to take a closer look at Google's controversial auction process. From a report: The news comes hot on the heels of the U.S. Department of Justice's (DOJ) antitrust case, which formally launched last week. The suit alleges that Google violates anti-competition laws by crowding out rivals in the internet search and advertising markets. DuckDuckGo (U.S.), Ecosia (Germany), Lilo and Qwant (France), and Seznam (Czech Republic) have penned an open letter to European Commission executive VP Margrethe Vestager asking her to take a "renewed look" at the policing of Google's search market dominance. As an initial step, the companies are calling for a trilateral meeting between themselves, the European Commission (EC), and Google to look at the issue of search engine competition in Europe and elsewhere. More specifically, Google's smaller rivals want to establish a more "effective preference menu," giving Android users an easier way to choose a default search engine when setting up their device for the first time.

Google doled out more than $1 billion last year to U.S. mobile carriers to distribute its search engine, according to the landmark antitrust lawsuit from the Justice Department. From a report: The DOJ suit, filed Tuesday, details several methods Google uses to make its search the default service on browsers, smartphones and other devices. That includes deals with Apple and Android manufacturers such as Samsung Electronics. Google also cut hefty revenue sharing agreements with major mobile carriers to box out competing search engines and browsers, the Justice Department said. In exchange for placing Google search as the default on phones, carriers received a portion of search advertising revenue. "If a carrier or manufacturer does not renew its revenue sharing agreement with Google, the distributor loses out on revenue share not only for new mobile devices but also for the phones and tablets previously sold and in the hands of consumers," the Justice Department said in the suit. "This provision is punitive to the carrier or manufacturer and helps to ensure that carriers and manufacturers will not stray from Google."

A system Google set up to promote competition on Android has left some smaller search engines having trouble gaining traction, fueling rivals' complaints about the tech giant's compliance with a European Union antitrust decision ahead of potential U.S. charges. From a report: Since March, Alphabet-owned Google has been showing people in Europe who set up new mobile devices running the company's Android operating system what it calls a "choice screen," a list of rival search engines that they can select as the device's default. The system is part of Google's compliance with a 2018 decision that found the company used Android's dominance to strong-arm phone makers into pre-installing its search engine. But some small search engines that are relatively popular in Europe failed to win spots in large European countries in the latest round of auctions to appear on the choice screen, according to people familiar with the results. The results, which cover the fourth quarter of the year, are set to be announced on Monday.

DuckDuckGo, maker of a U.S.-based search engine that doesn't collect data about its users, lost the auction in all but four small European countries, the people said. Berlin-based Ecosia, which donates most of its profit to planting trees, also didn't win a slot in any large European country, the people added. The major winners of the auctions -- which offer three spots in each of 31 countries to outside search engines -- include Microsoft's Bing, as well as a handful of other small search engines, the people said. Google doesn't participate in the auctions but is offered automatically as a choice in every country along with the auction winners. The elimination of some smaller search engines gives fodder to Google rivals who have complained that the company has crafted its compliance with the EU's antitrust decisions in ways that don't fundamentally change the competitive landscape.

Google said it will block some autocomplete search suggestions to stop misinformation spreading online during the U.S. presidential election in November. From a report The autocomplete feature of the world's largest search engine regularly recommends full queries once users begin typing words. The company said on Thursday it will remove predictions that could be interpreted as claims for or against any candidate or political party. In addition, Google said it will pull claims from the autocomplete feature about participation in the election, including statements about voting methods, requirements, the status of voting locations and election security. For instance, if you type in "you can vote" into Google's search engine, the system may have suggested a full query that includes misleading or incorrect information. Typing those three words into Google on Thursday produced the full phrase "You can vote yourself into socialism" as the top recommended query.

Early this year, Google pushed out a seemingly tiny tweak to how it displays search ads for desktop computers. From a report: Previously, the search engine had marked paid results with the word "Ad" in a green box, tucked beneath the headline next to a matching green display URL. Now, all of a sudden, the "Ad" and the URL shifted above the headline, and both were rendered in discreet black; the box disappeared. The organic search results underwent a similar makeover, only with a new favicon next to the URL instead of the word "Ad." The result was a general smoothing: Ads looked like not-ads. Not-ads looked like ads. This was not Google's first time fiddling with the search results interface. In fact, it had done so quite regularly over the last 13 years, as handily laid out in a timeline from the news site Search Engine Land. Each iteration whittled away the distinction between paid and unpaid content that much more. Most changes went relatively unnoticed, internet residents accepting the creep like the apocryphal frog in a slowly boiling pot.

But in January, amid rising antitrust drumbeats and general exhaustion with Big Tech, people noticed. Interface designers, marketers, and Google users alike decried the change, saying it made paid results practically indistinguishable from those that Google's search algorithm served up organically. The phrase that came up most often: "dark pattern," a blanket term coined by UX specialist Harry Brignull to describe manipulative design elements that benefit companies over their users. That a small design tweak could inspire so much backlash speaks to the profound influence Google and other ubiquitous platforms have -- and the responsibility that status confers to them. "Google and Facebook shape realities," says Kat Zhou, a product designer who has created a framework and toolkit to help promote ethical design. "Students and professors turn to Google for their research. Folks turn to Facebook for political news. Communities turn to Google for Covid-19 updates. In some sense, Google and Facebook have become arbiters of the truth. That's particularly scary when you factor in their business models, which often incentivize blurring the line between news and advertisements."

Google's not the only search engine to blur this line. If anything, Bing is even more opaque, sneaking the "Ad" disclosure under the header, with only a faint outline to draw attention. [...] But Google has around 92 percent of global search marketshare. It effectively is online search. Dark patterns are all too common online in general, and January wasn't the first time people accused Google of deploying them. In June of 2018, a blistering report from the Norwegian Consumer Council found that Google and Facebook both used specific interface choices to strip away user privacy at almost every turn. The study details how both platforms implemented the least privacy-friendly options by default, consistently "nudged" users toward giving away more of their data, and more. It paints a portrait of a system designed to befuddle users into complacency. [...] That confusion reached its apex a few months later, when an Associated Press investigation found that disabling Location History on your smartphone did not, in fact, stop Google from collecting your location in all instances.