Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?
Businesses Google Technology

Google's Manual For Its Unseen Human Raters 67

concealment writes "It's widely believed that Google search results are produced entirely by computer algorithms — in large part because Google would like this to be widely believed. But in fact a little-known group of home-worker humans plays a large part in the Google process. The way these raters go about their work has always been a mystery. Now, The Register has seen a copy of the guidelines Google issues to them."
This discussion has been archived. No new comments can be posted.

Google's Manual For Its Unseen Human Raters

Comments Filter:
  • Re:Work from home? (Score:3, Insightful)

    by Anonymous Coward on Tuesday November 27, 2012 @11:00AM (#42105445)

    Nowhere near that much. I know someone who was a rater. Pay rate was ok for someone in Idaho who needs part time work (something like $15 an hour), but there are limits on the number of hours you can work (both over and under), and you're often limited by the number of tasks available.

  • Re:Could it be... (Score:5, Insightful)

    by mbkennel ( 97636 ) on Tuesday November 27, 2012 @11:14AM (#42105573)

    This is almost certainly what is happening. It is impossible for humans to rate any significant fraction of searches/websites to be quantitatively useful for Google's search volume.

    In machine learning, the name is "tags", a.k.a. ground truth for a supervised prediction/ranking model. Google gets zillions of weak, noisy, tag proxies in the sense of being able to measure when a user clicks on a link and then within a minute clicks on another link on the same search page, potentially indicating that the first link was undesirable.

    These are the relatively expensive but highest quality "ground-truth" tags from which Google can calibrate the value and interpretation of the weak automatic tags and the algorithms themselves.

    The final machine learning algorithms may be as simple as linear regression---performed on some rather complex features. These ground truth tags are used to calibrate and weight the importance of various features in making a final ranking.

If it's not in the computer, it doesn't exist.