Google's Manual For Its Unseen Human Raters 67
concealment writes "It's widely believed that Google search results are produced entirely by computer algorithms — in large part because Google would like this to be widely believed. But in fact a little-known group of home-worker humans plays a large part in the Google process. The way these raters go about their work has always been a mystery. Now, The Register has seen a copy of the guidelines Google issues to them."
This is just standard expert-tagging, right? (Score:5, Informative)
This is only believed by people who haven't thought about it very hard.
At an abstract level, it makes no sense to think that computer code can be optimized to perform a task without any human intervention. The reason is simple: the task we want the code to perform is always something that a human cares about. So, somehow we need a human to instruct the computer about the goals. This can take the form of a programmer meticulously coding the entire thing, with a particular human-relevant code in mind. Or it can involve non-programmers providing feedback about how well the software is doing at its stated goal (depending on context, these people may be testers, evaluators, users, taggers, etc.).
More specifically, in the case of AI-software, a typical procedure is to have a store of 'pre-tagged' training examples. These are example of problem, with associated 'correct' answers. The training data is used to optimize the AI algorithm: the software can tweak its behavior in order to maximize accuracy of output on the training examples, with the hope that this will then generalize to general use. For something like web-search, where the goal is to make a human end-user happy with the quality and relevance of the results, of course you need humans to assess the quality of the algorithmic results. This is the only way to keep the results relevant. (For search results, this is a continual and iterative process, since the web constantly changes, people are trying to game the system, etc.)
Thus, it's probably better to think of these raters as providing input for evaluating and refining the search algorithms; rather than thinking of them as people who get to uniquely decide the rank of pages. Obviously they will have an influence on the rank of the pages they rate, but overall they are evaluating a rather tiny fraction of the web-pages in the Google database. Thus, when you perform an arbitrary web-search, chances are the results you are seeing are purely algorithmic (none of the listed results were manually rated/adjusted by anyone).
Re:Work from home? (Score:4, Informative)
Leapforce isn't capped in the same way, but it has a lower rate of pay. Individual raters see limited hours at first, but as long as you perform well there is usually way more than 40+ hours.
This isn't news, as old versions of the General Guidelines have been leaked to the public before.
Re:Work from home? (Score:5, Informative)
I've done this work (for LeapForce; there is another company LionBridge that does the same work, but I have no experience with them). It's really, really mind-numbingly boring. I didn't last very long due to that. And it's worse than say a boring service job where you can interact with other people because you interact with nobody (I realize this may appeal to some). The other thing is that there is not always work available when you want to work. So, you may sign in and be ready to go, but there's no work there. This may cycle somewhat throughout the year with their hiring cycles. You may end up visiting sketchy websites, so having everything up to date is a must. They allow you to opt-out of adult ratings, but that doesn't mean that you will never come across an inappropriate site. You also have minimum performance requirements, and I think it would be difficult to maintain those requirements if you have lots of distractions going on while you work (kids, etc.). Some people are not able to attain the minimum at all within their required time frame (I had a couple of friends try and fail).
They do pay as agreed, but don't expect your check in a hurry. When I did work, you were able to submit an invoice on the 1st of the month for the previous month. They then paid net-30 on that invoice. That means, if you start December 1, then on January 1 you can submit your invoice for December. At the end of January you get paid for that invoice (for the work you did in December). So, if you need money quickly it doesn't work out so well, but once you get started it is a monthly income. They may have changed policies, so be sure to check it out before starting work.
That being said, the pay is good for a job you can do from home (I think around $13.50/hour). So, you certainly may be able to make $5000 a month, but that would be some insane hours (> 90 a week). If you need some extra income, try it out. There's a couple of tests to take before you are hired, and those are a good way to see what you think of the work. If you hate taking the tests, you will hate the work.