It seems that Google uses more than just an algorithm to produce its search results, and employs people to review specific queries.
Google’s Director of Research Peter Norvig has been talking about the methods the search engine uses to ensure the quality and accuracy of its results.
According to Norvig:
“Another way we do it [improve the quality of search results] is to randomly select specific queries and hire people to say how good our results are. These are just contractors that we hire who give their judgment.”
“We train them on how to identify spam and other bad sites, and then we record their judgments and track against that. It’s more of a gold standard because it’s someone giving a real opinion, but of course, since there’s a human in the loop, we can’t afford to do as much of it.”
This is not previously unknown, but Google has been coy in talking about the issue, though the search engine’s Marissa Mayer revealed in July that it has 10,000 human evaluators that manually review the results.
Norvig also revealed some other ways that Google ensures the quality of its results:
The problem with this theory is that, it cannot hope to index anywhere near the amount of webpages that Google can