Search engines store web pages in an index. It is this search index that is ordered and filtered to deliver SERPs (search engine results) to everyday searchers.
If you perform an unusual or complex search, results may be scarce. If so, Google will ask you if you would like to re-try your search, including results from the supplementary index.
Why do search engines hide additional results that can assist in your search?
Because search engines don’t consider those results good enough, though they will still serve them up as a last resort.
The reason these results were removed from the main index in the first place is because the search engines either didn’t think they were high enough quality (e.g. SPAM) or because they were a carbon copy of another webpage, or in technical jargon: duplicate content.
Search engines want to deliver the best possible result to your search query with their first result, but they know it’s unlikely. The technology isn’t intelligent enough, yet, and the context of your search term is sometimes hard to gauge. That’s why search engines aim to give you ten different, yet high quality results to choose from.
There may be hundreds of results spanning tens of pages, but when search engines don’t think any are high enough quality or unique enough to include in the top ten, they place them in the supplementary index.



{ 1 comment… read it below or add one }
Hi Danny,
This was something I knew happened but never knew how it worked. Very interesting. You’ve taught me something new. Looking forward to learning more.
Bryce