Google announced today that their search engine has hit a huge milestone, and it’s that they’ve come across an astounding 1 trillion unique URL’s on the web. Don’t confuse this with the number of pages that the search engine has indexed, because they flat out said that they are not indexing each of those trillion URL’s since many of them have duplicated/useless content:
So how many unique pages does the web really contain? We don’t know; we don’t have time to look at them all! :-) Strictly speaking, the number of pages out there is infinite — for example, web calendars may have a “next day” link, and we could follow that link forever, each time finding a “new” page. We’re not doing that, obviously, since there would be little benefit to you. But this example shows that the size of the web really depends on your definition of what’s a useful page, and there is no exact answer.
We don’t index every one of those trillion pages — many of them are similar to each other, or represent auto-generated content similar to the calendar example that isn’t very useful to searchers. But we’re proud to have the most comprehensive index of any search engine, and our goal always has been to index all the world’s data.
It’s crazy to think that there are over a trillion different URL’s that you can visit, and yet it was just a few years ago that Google was trying to tell us that we reached the end of the Internet? Uh, not quite. ;)
The problem that we all run into now is trying to find that one website that has exactly what we’re looking for. Search engines like Google try to make the information easier to find, but sometimes they don’t get the job done. To help discover the best of the trillion+ URL’s we have to rely on other services like StumbleUpon. Just think about how many different sites StumbleUpon has introduced to people. Heck, we’ve gotten over 2,000 visits alone from StumbleUpon today, and I’m sure a majority of those people are brand new visitors.
So grab yourself a few large pizzas, a couple cases of pop, and get back to browsing the web. You’ve got a few hundred billion more URL’s to discover. :D
P.S. I wonder what the lucky 1 trillionth URL was?