A lot of people lately have been complaining about Google dropping their sites from search results, or presenting weird versions of their sites. Was it a conspiracy? Was it a computer glitch? Now, a site called SEO chat claims that these problems are part of a larger mess at Google:
In January of last year (2006) Google went through “The Big Daddy” update. Unfortunately, since then things at Google have been … unstable, for lack of a better way of putting it. The reason for this is simple. Google servers ran out of space…
[Google's CEO said in April]: “Those machines are full. We have a huge machine crisis.”… [Google] started making changes to the spider. The spider would no longer even attempt to index every page of a site. Instead, it would index only “entry pages,” or those pages that could be gotten to from another source (links from other sites) or had a “high likelihood” of being clicked on if the page came up in a search (how that was determined I don’t know)…
The problem however is that I have reason to believe that those changes had some rather significant bugs.
I’d be curious to know if there’s any truth to this story, and what Google is actually doing to solve this alleged problem. (Thanks to Joe for the heads up!)
This may be referring to a major change at Google a few years back. This was reported at Google Watch, and supposedly was caused by an internal change over at Google when the number of pages indexed surpassed four billion, or 2^32, unsigned. A number of people noticed that Google had stopped updating its index, new pages did not appear, the search ranking was often out of date and so on. The problem was fixed after a few months, and Google has seemed pretty regular since.
I couldn’t understand some parts of this article Over-stuffed servers and sick spiders?, but I guess I just need to check some more resources regarding this, because it sounds interesting.