| |
Google introduced a "fresh crawl" process to make their results as relevant and as fresh as possible. It runs each day. The purpose of the daily fresh crawl is to update Web pages in the index that change regularly. This allows Google to provide results that are up-to-date with current events.
google does one major update per month, which generally begins anywhere from around the 19th or 20th of the month to approximately the 28th of the month. The update process continues for several days, with search results appearing to fluctuate as the update continues. Once the update has been completed, the new data migrates to google's partner sites.
The regular monthly crawl takes place at around this time, beginning at different times for different web sites. The results of this crawl are generally reflected at the time of the following update.
For a number of months, beginning early Summer 2002, spidering of sites and changes have been observed to be going on all month, in between the regular monthly updates.
This has come to be known as everflux, and represents google's continuing desire and efforts to keep their search relevant, of high quality, and "minty fresh."
Google Response
If your site is new, or hasn't shown up in google for long, it may because our "fresh crawl" (which runs each day) was finding your site instead of our main crawl (which runs about once a month).
Our "fresh crawl" is a newer feature, and we're still experimenting with which pages to crawl, how deeply to crawl, etc.
We even reserve the right to (gasp!) not do a fresh crawl on some days because we're doing tests or reviewing new code.
Someone wrote in recently and said "my site got in google three weeks ago, and you've dropped me four times!" Nope, it's just that we don't always crawl the same pages in our fresh crawl, and we don't always crawl to the same depth.
As we do a full crawl of the web, we find most of the sites from our fresh crawl and put them in our regular index. My advice on our fresh crawl is to view it as a nice "bonus" on top of google's deep index. Users can always search our full index, but sometimes we can serve up even fresher pages as an extra nicety.
What does this mean for the average webmaster? In the word of the great Hitchhiker's Guide, "Don't Panic." Just do the normal things you should do:
1. Create a great site.
2. Submit your site to google on our "add url" form.
3. Get a link from the Open Directory Project or other directories (Yahoo, etc.).
4. Don't panic if your site takes a little while to show up in google. Be patient, and start to look around the web--there's lots of great advice about improving your site for users and search engines.
Hope this helps,
google 
|
