It is now several weeks after the upgrade of drupal.org to Drupal 6 and I've taken a look at google's crawling statistics for drupal.org.
This is the most interesting graph for me as infrastructure manasger, it shows the average time that googlebot needs to download a html page from drupal.org. We apparently had a bit of a rough ride in January, but recently this has smoothed out. About 600ms per page seems quite a good value to me.
This second graph shows the bandwidth consumed by googlebot in kB per day. Googlebot consumes about 80GB per month.
The third graph shows the number of crawled pages per day, up to 500k. It is interesting to note that despite the number of pages increased considerably after the upgrade, the amount of data transferred did not. I don't think that Drupal 6 has a smaller html footprint than Drupal 5. I attribute this to the fact that we removed the long "all modules and themes" pages (due to unrelated reasons).
| Attachment | Size |
|---|---|
 chart.png | 10.51 KB |
| chart_002.png | 13.19 KB |
| chart_003.png | 12.87 KB |
