Google蜘蛛的访问

2008/4/14 12:22  

上次提到新网址提交Google Webmasters后,立即引来了Google蜘蛛。现在又有了新进展。

连续几天,Google蜘蛛每天到访,但只放问/robots.txt。因为/robots.txt并不存在,所以返回500错。

13日,我增加了/robots.txt文件。当天,Google蜘蛛在访问到/robots.txt后,又访问了网站的根目录:
/robots.txt
Http Code: 200 Date: Apr 13 21:10:33 Http Version: HTTP/1.1 Size in Bytes: 11
Referer: –
Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

/
Http Code: 200 Date: Apr 13 21:10:34 Http Version: HTTP/1.1 Size in Bytes: 0
Referer: –
Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

转载请注明来自:jijian91与小z - google

永久链接:http://jijian91.com/blog20080414/googlebot-log.html