Thread: Adivse on robots.txt
View Single Post
  #3 (permalink)  
Old Feb 14th, 2008, 16:16
Oak Oak is offline
SuperMember

SuperMember
Join Date: Dec 2007
Location: London
Age: 35
Posts: 265
Thanks: 2
Thanked 0 Times in 0 Posts
Re: Adivse on robots.txt
Yes I have been on that page but I am still not sure. What is best practice, for allowing and disallowing?

Will this do?

Quote:
# robots.txt for http://www.mysite.com

User-agent: Google
Disallow:

User-agent: *
Disallow: /
Thanks
Reply With Quote