View Single Post
  #6 (permalink)  
Old Sep 2nd, 2007, 12:33
welshstew's Avatar
welshstew welshstew is offline
Lead Administrator

SuperMember
Join Date: May 2007
Location: inside the outside
Posts: 1,325
Blog Entries: 12
Thanks: 1
Thanked 4 Times in 4 Posts
Re: robots and hiding url's

Yeah, the problem is that a robots.txt file is browsable to the end user. i.e. http://www.bbc.co.uk/robots.txt

Which means that any inquisitive user can just have a quick look at one file to find anything that I don't want them to see (yet !).

Will the robots meta data hide the folder path (www.mysite.co/folder1/) as well as the direct url (www.mysite.co/folder1/default.asp) if I only put one page into that folder?
__________________
WelshStew
Lead Administrator

tierney rides tboard - uk site | xtreme wales - extreme clothing
If you think I've helped, click the "Thanks"
webforumz - facebook | LinkedIn
Reply With Quote