View Single Post
  #13 (permalink)  
Old Sep 2nd, 2007, 16:52
HostingDeals HostingDeals is offline
Junior Member
Join Date: Sep 2007
Location: Utah
Posts: 24
Thanks: 0
Thanked 0 Times in 0 Posts
Re: robots and hiding url's

Ok, it sounds like you might be trying to hide a sales page or download page or something. To ensure google is blocked, I'd suggest a password prompt that you'll give to your paying customers.

Log into your control panel and look for a authentication option.

Google won't be able to index it, you won't need robots.txt and your job is done.
Quote:
Originally Posted by welshstew View Post
Yeah, the problem is that a robots.txt file is browsable to the end user. i.e. http://www.bbc.co.uk/robots.txt

Which means that any inquisitive user can just have a quick look at one file to find anything that I don't want them to see (yet !).

Will the robots meta data hide the folder path (www.mysite.co/folder1/) as well as the direct url (www.mysite.co/folder1/default.asp) if I only put one page into that folder?
Reply With Quote