robots and hiding url's

Hi,

I was wondering if anyone had any knowledge about how to hide urls from search engines and the curious.

I have a number of pages in development which I don't want to display quite yet. The obvious way is to implement a robots.txt file. However, this is easily viewable and the 'curious' user would be able to go straight to the url. i.e. www.natwest.co.uk/robots.txt brings up the folders that they want to hide.

the other option is to use robots meta data to hide the page. however, does this also hide the folder url?
i.e. if implemented it will hide www.webaddress.com/folder/default.asp but will it also hide www.webaddress.com/folder/

So I have probably confused you all, but in summery, I want to hide both the page and its associated file path. (www.webaddress.com/folder/default.asp and www.webaddress.com/folder/) and want to know if the meta tag for robots will do this

Any suggestions would be appreciated.

Thanks,

Stew

__________________
WelshStew
Lead Administrator
tierney rides tboard - uk site | xtreme wales - extreme clothing
If you think I've helped, click the "Thanks"
webforumz - facebook | LinkedIn

this should help
http://www.robotstxt.org/wc/meta-user.html

just put this on the pages you don't want indexed
<meta name="robots" content="noindex">

Not all engines obey that Alex. I recommend using robots.txt for this. Wiki explains the basics of using it. http://en.wikipedia.org/wiki/Robots.txt

ah okay,
i'm no SEO specialist sorry.
that's rob

Yeah as HostingPlus says the best way is by using robots.txt

Yeah, the problem is that a robots.txt file is browsable to the end user. i.e. http://www.bbc.co.uk/robots.txt

Which means that any inquisitive user can just have a quick look at one file to find anything that I don't want them to see (yet !).

Will the robots meta data hide the folder path (www.mysite.co/folder1/) as well as the direct url (www.mysite.co/folder1/default.asp) if I only put one page into that folder?

__________________
WelshStew
Lead Administrator
tierney rides tboard - uk site | xtreme wales - extreme clothing
If you think I've helped, click the "Thanks"
webforumz - facebook | LinkedIn

what about CHMOD?

bless you, nasty cold you have.....

not sure what that is but a bit of googling should so me the way.

thanks for the pointer Dan

__________________
WelshStew
Lead Administrator
tierney rides tboard - uk site | xtreme wales - extreme clothing
If you think I've helped, click the "Thanks"
webforumz - facebook | LinkedIn

Thanks Dan, read through some stuff and it looks like I'll need to use cacls (change access control list) as I use a windows environment.

Looks like it might be a bit of overkill though, so will probably stick with the robots meta and hope it keeps most people out.

Thanks for all your help.

Stew

__________________
WelshStew
Lead Administrator
tierney rides tboard - uk site | xtreme wales - extreme clothing
If you think I've helped, click the "Thanks"
webforumz - facebook | LinkedIn

CHMOD - Its the permissions - like 777, 000 etc.

windows server doesn't use it
you just right click the file and change the permissions that way

Still CHMOD

Ok, it sounds like you might be trying to hide a sales page or download page or something. To ensure google is blocked, I'd suggest a password prompt that you'll give to your paying customers.

Log into your control panel and look for a authentication option.

Google won't be able to index it, you won't need robots.txt and your job is done.