pesky Search Engine bots

eon201

Hi,

Im currently building my own stats counter (yes I know that very good ones exist already!).

But whilst going through my server log files i found that a HUGE amount of requests were coming from 'compatible;+MSIE+4.0;+Windows+NT;+....../1.0+'

It grabs literally everything. Even if you have dissalowed files in your robots.txt, and it will grab external files that you have no links to ANYWHERE within your site! Weird eh?

So has anyone else found this?? Have a look through your server logs and see!

Is this a bot?? If so I will overlook it in my stat counter!

Thanks in advance. eon201