My pet crawler

My pet project over the years has been building a web crawler. I just recently built a page that outputs some of the info being collected when it crawls a page instead of just storing the info. I get a kick out of using it on my websites and I thought some other people on webforumz might like to check it out.

http://www.mysitetrack.com/crawler/index.php

Just enter a url and click Generate Report.

Hey! Thats nifty...even comes with a to do list....

That's pretty cool. I love stuff like that.

Nice one.

Pete.

Thanks guys, I figured some people would enjoy it.
I just started to combine it with my content analysis script to display more info about a page

Wow, I like that kinda handy. I think I have some sites I am going to check out. Thanks Man!!

Neat. You deserve a pat on the old back for this. Yaaaaah!

Thank you... Thank you very much... (said in an Elvis voice)

thats sweet ..... its been added to my bookmarks ... its mine now muahha ha ha ha

ahem .... very nice well done

*Nicks a rabbit of accurax*

Very nice... I found something quite interesting.

At the end of the report, you analyse keywords within the content.
one of my sites (pinesandneedles.com) had this:

Keywords appearing once:
one word: christmas, trees, london (and many more unrelated)
two word: christmas tree, christmas trees (and many more unrelated)

Keywords appearing twice:
one word: christmas, trees, london (and a few more related)
two word: christmas tree, christmas trees (and a few more related)

Keywords appearing 3 times:
one word: christmas, trees, london (and only)
two word: christmas tree, christmas trees (and only)

Keywords appearing 4 times:
one word: christmas, trees, london (and only)
two word: christmas tree, christmas trees (and only)

Which is why we do so well on searches for "christmas trees london" and respectably well on searches for "christmas trees", considering the competition and the fact we're the only site on the top 10 results that doesn't have the words "christmas" or "tree" or even "london" in the url....

All this means is that your analyser is obviously imitating the behaviour of search engines very well and you should indeed be very proud of it.

Nice work ;-)

Thats all i do on rabbitdom mate.... i have no idea why... but its amusing to steal ur carrots

Cool! I'll be following your project's progress. I'll be bookmarking your site.

ArdRigh

nice i like it *adds to fav's*