Yahoo! Slurp Spider, Google Spider, Google AdSense Spider

[Michael]

Hi--I was looking at who was on line just a bit ago and noticed that in addition to me, and a couple members, there were a number of other non-members that included:

13 Yahoo! Slurp Spider's, 1 Google Spider, 1 Google AdSense Spider

Forgive my ignorance--are these really users, or some sort of search engine "bot" (I think that's the term I've heard used) scanning all of the posts?

Is this something to worry about? Curiously, Michael

 

[Not a Number]

Those are indeed search engine bots crawling through the postings. That's why when you do a Google search on some CP related phrase you often get hits that link to posts on this board.

This board is also using Google's AdSense as you can see from the banners on top. Ever notice how the banner ads often are related to the topics in the current post or thread? For example you could be reading a thread about orchids and the ads will be orchid related.

There's a search bot opt-out directive that can be added to the .htaccess file on Apache webservers but if the bot ignores this directive there's little you can do to block them. Since TerraForums is using AdSense blocking this is not really an option.

 

[Michael]

Thanks Not a Number--I had wondered about how the ad's always matched--here and elsewhere, just hadn't quite worked out the technology of it all. That and I recently read a Wired article about 'bot' attacks... So, now I'm seeing bad-bot's everywhere...bot's in the shadows...maybe even a few bot's in the belfry's?
Sorry--couldn't resist. At any rate, just a heightened awareness, that needs a little fine tuning so I thought I'd check...:grin:

Happy Trails, Mike