Oct 26, 2006

pdxg1n-o.webtrends.com What is it? Scrapper?

400 Required header 'Accept' missing
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322) pdxg1n-o.webtrends.com

Webtrends robot fakes a useragent and doesn't send the proper headers so it can not pass Bad Behaivor scanning.

From looking at the site it says nothing about a cloked robot. So what are they doing on my site?

For now based on all the text on the website about helping you get a top position it looks like they may be scraping other sites to get content to raise you in the listings this would make them a scrapper. If thats not whats going on someone explain it.

Why do they hide what they are doing?

No comments: