72.5.173.21
As part of our ongoing efforts to improve the buying experience for shoppers online, Shopping.com is experimenting with new ways to collect and aggregate data through web crawling. At this point we do not plan on integrating inventory from our web crawling index with inventory
Say what! You pirate our content and bandwidth for your own internal test and don't even plan to index the data on your website. Now thats what you call abuse.
Why on earth should we allow you to do that.
Shopping.com is a pay site you have to pay to get listed unlike you do on google's froogle where its free. Stay off my site unless you want to pay me or actualy list teh info on your site!
To start with your using nutch a free package you downloaded. Nutch is banned from all my sites.
And you were not even scanning my store you were scanning a local blog how lame is that?
How often will sdcresearchlabs-testbot access my web pages?
For most sites, sdcresearchlabs-testbot shouldn't access your site more than once every few seconds on average.
What the ________? ONCE ever FEW SECONDS do you know how much bandwidth that is?
They say you should add "sdcresearchlabs-testbot" to your robots file. I am gona try that and see what it does.
No comments:
Post a Comment