Exalead Snapshots your site and lists it in its search system. It also tries to hot link all your images in a page view window. Sites like mine using hotlink protection will display a image theft notice when they do this.
I thought I has stoped this snapshot bot without stoping its crawler but they have again changed the useragent for it.
See here and here for more.
block useragents.
NG/2.0,Image crawler
NG/4.0,image crawler
Robot does not comply with simple basic robots.txt commands to not load images.
User-agent: *
Disallow: /images/
skip to main |
skip to sidebar
(c) 2007 / (c) 2016 All Rights Reserved
Web robots are visiting sites to hack, spam, email harvest and to scrap your website contect for profit.
This blog is an atempt to keep track of them and to help webmasters by listing the abuse in google.
Blog Archive
-
▼
2006
(125)
-
▼
November
(29)
- ; Windows NT; ....../1.0 What is this
- christiandnsonline.com
- anothrrobot (http://www.anothr.com) RSS ABUSE
- outfoxbot/0.5
- keymachine.de abuse probes
- converacrawler/0.9d 7-9745.san2.attens.net bot
- www.exalead.com Violates robots.txt
- abuse from thenewpush.com
- 66.199.236.106 duns.dunnaonline.com spamer
- PHPNuke Atacker Bots
- Exalead image theft
- nodomaintransfer abuse nodomaintransfer27.com is back
- sumitbot_hansrajbot RufusBot Submit Bot spammer
- scspider/0.2 64.28.178.66
- blogbot/1.0 Locus.CS.UCLA.EDU 131.179.64.248
- webbot.org www.webbot.ru webbot/0.1
- 72.20.99.48 c08.ba.accelovation.com www.accelobot....
- Running getmyarticles.com remote scripts
- Website Contact form How the robots atack
- Google IP 72.14.194.33 Falls into bot traps
- 83.206.210.131 billythekid.ivelem.net
- render-dream.com
- 38.113.234.180 crawl1.cosmixcorp.com
- tm.net.my
- Jakarta Commons
- 400 Header 'Connection' contains invalid values
- 84.244.8.86 pejantantangguh-a.biz
- tmhaos04.imsbiz.com
- s7.buzzlogic.com
-
▼
November
(29)
Brogroll
Eric Romang Blog
Bad Behavior / Bad Behaviour
(c) 2007 / (c) 2016 All Rights Reserved
2 comments:
Hello,
Since then we had not had any bad feedback from webmasters for our preview. But I understand your point, we will add the ability for the webmasters to refuse their pages to be previewed and thumbnailed by the use of these two meta:
meta name="robots" content="nopreview"
or
meta name="robots" content="nothumbnail"
It should be online very soon.
Would that be ok for you ?
No thats not good enough you are violating robots.txt
We have restricted bots from loading images by placing
User-agent: *
Disallow: /images/
in the robots.txt and you are violating this rule. Please comply with robots.txt file. As far as hotlinking you need to stop that for all sites because its theft of bandwidth.
Post a Comment