Nov 22, 2006

Exalead image theft

Exalead Snapshots your site and lists it in its search system. It also tries to hot link all your images in a page view window. Sites like mine using hotlink protection will display a image theft notice when they do this.

I thought I has stoped this snapshot bot without stoping its crawler but they have again changed the useragent for it.

See here and here for more.

block useragents.

NG/2.0,Image crawler
NG/4.0,image crawler

Robot does not comply with simple basic robots.txt commands to not load images.

User-agent: *
Disallow: /images/

2 comments:

ExaleadGuy said...

Hello,

Since then we had not had any bad feedback from webmasters for our preview. But I understand your point, we will add the ability for the webmasters to refuse their pages to be previewed and thumbnailed by the use of these two meta:
meta name="robots" content="nopreview"
or
meta name="robots" content="nothumbnail"
It should be online very soon.
Would that be ok for you ?

tm said...

No thats not good enough you are violating robots.txt

We have restricted bots from loading images by placing

User-agent: *
Disallow: /images/

in the robots.txt and you are violating this rule. Please comply with robots.txt file. As far as hotlinking you need to stop that for all sites because its theft of bandwidth.