War Blogging: Announcing the “Index of Evil”
WARBLOGGING.COM operates a Web spider called EvilBot which downloads a list of currently changed weblogs from Weblogs.com on an hourly basis. EvilBot then downloads every blog that’s changed since it last downloaded the list and looks for words like “Ashcroft”, “Hussein”, “Saddam” or “Osama”. If it finds one of these words the appropriate Index is raised by one.
I noticed that something called EvilBot had come a-calling: since well-mannered robot owners put contact information in the HTTP_USERAGENT field, I was able to follow up and see what this was all about.
As of this writing, the Index of Evil was at 828 with the individual indices as follows: Ashcroft: 185, Hussein: 25, bin Laden: 372, Mullah Omar: 20.
I suppose you could have spiders doing lots of stuff like this: looking for names or buzzwords, whathaveyou.