How to stop bots (see earlier post for definition)?
Take care to pair up correctly Apache logs and apropriate fail2ban function! In my case apache-badbots is paired up with all access type Apache logs (this does include also SquirrelMail access log).
The definition in /etc/faiul2ban/jail.conf (badbots part) :
enabled = true
filter = apache-badbots
action = iptables-multiport[name=BadBots, port=”http,https”]
sendmail-buffered[name=BadBots, lines=5, dest=admin-postmaster@moj.domen]
logpath = /var/log/httpd/*access_log
bantime = 600
maxretry = 1

And the configuration file /etc/faiul2ban/filter.d/apache-badbots.conf :
badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider
badbots = atSpider/1\.0|autoemailspider|China Local Browse 2\.6|ContentSmartz|DataCha0s/2\.0|DataCha0s/
2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DS
urf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Frankli
n Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Industry Program 1\
.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsC
rawl\.com/1\.0 +http\://letscrawl\.com/|Lincoln State Web Browser|LWP\:\:Simple/5\.803|Mac Finder 1\.0\
.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control – 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missi
gua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|Mozilla/2\.0 \(compatible; NEW
T ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/4\.0 \(compatible; Advanced Email
Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0
\(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Versio
n\: xxxx Type\:xx\)|MVAClient|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Hur
on Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PS
urf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@googl
e\.com|sogou spider|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.
0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|WebVulnCrawl\.blogspot\.com/1\.0 libwww-perl/5\.803|Wells
Search II|WEP Search 00
failregex = ^ -.*”(GET|POST).*HTTP.*”(?:%(badbots)s|%(badbotscustom)s)”$
ignoreregex =

How to chect if apache-badbots is up/active :
# iptables -nvL
Chain fail2ban-BadBots (1 references)
pkts bytes target prot opt in out source destination

How to test apache-badbots :
# fail2ban-regex ‘ – – [12/Feb/2013:10:53:59 +0100] “GET / HTTP/1.1 200” 39460 “-” “autoemailspider”‘ /etc/fail2ban/filter.d/apache-badbots.conf
Running tests
Use regex file : /etc/fail2ban/filter.d/apache-badbots.conf
Use single line: – – [12/Feb/2013:10:53:59 +0100] “GET / HT…
`- Number of matches:
[1] 1 match(es)
Addresses found:
[1] (Tue Feb 12 10:53:59 2013)

This entry was posted in Linux and tagged , , . Bookmark the permalink.

Comments are closed.