HAProxy Block by User-Agent

From Brian Nelson Ramblings
Revision as of 08:32, 12 December 2019 by Brian (Talk | contribs) (Created page with "==HAProxy block by User-Agent== As with any proxy service, you will want to block bad bots. Blocking by the a keyword in the user-agent before it hits the backend servers wi...")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

HAProxy block by User-Agent

As with any proxy service, you will want to block bad bots. Blocking by the a keyword in the user-agent before it hits the backend servers will save on server resources.

Add the following to your haproxy configuration file 

vim /etc/haproxy/haproxy.cfg

Add the following to your frontend 

  # Block badbots
 acl is-blockedagent hdr_sub(user-agent) -i -f /etc/haproxy/blacklist-agent.txt
 http-request deny if is-blockedagent

Save then add the blacklist-agent.txt file. 

vim /etc/haproxy/blacklist-agent.txt

Block the basics 

SemrushBot
AhrefsBot
MJ12bot
ZoominfoBot
DotBot
MauiBot

Save and check your configuration 

service haproxy check

You are looking for

Configuration file is valid

Next restart or start the server to start blocking by user-agent 

service haproxy restart/start
Retrieved from "https://briansnelson.com/index.php?title=HAProxy_Block_by_User-Agent&oldid=1029"