https://briansnelson.com/index.php?action=history&feed=atom&title=HAProxy_Block_by_User-Agent 2026-06-04T02:49:10Z Revision history for this page on the wiki MediaWiki 1.24.1 https://briansnelson.com/index.php?title=HAProxy_Block_by_User-Agent&diff=1030&oldid=prev 2019-12-12T08:33:13Z

<p></p> <table class='diff diff-contentalign-left'> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr style='vertical-align: top;'> <td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td> <td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 08:33, 12 December 2019</td> </tr><tr><td colspan="2" class="diff-lineno">Line 32:</td> <td colspan="2" class="diff-lineno">Line 32:</td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>You are looking for</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>You are looking for</div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>Configuration file is valid</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins class="diffchange diffchange-inline">'''</ins>Configuration file is valid<ins class="diffchange diffchange-inline">'''</ins></div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>Next restart or start the server to start blocking by user-agent</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>Next restart or start the server to start blocking by user-agent</div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>&#160; service haproxy restart/start</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>&#160; service haproxy restart/start</div></td></tr> <!-- diff cache key briansne_wiki-wiki_:diff:version:1.11a:oldid:1029:newid:1030 --> </table>

Brian https://briansnelson.com/index.php?title=HAProxy_Block_by_User-Agent&diff=1029&oldid=prev 2019-12-12T08:32:56Z

<p>Created page with &quot;==HAProxy block by User-Agent== As with any proxy service, you will want to block bad bots. Blocking by the a keyword in the user-agent before it hits the backend servers wi...&quot;</p> <p><b>New page</b></p><div>==HAProxy block by User-Agent==<br /> <br /> As with any proxy service, you will want to block bad bots. Blocking by the a keyword in the user-agent before it hits the backend servers will save on server resources.<br /> <br /> Add the following to your haproxy configuration file<br /> <br /> vim /etc/haproxy/haproxy.cfg<br /> <br /> Add the following to your frontend<br /> <br /> # Block badbots<br /> acl is-blockedagent hdr_sub(user-agent) -i -f /etc/haproxy/blacklist-agent.txt<br /> http-request deny if is-blockedagent<br /> <br /> Save then add the blacklist-agent.txt file.<br /> <br /> vim /etc/haproxy/blacklist-agent.txt<br /> <br /> Block the basics<br /> <br /> SemrushBot<br /> AhrefsBot<br /> MJ12bot<br /> ZoominfoBot<br /> DotBot<br /> MauiBot<br /> <br /> Save and check your configuration<br /> <br /> service haproxy check<br /> <br /> You are looking for<br /> <br /> Configuration file is valid<br /> <br /> Next restart or start the server to start blocking by user-agent<br /> <br /> service haproxy restart/start</div>

Brian