{"id":1200,"date":"2025-09-12T21:31:13","date_gmt":"2025-09-12T21:31:13","guid":{"rendered":"https:\/\/www.woodcentral.com\/-\/peter\/?p=1200"},"modified":"2026-05-24T11:28:10","modified_gmt":"2026-05-24T11:28:10","slug":"mitigating-bots-and-unwanted-traffic","status":"publish","type":"post","link":"https:\/\/www.woodcentral.com\/-\/peter\/mitigating-bots-and-unwanted-traffic\/","title":{"rendered":"Mitigating bots and unwanted traffic"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Use a Content Delivery Network (CDN). Cloudflare is the most popular and offers a free plan.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Security &gt; Security Rules &gt; Create Rule &gt; Custom Rule &gt; Edit Expression<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/www.woodcentral.com\/-\/peter\/wp-content\/uploads\/sites\/9\/2025\/09\/security-rule-1024x576.jpg\" alt=\"\" class=\"wp-image-1202\" srcset=\"https:\/\/www.woodcentral.com\/-\/peter\/wp-content\/uploads\/sites\/9\/2025\/09\/security-rule-1024x576.jpg 1024w, https:\/\/www.woodcentral.com\/-\/peter\/wp-content\/uploads\/sites\/9\/2025\/09\/security-rule-300x169.jpg 300w, https:\/\/www.woodcentral.com\/-\/peter\/wp-content\/uploads\/sites\/9\/2025\/09\/security-rule-768x432.jpg 768w, https:\/\/www.woodcentral.com\/-\/peter\/wp-content\/uploads\/sites\/9\/2025\/09\/security-rule-1536x864.jpg 1536w, https:\/\/www.woodcentral.com\/-\/peter\/wp-content\/uploads\/sites\/9\/2025\/09\/security-rule.jpg 1920w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Add the following expression modified for your needs. The first part defines who NOT to allow. The second &#8220;and not&#8221; part is who to allow. In this case, it is blocking an IP range and user agents containing terms often found in unwanted bots.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>(\n  ip.src in {34.174.0.0\/16}\n  or\n  (\n    http.request.uri.path ne \"\/healthcheck\"\n    and (\n      lower(http.user_agent) contains \"bot\"\n      or lower(http.user_agent) contains \"crawler\"\n      or lower(http.user_agent) contains \"spider\"\n      or lower(http.user_agent) contains \"slurp\"\n      or lower(http.user_agent) contains \"curl\"\n      or lower(http.user_agent) contains \"wget\"\n      or lower(http.user_agent) contains \"python\"\n      or lower(http.user_agent) contains \"java\"\n      or lower(http.user_agent) contains \"php\"\n      or lower(http.user_agent) contains \"perl\"\n      or lower(http.user_agent) contains \"scrapy\"\n      or lower(http.user_agent) contains \"libwww\"\n      or http.request.uri.path contains \"\/wire\/\"\n    )\n  )\n)\nand not (\n  lower(http.user_agent) contains \"w3c_validator\"\n  or lower(http.user_agent) contains \"googlebot\"\n  or lower(http.user_agent) contains \"bingbot\"\n  or lower(http.user_agent) contains \"oai-searchbot\"\n  or lower(http.user_agent) contains \"freshrss\"\n  or lower(http.user_agent) contains \"duckduckbot\"\n  or lower(http.user_agent) contains \"baiduspider\"\n  or lower(http.user_agent) contains \"yandexbot\"\n)\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The <em>&#8220;Then take action&#8221;<\/em> selection can be <strong>Managed Challenge<\/strong> or <strong>Block<\/strong>. The Managed Challenge will allow a blocked human to prove they are human.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The <em>&#8220;Place at&#8221;<\/em> selection should be set to <strong>First<\/strong>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Use a Content Delivery Network (CDN). Cloudflare is the most popular and offers a free plan.<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-1200","post","type-post","status-publish","format-standard","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/posts\/1200","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/comments?post=1200"}],"version-history":[{"count":0,"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/posts\/1200\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/media?parent=1200"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/categories?post=1200"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.woodcentral.com\/-\/peter\/wp-json\/wp\/v2\/tags?post=1200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}