If you’re using Cloudflare for website security and performance, you’ll want to whitelist SEMRush IPs to ensure that their crawlers can continue to index your site correctly. In this article, we’ll show you how to do just that!
Table of Contents
How do I whitelist my SEMRushBot?
If you’re using Cloudflare to manage your website’s DNS, you may need to whitelist SEMrush’s IP addresses to ensure SEMrush crawlers can reach your site. To do this, simply add the following IP addresses to your Cloudflare account:
Log in to Cloudflare admin
To log in to Cloudflare, enter your email address and password. After you log in, you’ll be taken to the Cloudflare dashboard.
Open Firewall Settings
Navigate to Security >> WAF >> Tools in the left-hand navigation bar.
Create Firewall Rule
Click on the blue Create Firewall Rule button and give your rule a descriptive name
This firewall rule will be used to unblock specific IP addresses associated with SEMRushBot.
Enter SEMRushBot IP ranges
Under Field, select IP source address.
Operator should be set to Is In.
Value using the IP ranges below:
Then… heading set to Allow (changed from block).
Note: The bot is using standard 80 HTTP and 443 HTTPS ports to connect.
Deploy Firewall Rule
Once you’ve created your rule, click the Deploy Firewall Rule button to deploy it.
Once you’ve added these IP addresses to your Cloudflare account, you should be all set! SEMrush’s crawlers should now be able to reach your site without any issues. Thanks for taking the time to whitelist SEMrush!
Is Semrush a good bot?
Published on: 2022-09-14
Updated on: 2022-10-28