Set User agent
complete
Sam Underwood
Did this feature get removed? Not having this is one of the main reasons I struggle to consistently use the site auditor.
Matt Kohli
Experiencing a similar issue at the moment. Ownership is verified for the project but I still can't change the user agent from ahrefs. Could anyone lend a hand please?
Andrey Kirillov
complete
Matt Kohli
Andrey Kirillov: Please could you re-open the post and take a look at my issue? Thanks a lot
Andrey Kirillov
Good news! This feature has been added to Project settings. Please note the ownership verification required.
Andrey Kirillov
Merged in a post:
Ability to provide a custom user-agent for site audits
V
Volker
Hey,
we'd love to have the site auditor crawl a million of our pages a day (or more if you raise your limits) but our DDOS/scraping prevention setup limits the bots ability to make that many concurrent requests.
We can whitelist the IPs of course but it's annoying to keep the list up to date. Being able to provide hrefs with a custom user agent to use for the crawling would be amazing as we could use this 'shared secret' to whitelist all crawling requests from you.
Andrey Kirillov
in progress
Michaela Light
My concern is not distorting our analytics by running the site audit
Richard Balding
We can't even crawl our site unless we set it as Google-bot. The rest are GEOIP redirected. I am trying to get that fixed, but right now I can't use the tool until this is an option.
Asad Zulfahri
Asad Zulfahri
Duplicate of https://ahrefs.canny.io/site-audit/p/set-user-agent
Load More
→