Site Audit without following robots.txt rules | Voters

Site Audit without following robots.txt rules

complete

Antonyms

Allow Ahrefs bot to crawl all the pages without following robots.txt rules. As a site owner I'd like to check seo errors before making it LIVE. (Allow this features to verified site owner (User))

April 5, 2018

Andrey Kirillov

marked this post as

complete

Andrey Kirillov

This feature has been added to Project settings! Please note the ownership verification required.

Andrey Kirillov

marked this post as

in progress

May 22, 2018

Gyorgy

+1 to this. I need to test websites in staging environment before publishing and Screaming Frog has this option too.