General / Html – Robots.txt

Posted in: Getting Started (4) Plugin Options Explained (35) Actions / Filters (30) How To (14)   

Robots.txt file tells search engines how to crawl your website which makes it an incredibly powerful SEO tool.

As default the robots.txt also includes an allow clause to admin URL and run-ajax.php url. Once customized those areas, the new slugs might not want to be show to anyone. Turn this option to Yes removed any reference to new wp-admin and run-ajax.php.

The robot file is available at the following url:

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedIn
Scroll to top