KB Robots.txt

Edit your robots.txt from within WordPress to control search engine activities.

Author:Adam R. Brown (profile at wordpress.org)
WordPress version required:2.0
WordPress version tested:2.5
Plugin version:1.0.1
Added to WordPress repository:18-04-2007
Last updated:14-08-2007
Warning! This plugin has not been updated in over 2 years. It may no longer be maintained or supported and may have compatibility issues when used with more recent versions of WordPress.
Rating, %:100
Rated by:1
Plugin URI:http://adambrown.info/b/widgets/kb-robots-txt...
Total downloads:104 626
Active installs:10 000+
plugin download
Click to start download

When robots (like the Googlebot) crawl your site, they begin by requesting http://example.com/robots.txt and checking it for special instructions. Use this plugin to create and edit your robots.txt file from within WordPress (using Options -> Robots.txt).

Whenever a user (or a robot, more likely) appends "robots.txt" to your blog URL (e.g. http://blog.example.com/robots.txt), this plugin will serve up the robots.txt file that you created in the WordPress admin menu.

This plugin should work with most versions of WordPress, but it is particularly intended for WP-MU installations, since it allows each WPMU blog to have a unique robots.txt file.

Limitations

Note that robots make only top-level requests for robots.txt files. If you have WordPress installed in a subdomain (e.g. http://blog.example.com/) or in your root (e.g. http://example.com/), this plugin will work as intended. But if you have WordPress installed in a subdirectory (e.g. http://example.com/blog/), then this plugin won't do much for you, since the search engines won't look for http://example.com/blog/robots.txt, only for http://example.com/robots.txt.

Also, requires that you be using some form of permalink structure. If links to blog posts look like http://example.com/index.php?p=36, this won't work.

Support

If you post your support questions as comments below, I probably won't see them. If the FAQs don't answer your questions, you can post support questions at the KB Robots.txt plugin page on my site.