Virtual Robots.txt

Installation Instructions

1. Upload the plugin files to /wp-content/plugins/virtual-robotstxt-littlebizzy
2. Activate the plugin through the ‘Plugins’ screen in WordPress
3. Check your WP Admin dashboard (admin users only) and footer area to view the statistics
4. For Apache servers, it should work aotomatically

5. For Nginx servers (without Apache in front) make sure try_files sends requests to your PHP processor:

   location = /robots.txt {
           try_files $uri $uri/ /index.php?$args;
           allow all;
           log_not_found off;
           access_log off;
       }
   

Does this plugin replace the need for my physical robots.txt file?

Yes. Currently its a totally virtual robots.txt solution that will “delete” any physical robots.txt that exists. The reason for this is to make WordPress as self-contained as possible, and since there is no real downside to managing robots.txt in such a virtual fashion. In future versions, we may create the option to switch between either a virtual or physical file, but for now its totally virtual as explained.

Does this plugin edit a physical robots.txt or only store a virtual robots.txt content?

This plugin will NOT edit a physical robots.txt file and only offers a virtual robots.txt content via the wp_options table.

How can I change this plugin’s settings?

Under the Settings menu on the left hand site, a new link is generated called Robots.txt

Are there any performance drawbacks to using this plugin?

Absolutely not. All wp_options are cached by default in WordPress and loaded prior to other data. In a theoretical world, a physical robots.txt file may in fact load more quickly, but all purposes both humans and robots wouldn’t notice a difference when using a virtual file. Please note that if you are using CloudFlare or other special cache systems, you may need to clear those to see your robots.txt update correctly. However, traditional caches such as page cache plugins or oboject caching (i.e. Memcached or Redis) should not effect the content of this plugin.

I’m using Nginx and this plugin is not working?

Make sure you are sending requests for the static file robots.txt via your PHP proessor such as PHP-FPM:

    location = /robots.txt {
            try_files $uri $uri/ /index.php?$args;
            allow all;
            log_not_found off;
            access_log off;
        }

I have a suggestion, how can I let you know?

Please avoid leaving negative reviews in order to get a feature implemented. Instead, we kindly ask that you post your feedback on the wordpress.org support forums by tagging this plugin in your post. If needed, you may also contact our homepage.

1.0.1

• Tweaked menu link to use Robots.txt instead of Virtual Robots.txt
• Added links to blog post and “Search Engine Visibility” on the settings page
• Changed containing folder (plugin slug) from virtual-robots-txt to virtual-robotstxt

1.0

• Initial (private) release based on private experimentation.