Wednesday, October 08, 2008

Free Online Robots.Txt Generator

Free Online Robots.Txt Generator

For many amateur and startup webmasters configuring robots.txt is a little tedious task. Robots.txt is a simple plain vanilla text file in the root directory of a website to allow/deny web crawlers to particular locations of the website.

For a complete comprehensive discussion of robots.txt, check out Web Robots.txt here. I just came across a free tool which supports online generation of robots.txt

Webservice: Yellowpipe Robots Txt Generator

Features:

  1. Free
  2. Online hosted webservice
  3. Supports 'Deny All'
  4. Supports Configuration of Specific Folders
  5. Supports Configuration of Compiled List of Robots

No comments:

[Imported from Blogdrive]Online Virus Scanners

Online Virus Scanners Virus Scanners are no longer difficult to install, tedious to configure. There are easy to use Online Virus Scanne...