File: Robots.txt.class.php

Recommend this page to a friend!
  Classes of Andy Pieters  >  Robots_txt  >  Robots.txt.class.php  
File: Robots.txt.class.php
Role: Class source
Content type: text/plain
Description: Core file
Class: Robots_txt
Test if a URL may be crawled looking at robots.txt
Author: By
Last change: Bug fix for regexp problem
Date: 13 years ago
Size: 12,447 bytes


You need to be a registered user and login to get this file.

Login Immediately with your account on:


For more information send a message to info at phpclasses dot org.