Class: Spider website
| Recommend this page to a friend! |  Stumble It! |
 Bookmark in del.icio.us |
| Classes of Karol Janyst | > | Spider website | > | Download .tar.gz .zip | > |  Support forum (2) |
> |  Blog |
> |   Latest changes |
||
|
| |||||||||||||||||||||||||||||||||||||||||||
| Detailed description | ||
| This class can be used to crawl a site and retrieve the the URL of all links. It can retrieve a page of a site and follow all links recursively to retrieve all the site URLs. The class can restrict the crawling to URLs with a given extension and avoids accessing pages listed in the site robots.txt file, or pages set with the no index or no follow meta tags. |
| Groups | ||
 |
HTML | HTML generation and processing | View top rated classes |
|
PHP 5 | Classes using PHP 5 specific features | View top rated classes |
|
Searching | Search engines, crawling and indexing | View top rated classes |
 User ratings |
Applications |  Files |
||||
No application links were specified for this class.
 If you know an application of this package, send a message to the author to add a link here.
|
|||||||||||||||||||||||||||||||||||||||||
For more information send a message to :
info at phpclasses dot org.
info at phpclasses dot org.
| Copyright (c) Icontem 1999-2012 | PHP Classes | - PHP Class Scripts |
| PHP Book Reviews | - Reviews of books and other products |