Spider website
| Recommend this page to a friend! |  Stumble It! |
 Bookmark in del.icio.us |
| Classes of Karol Janyst | > | Spider website | > | Download .tar.gz .zip | > |  Support forum (2) |
> |  Blog |
> |   Latest changes |
||
|
|||||||||||||||||||||||||
| Author | ||
 |
|
|||||||||||||||||||
Browse all classes by 
This class can be used to crawl a site and retrieve the the URL of all links.
It can retrieve a page of a site and follow all links recursively to retrieve all the site URLs.
The class can restrict the crawling to URLs with a given extension and avoids accessing pages listed in the site robots.txt file, or pages set with the no index or no follow meta tags.
It can retrieve a page of a site and follow all links recursively to retrieve all the site URLs.
The class can restrict the crawling to URLs with a given extension and avoids accessing pages listed in the site robots.txt file, or pages set with the no index or no follow meta tags.
| Groups |  User ratings |
Applications |  Files |
|||||
| Groups | ||
 |
HTML | HTML generation and processing | View top rated classes |
|
PHP 5 | Classes using PHP 5 specific features | View top rated classes |
|
Searching | Search engines, crawling and indexing | View top rated classes |
| User ratings | ||
| Ratings | ||||||||
|---|---|---|---|---|---|---|---|---|
| All time: | ||||||||
| Month: | ||||||||
| Applications that use this class | ||
No application links were specified for this class.
If you know an application of this package, send a message to the author to add a link here.| Download all files: spider.tar.gz spider.zip |
For more information send a message to info at phpclasses dot org.
Copyright (c) Icontem 1999-2012