This class can be used to crawl a site and retrieve the the URL of all links.
It can retrieve a page of a site and follow all links recursively to retrieve all the site URLs.
The class can restrict the crawling to URLs with a given extension and avoids accessing pages listed in the site robots.txt file, or pages set with the no index or no follow meta tags.
|
|
| Name: |
Spider website |
| Base name: |
spider |
| Description: |
Crawl a site and retrieve the the URL of all links |
| Version: |
0.1 |
| PHP version: |
5.0 |
| License: |
GNU General Public License (GPL) |
| All time users: |
2049 users |
| All time rank: |
1714 |
| Week users: |
7 users |
| Week rank: |
348  |
| |
|
| Ratings | Utility |
Consistency |
Documentation |
Examples |
Tests |
Videos |
Overall |
Rank |
| All time: |
Sufficient (66.7%) |
Good (87.5%) |
- |
Sufficient (70.8%) |
- |
- |
Not sure (49.2%) |
1605 |
| Month: |
Not yet rated by the users |
| |
Applications that use this class |
|
|
No application links were specified for this class.
If you know an application of this package, send a message to the
author to add a link here.
| |
 Files |
|
|
Stumble It!
Bookmark in del.icio.us
Browse all classes by 
Support forum (2)
Blog
Latest changes
User ratings
Files
