Login   Register  
PHP Classes
elePHPant
Icontem

Spider website: Crawl a site and retrieve the the URL of all links

Recommend this page to a friend!
Stumble It! Stumble It! Bookmark in del.icio.us Bookmark in del.icio.us

  Author Author  
Picture of Karol Janyst
Name: Karol Janyst <contact>
Classes: 2 packages by
Country: Poland Poland
Age: 27
All time rank: 42113 in Poland Poland
Week rank: 230 Up8 in Poland Poland Up


  Detailed description   Download Download .zip .tar.gz   Install with Composer Install with Composer  
This class can be used to crawl a site and retrieve the the URL of all links.

It can retrieve a page of a site and follow all links recursively to retrieve all the site URLs.

The class can restrict the crawling to URLs with a given extension and avoids accessing pages listed in the site robots.txt file, or pages set with the no index or no follow meta tags.

  Classes of Karol Janyst  >  Spider website  >  Download Download .zip .tar.gz  >  Support forum Support forum (2)  >  Blog Blog  >  RSS 1.0 feed RSS 2.0 feed Latest changes  
Name: Spider website
Base name: spider
Description: Crawl a site and retrieve the the URL of all links
Version: 0.1
PHP version: 5.0
License: GNU General Public License (GPL)
All time users: 2467 users
All time rank: 1456
Week users: 4 users
Week rank: 414 Up
 
  Groups   Rate classes User ratings   Applications   Files Files  

  Groups  
Group folder image HTML HTML generation and processing View top rated classes
Group folder image PHP 5 Classes using PHP 5 specific features View top rated classes
Group folder image Searching Search engines, crawling and indexing View top rated classes


  User ratings  
RatingsUtility Consistency Documentation Examples Tests Videos Overall Rank
All time: Sufficient (66%) Good (87%) - Sufficient (70%) - - Not sure (49%) 1700
Month: Not yet rated by the users

  Applications that use this package  
No pages of applications that use this class were specified.
Add link image If you know an application of this package, send a message to the author to add a link here.
  Files folder image Files  
File Role Description
Plain text file spider.class.php Class Main class file
Accessible without login Plain text file example.php Example Example file

Install with Composer Install with Composer - Download Download all files: spider.tar.gz spider.zip
NOTICE: if you are using a download manager program like 'GetRight', please Login before trying to download this archive.