PHP Classes
elePHPant
Icontem

Blue Crawler: Crawl and retrieve links of Web pages

Recommend this page to a friend!
Stumble It! Stumble It! Bookmark in del.icio.us Bookmark in del.icio.us
  Info   View files View files (4)   DownloadInstall with Composer Download .zip   Reputation   Support forum   Blog    
Last Updated Ratings Unique User Downloads Download Rankings  
2010-04-15 (5 years ago) RSS 2.0 feedStarStarStarStar 65%Total: 1,566 This week: 1All time: 2,375 This week: 1,052Up
Version License PHP version Categories  
bluecrawler 1GNU General Publi...4.0Searching, Web services
Description Author  

This class can be used to crawl and retrieve links of Web pages

It can retrieve a given Web pages and retrieve the URL of all linked pages recursively.

The class can also crawl all linked pages until it finds given search keywords.

Picture of kai
Name: kai <contact>
Classes: 2 packages by
Country: South Africa South Africa
Age: ???
All time rank: 114112 in South Africa South Africa
Week rank: 843 Up5 in South Africa South Africa Up

Details provided by the author  
Crawling a complete site:

construct the Crawler class with a string representing the URL you want to crawl

call getAllLinks(); on the object to return an array of found links



To crawl the site for a specific word or phrase construct the Crawler class with with a string representing the URL you want to crawl and the phrase you are looking for.

Reference the “thelink” variable owned by the Crawler object for the link the phrase were found on.
  Files folder image Files  
File Role Description
Plain text file Crawler.php Class main class
Accessible without login Plain text file Readme.txt Doc. readme
Accessible without login Plain text file test.php Example example
Accessible without login Plain text file WordSeperators.php Conf. support code

 Version Control Unique User Downloads Download Rankings  
 0%Total:1,566All time:2,375
 This week:1This week:1,052Up
 User Ratings  
 
 All time
Utility:83%StarStarStarStarStar
Consistency:83%StarStarStarStarStar
Documentation:75%StarStarStarStar
Examples:83%StarStarStarStarStar
Tests:-
Videos:-
Overall:65%StarStarStarStar
Rank:478