Top rated packages with tag crawler

Recommend this page to a friend!

			All class groups	>	Search tags: crawler

	1. Blue Crawler		crawler

Supplied by: kai <e-mail contact>

Crawl and retrieve links of Web pages

This class can be used to crawl and retrieve links of Web pages

It can retrieve a given Web pages and retrieve the URL of all linked pages recursively.

The class can also crawl all linked pages until it finds given search keywords.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Good (83.3%)`	`Good (83.3%)`	`Sufficient (75.0%)`	`Good (83.3%)`	`-`	`-`	`Sufficient (65.8%)`	386
Month:	Not yet rated by the users

	2. Simple Page Crawler		crawler

Supplied by: Jacek Lukasiewicz <e-mail contact>

Retrieve HTML pages and extract its elements

This package can retrieve HTML pages and extract its elements.

There is one class that can retrieve remote Web pages.

There are also other classes that can extract page elements like the words in the page body, links, count the words in a page, etc..

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Good (91.7%)`	`Good (91.7%)`	`-`	`Good (91.7%)`	`-`	`-`	`Sufficient (64.2%)`	504
Month:	Not yet rated by the users

	3. Web Crawler using MySQL DB		crawler

Supplied by: vivek <e-mail contact>

Retrieve Web pages and store links in a database

This class can be used to retrieve Web pages and store the URLs links in a MySQL database.

It retrieves a given Web page and parses its HTML content to extract the URLs of links and frames.

The URLs that are crawled are stored in a MySQL database table if the URL was not yet stored previously.

The class can also display in a Web page the list of URLs already stored from a given domain.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Good (83.3%)`	`Sufficient (75.0%)`	`-`	`Good (83.3%)`	`-`	`-`	`Not sure (57.5%)`	989
Month:	Not yet rated by the users

	4. Crawler		crawler

Supplied by: Md. Shaiful islam <e-mail contact>

Extract links and images from remote Web pages

This class can be used to extract links and images from remote Web pages.

It can access Web pages, parse the pages HTML and extract the URLs of the links and the images.

If necessary, the class may access a login page and emulate the submission of a login form to subsequent accesses can be done on behalf of the logged user.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Sufficient (75.0%)`	`Sufficient (69.4%)`	`-`	`Sufficient (77.8%)`	`-`	`-`	`Not sure (52.2%)`	1347
Month:	Not yet rated by the users

	5. Fetch Link Class		crawler

Supplied by: Rahman Haghparast <e-mail contact>

This class extracts every link in a web page

This class uses text parsing and no regular expressions to find and extract every link in a webpage.

This class is suitable for developers who want to develop spider and crawler programs.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Sufficient (62.5%)`	`Sufficient (71.9%)`	`Sufficient (68.8%)`	`Sufficient (62.5%)`	`-`	`-`	`Not sure (51.6%)`	1386
Month:	Not yet rated by the users

	6. Spider website		crawler

Supplied by: Karol Janyst <e-mail contact>

Crawl a site and retrieve the the URL of all links

This class can be used to crawl a site and retrieve the the URL of all links.

It can retrieve a page of a site and follow all links recursively to retrieve all the site URLs.

The class can restrict the crawling to URLs with a given extension and avoids accessing pages listed in the site robots.txt file, or pages set with the no index or no follow meta tags.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Sufficient (66.7%)`	`Good (87.5%)`	`-`	`Sufficient (70.8%)`	`-`	`-`	`Not sure (49.2%)`	1554
Month:	Not yet rated by the users

	7. path to vars		crawler

Supplied by: kumar mcmillan <e-mail contact>

turns a pseudo-path into variable values.

allows you to use a pseudo directory path in your HTML links --instead of a query string-- to declare variables so that your pages will be properly crawled by search engines. This is an alternative to using mod_rewrite rules to handle your URIs (http://httpd.apache.org/docs/mod/mod_rewrite.html). I based this on several articles and news posts I read on how search engines crawl dynamic, php-generated URLs.

see the class comments for more info and links to the articles.

at the moment, PHP >= 4.1.0 is required

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Good (80.0%)`	`Sufficient (65.0%)`	`-`	`Insufficient (25.0%)`	`-`	`-`	`Not sure (49.0%)`	1562
Month:	Not yet rated by the users

	8. Google Crawler		crawler

Supplied by: Bari Hossain <e-mail contact>

Extract results from Google search pages

This class can be used to extract results from Google search result pages.

It retrieves the Google search result pages for given search keywords.

The class parses the result pages to retrieve the list of search result URLs, titles and excerpts from the pages that were found.

The limit number of result pages to be retrieved is configurable.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Not sure (56.2%)`	`Sufficient (68.8%)`	`-`	`Sufficient (71.9%)`	`-`	`-`	`Not sure (42.2%)`	1890
Month:	Not yet rated by the users

Copyright (c) Icontem 1999-2012	PHP Classes	- PHP Class Scripts
	PHP Book Reviews	- Reviews of books and other products