Top rated packages with tag spider

Recommend this page to a friend!

			All class groups	>	Search tags: spider

	1. User Info		spider

Supplied by: Fabrizio Parrella <e-mail contact>

Retrieve information of the user browser

This class can be used to retrieve information about the browser the user is using to access the current site script.

It can analyze the user agent string sent by the browser in the current HTTP request and extracts the type of browser, user machine operating system, user machine IP address, accepted character set encoding, language name, if the user is a spider/bot, security level, OS bits, and more to come!

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Good (91.7%)`	`Good (91.7%)`	`-`	`Perfect (100.0%)`	`-`	`-`	`Sufficient (65.0%)`	458
Month:	Not yet rated by the users

	2. Web Crawler using MySQL DB		spider

Supplied by: vivek <e-mail contact>

Retrieve Web pages and store links in a database

This class can be used to retrieve Web pages and store the URLs links in a MySQL database.

It retrieves a given Web page and parses its HTML content to extract the URLs of links and frames.

The URLs that are crawled are stored in a MySQL database table if the URL was not yet stored previously.

The class can also display in a Web page the list of URLs already stored from a given domain.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Good (83.3%)`	`Sufficient (75.0%)`	`-`	`Good (83.3%)`	`-`	`-`	`Not sure (57.5%)`	989
Month:	Not yet rated by the users

	3. Spider Engine		spider

Supplied by: Radu <e-mail contact>

Retrieve and process remote HTML pages

This class can be used to retrieve and process HTML pages from remote Web sites.

It can retrieve one or more pages from Web sites. The URL of the pages may follow a numeric pattern.

The class looks for patterns in the pages HTML. The patterns are configurable.

The data found in the pages is passed to a separate function for custom processing that can be implemented by a sub-class.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Sufficient (68.8%)`	`Sufficient (68.8%)`	`Not sure (50.0%)`	`Sufficient (62.5%)`	`-`	`-`	`Not sure (52.5%)`	1332
Month:	Not yet rated by the users

	4. Fetch Link Class		spider

Supplied by: Rahman Haghparast <e-mail contact>

This class extracts every link in a web page

This class uses text parsing and no regular expressions to find and extract every link in a webpage.

This class is suitable for developers who want to develop spider and crawler programs.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Sufficient (62.5%)`	`Sufficient (71.9%)`	`Sufficient (68.8%)`	`Sufficient (62.5%)`	`-`	`-`	`Not sure (51.6%)`	1386
Month:	Not yet rated by the users

	5. Spider Class		spider

Supplied by: greg jackson <e-mail contact>

Crawl a site following and retrieving linked pages

This class can retrieve Web pages and parse them to extract the list of their links to continue crawling all linked pages.

The pages may be retrieved iteratively until it is reached a given limit of pages or link depth.

It is possible to set regular expressions for both link definitions and content matches, changeable at every depth.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Sufficient (65.0%)`	`Sufficient (70.0%)`	`Not sure (55.0%)`	`Not sure (50.0%)`	`-`	`-`	`Not sure (50.0%)`	1477
Month:	Not yet rated by the users

	6. Spider website		spider

Supplied by: Karol Janyst <e-mail contact>

Crawl a site and retrieve the the URL of all links

This class can be used to crawl a site and retrieve the the URL of all links.

It can retrieve a page of a site and follow all links recursively to retrieve all the site URLs.

The class can restrict the crawling to URLs with a given extension and avoids accessing pages listed in the site robots.txt file, or pages set with the no index or no follow meta tags.

Ratings	Utility	Consistency	Documentation	Examples	Tests	Videos	Overall	Rank
All time:	`Sufficient (66.7%)`	`Good (87.5%)`	`-`	`Sufficient (70.8%)`	`-`	`-`	`Not sure (49.2%)`	1554
Month:	Not yet rated by the users

Copyright (c) Icontem 1999-2012	PHP Classes	- PHP Class Scripts
	PHP Book Reviews	- Reviews of books and other products