PHP Classes
elePHPant
Icontem

PHP Google Search Crawler: Perform searches and retrieve results from Google

Recommend this page to a friend!
  Info   View files Documentation   View files View files (28)   DownloadInstall with Composer Download .zip   Reputation   Support forum   Blog    
Last Updated Ratings Unique User Downloads Download Rankings
2017-08-03 (4 months ago) RSS 2.0 feedNot enough user ratingsTotal: 251 All time: 7,561 This week: 339Up
Version License PHP version Categories
google-crawler 1.1.4GNU General Publi...7Searching, Web services, PHP 7
Description Author

This class can perform searches and retrieve results from Google.

It can send requests to Google search pages Web server to perform searches for given keywords.

The requests may be sent using proxy servers.

The package parses the result pages and extracts the results as a result object that provides an array interface so it can be traversed using regular array functions.

Name: Vinícius Dias <contact>
Classes: 2 packages by
Country: Brazil Brazil
Age: ???
All time rank: 3679300 in Brazil Brazil
Week rank: 916 Up67 in Brazil Brazil Up

Details

Google Crawler

Latest Stable Version Build Status Code Coverage Scrutinizer Code Quality License

A simple Crawler for getting Google results.

This component can be used to retrieve the 100 first results for a search term.

Since google detects a crawler and blocks the IP when several requests are made, this component is prepared to use some online proxy services, such as hide.me.

Installation

Install the latest version with

$ composer require cviniciussdias/google-crawler

Usage

Crawler class constructor prototype

CViniciusSDias\GoogleCrawler\Crawler::__construct(
    SearchTermInterface $searchTerm, GoogleProxyInterface $proxy = null,
    string $googleDomain = 'google.com', string $countryCode = ''
)

Parameters

  • $searchTerm Term that will be searched on Google
  • $proxy Online proxy service that will be used to access Google [optional]
  • $googleDomain Your country specific google domain, like google.de, google.com.br, etc. [optional]
  • $countryCode Country code that will be added to gl parameter on Google's url, indicating the location of the search. E.g. 'BR', 'US', 'DE' [optional]

Examples

Without proxy

<?php
use CViniciusSDias\GoogleCrawler\{
    Crawler, SearchTerm
};

$searchTerm = new SearchTerm('Test');
$crawler = new Crawler($searchTerm); // or new Crawler($searchTerm, new NoProxy());

$results = $crawler->getResults();

With some proxy

<?php
use CViniciusSDias\GoogleCrawler\{
    Crawler, SearchTerm, Proxy\CommonProxy
};

$searchTerm = new SearchTerm('Test');
$commonProxy = new CommonProxy('https://us.hideproxy.me/includes/process.php?action=update');
$crawler = new Crawler($searchTerm, $commonProxy);

$results = $crawler->getResults();

More details on proxies

To know more details about which proxies are currently supported, see the tests/Functional/CrawlerTest.php file. There you'll see all the available proxies.

About

Requirements

  • This component works with PHP 7.1 or above
  • This component requires the extension php-ds to be installed

Author

Vinicius Dias (ZCE) - carlosv775@gmail.com - https://github.com/CViniciusSDias/ - http://www.zend.com/en/yellow-pages/ZEND030134

License

This component is licensed under the GPL v3.0 License - see the LICENSE file for details

  Files folder image Files  
File Role Description
Files folder imagesrc (5 files, 2 directories)
Files folder imagetests (2 directories)
Accessible without login Plain text file .travis.yml Data Auxiliary data
Accessible without login Plain text file build.xml Data Auxiliary data
Accessible without login Plain text file composer.json Data Auxiliary data
Accessible without login Plain text file composer.lock Data Auxiliary data
Accessible without login Plain text file LICENSE Lic. License text
Accessible without login Plain text file phpunit.xml Data Auxiliary data
Accessible without login Plain text file README.md Doc. Documentation

  Files folder image Files  /  src  
File Role Description
Files folder imageException (2 files)
Files folder imageProxy (4 files)
  Plain text file Crawler.php Class Class source
  Plain text file Result.php Class Class source
  Plain text file ResultList.php Class Class source
  Plain text file SearchTerm.php Class Class source
  Plain text file SearchTermInterface.php Class Class source

  Files folder image Files  /  src  /  Exception  
File Role Description
  Plain text file InvalidResultException.php Class Class source
  Plain text file InvalidUrlException.php Class Class source

  Files folder image Files  /  src  /  Proxy  
File Role Description
  Plain text file CommonProxy.php Class Class source
  Plain text file GoogleProxyInterface.php Class Class source
  Plain text file KProxy.php Class Class source
  Plain text file NoProxy.php Class Class source

  Files folder image Files  /  tests  
File Role Description
Files folder imageFunctional (3 files)
Files folder imageUnit (4 files, 1 directory)

  Files folder image Files  /  tests  /  Functional  
File Role Description
  Plain text file AbstractCrawlerTest.php Class Class source
  Plain text file DefaultCrawlerTest.php Class Class source
  Plain text file PersonalizedCrawlerTest.php Class Class source

  Files folder image Files  /  tests  /  Unit  
File Role Description
Files folder imageProxy (3 files)
  Plain text file CrawlerTest.php Class Class source
  Plain text file ResultListTest.php Class Class source
  Plain text file ResultTest.php Class Class source
  Plain text file SearchTermTest.php Class Class source

  Files folder image Files  /  tests  /  Unit  /  Proxy  
File Role Description
  Plain text file CommonProxyTest.php Class Class source
  Plain text file KProxyTest.php Class Class source
  Plain text file NoProxyTest.php Class Class source

 Version Control Unique User Downloads Download Rankings  
 100%
Total:251
This week:0
All time:7,561
This week:339Up