PHP Classes
elePHPant
Icontem

Forker PHP: Split tasks into multiple forked processes

Recommend this page to a friend!
Stumble It! Stumble It! Bookmark in del.icio.us Bookmark in del.icio.us
  Info   View files View files (29)   DownloadInstall with Composer Download .zip   Reputation   Support forum   Blog    
Last Updated Ratings Unique User Downloads Download Rankings  
2014-11-24 (8 months ago) RSS 2.0 feedNot enough user ratingsTotal: 304 All time: 6,723 This week: 1,118Up
Version License PHP version Categories  
forker 1.0.4MIT/X Consortium ...5.3PHP 5, Unix, Language
Description Author  

This package can split tasks into multiple forked processes.

The main class takes an array of data to be processed as the whole task and creates child processes giving one element of the tasks array to be processed by each child process by calling given a given function that will process the respective task data.

It uses a separate storage object to store and retrieve task data to be shared between processes.

Currently it provides different storage classes that can store task data in memory as arrays, in files, or in a MemCached server.

Recommendations  

Threads PHP
php_fork

Innovation Award  
PHP Programming Innovation award nominee
May 2014
Number 4


Prize: PhpStorm IDE personal permanent license
Some CPU intensive tasks can be executed more efficiently if they are split into processes running in parallel. That way it takes advantage of multiple CPU cores that may be available.

This class can manage the split of processing of array data using forked processes to execute the same processing task in parallel to process each array element.

Manuel Lemos
Picture of Daniel Martinez
Name: Daniel Martinez <contact>
Classes: 3 packages by
Country: Spain Spain
Age: 29
All time rank: 156533 in Spain Spain
Week rank: 571 Up14 in Spain Spain Up
Innovation award
Innovation award
Nominee: 2x

Details provided by the author  
Forker
------------
[![Build Status](https://travis-ci.org/danibrutal/Forker.svg?branch=master)](https://travis-ci.org/danibrutal/Forker)
[![Latest Stable Version](https://poser.pugx.org/danibrutal/forker/v/stable.png)](https://packagist.org/packages/danibrutal/forker)

## Synopsis

A structured , safe, and easiest way to perform tasks parallely in PHP.

## Code Example

```php
<?php
/**************************************************
 * Example: Retrieving the city-weather using external api
 * Usage  : php examples/demo.api.weather.php 
 * Storage: File
 **************************************************/
require 'vendor/autoload.php';

use Forker\Forker;
use Forker\Storage\FileStorage;

$allCitiesWeather = "";

$urlApiWeather = "http://api.openweathermap.org/data/2.5/weather?q=%s&mode=xml";

$myTasks = array(
  'madrid'    => sprintf($urlApiWeather, 'Madrid'),
  'london'    => sprintf($urlApiWeather, 'London'),
  'new-york'  => sprintf($urlApiWeather, 'NewYork'),
  'barcelona' => sprintf($urlApiWeather, 'barcelona'),
  'lisboa'    => sprintf($urlApiWeather, 'lisboa'),
  'iasi'      => sprintf($urlApiWeather, 'iasi'),
);

// a way to keep our data
$storageSystem = new FileStorage;
$numberOfSubTasks = 6;

$forker = new Forker($storageSystem, $myTasks, $numberOfSubTasks);

$forker->fork(function($city, $url,  $emit) {
  echo "Retrieving weather in $city\n";
  
  $contents = file_get_contents($url);
  $emit($city, $contents);
});

$allCitiesWeather = $forker->fetch();

var_dump($allCitiesWeather);
```
## Motivation

Sometimes we have to work with a huge amount of data. 
[Lately](http://en.wikipedia.org/wiki/Big_data), more and more, and it's just no possible to work sequentially these times. 

For example, we have two cooks in a kitchen and a very big carrot.
Well, we want our two workers not to be waiting for each other. 
So why we don't split the carrot in two parts so they can work each one with a part ?

This way both cooks can work together, at the same time, and we will have our dinner soon ! Great huh ?

So, the intention is to make an agile and encapsulated way to split a task in several child subtasks in parallel.

## Install

Using composer:
```
  "require": {
        "danibrutal/forker": "dev-master"
    }
```


## API Reference

You can check the API out [here](http://testdouble.es/Forker/API/);
## Creating your own StorageSystem:
We follow here a TDD aproach so is extremely easy to develop a new system:

1º Create your own storage system following the StorageSystem interface's signature:
```php
/**
   * @param key
   * @param value
   * @return bool
   */
  public function store($key, $value);

  /**
   * @param key
   * @return value | false
   */
  public function get($key);

  /**
   * @return array $tasks
   */
  public function getStoredTasks();

  /**
   * @return bool
   */
  public function cleanUp();
```
2º Creates a test
```php
<?php
use Forker\Storage\ArrayStorage;
require_once 'BaseStorageTest.php';

class ArrayStorageTest extends BaseStorageTest
{
    protected function getSystemStorage()
    {        
        return new ArrayStorage();        
    }
}
```
Hard, huh?

3º Then, type phpunit so you can see 3 errors to solve:
```
There were 4 failures:

1) ArrayStorageTest::testWeCanGetASimpleStoredValue
Failed asserting that null matches expected 'value'.

2) ArrayStorageTest::testWeCanSToreValues
Failed asserting that null is true.

3) ArrayStorageTest::testIcanGetAllMyStoredTasks
Failed asserting that a NULL is not empty.

4) ArrayStorageTest::testWeCanCleanUpAllPreviousTasks
Failed asserting that a NULL is not empty.

```
4º Just solve the errors. Create your implementation and you are done!
Easy and funny :)

## Contributors
Please, feel free to colaborate. Fork the project and check [issues][2].
We still have so much work ahead!.

## License

MIT
  Files folder image Files  
File Role Description
Files folder imageexamples (4 files, 1 directory)
Files folder imagesrc (1 directory)
Files folder imagetests (1 file, 1 directory)
Accessible without login Plain text file .travis.yml Data Auxiliary data
Accessible without login Plain text file composer.json Data Auxiliary data
Accessible without login Plain text file LICENSE.md Data Auxiliary data
Accessible without login Plain text file phpunit.xml Data Auxiliary data
Accessible without login Plain text file README.md Data Auxiliary data

  Files folder image Files  /  examples  
File Role Description
Files folder imageMapReduce (4 files)
  Accessible without login Plain text file demo.api.weather.file-storage.php Example Example script
  Accessible without login Plain text file demo.api.weather.php Example Example script
  Accessible without login Plain text file demo.sum.php Example Example script
  Accessible without login Plain text file demo.timeout.php Example Example script

  Files folder image Files  /  examples  /  MapReduce  
File Role Description
  Accessible without login Plain text file demo.map-reduce.php Example Example script
  Accessible without login Plain text file quijote-1.txt Data Auxiliary data
  Accessible without login Plain text file quijote-2.txt Data Auxiliary data
  Accessible without login Plain text file quijote-3.txt Data Auxiliary data

  Files folder image Files  /  src  
File Role Description
Files folder imageForker (5 files, 2 directories)

  Files folder image Files  /  src  /  Forker  
File Role Description
Files folder imageException (2 files)
Files folder imageStorage (4 files)
  Plain text file Alarm.php Class Class source
  Plain text file ChilProcess.php Class Class source
  Plain text file Forker.php Class Class source
  Plain text file ISemaphore.php Class Class source
  Plain text file Semaphore.php Class Class source

  Files folder image Files  /  src  /  Forker  /  Exception  
File Role Description
  Plain text file ForkingErrorException.php Class Class source
  Plain text file StorageException.php Class Class source

  Files folder image Files  /  src  /  Forker  /  Storage  
File Role Description
  Plain text file ArrayStorage.php Class Class source
  Plain text file FileStorage.php Class Class source
  Plain text file MemcacheStorage.php Class Class source
  Plain text file StorageInterface.php Class Class source

  Files folder image Files  /  tests  
File Role Description
Files folder imageunit (1 file, 1 directory)
  Accessible without login Plain text file bootstrap.php Aux. Auxiliary script

  Files folder image Files  /  tests  /  unit  
File Role Description
Files folder imageStorage (3 files)
  Accessible without login Plain text file ForkerTest.php Test Unit test script

  Files folder image Files  /  tests  /  unit  /  Storage  
File Role Description
  Accessible without login Plain text file ArrayStorageTest.php Test Unit test script
  Accessible without login Plain text file BaseStorageTest.php Test Unit test script
  Accessible without login Plain text file FileStorageTest.php Test Unit test script

 Version Control Unique User Downloads Download Rankings  
 100%Total:304All time:6,723
 This week:0This week:1,118Up