Multiprocessing in PHP

Resources

Example Code

Stripped-down demo class for multiprocessing in PHP (requires PHP compiled with --enable-pcntl and a UNIX/Linux operating system):

<?php

class Multiprocess {
	
	protected $processes = array();
	protected $work_queue = array();
	protected $callback;
	
	public function __construct($data, $callback) {
		$this->work_queue = $data;
		$this->callback = $callback;
	}
	
	public function run($concurrent = 5) {
		
		$this->completed = 0;
		foreach($this->work_queue as $data) {

			$pid = pcntl_fork(); // clone

			switch($pid) {
				case -1:
					throw new Exception("Out of memory!");
					
				case 0:
					// child process
					call_user_func($this->callback, $data);
					exit(0);
					
				default:
					// parent process
					$this->processes[$pid] = TRUE; // log the child process ID
			}
		
			// wait on a process to finish if we're at our concurrency limit
			while(count($this->processes) >= $concurrent) {
				$this->reap_child();
				usleep(500);
			}
		}
		
		// wait on remaining processes to finish
		while(count($this->processes) > 0) {
			$this->reap_child();
			usleep(500);
		}
	}
	
	protected function reap_child() {

		// check if any child process has terminated,
		// and if so remove it from memory
		$pid = pcntl_wait($status, WNOHANG);

		if($pid < 0) {
 			throw new Exception("Error: out of memory!");
 		} elseif($pid > 0) {
			unset($this->processes[$pid]);
		}
	}	
}

2 thoughts on “Multiprocessing in PHP”

  1. Hi Jon,

    I have been searching for a clean php class for handling multi processing in php for my little internal project. I came across your article. I really liked what I see. However I am not seasoned enough to figure out how to properly use your class. Do you mind to enlighten me a bit?

    Much appreciate for sharing!

    Bo

  2. Hello Jonathon,

    Thanks a lot for this post explaining how to run multiple processes simultaneously, saved me a lot of time after already spending 2 days to found out how it works.

    @Bo,
    Maybe very late but just to notice tu usage:
    function myFunctionToExecuteMultipleTimes($dataArray){
    // code to execute
    }

    $oMultiProc = new Multiprocess($dataArray, ‘myFunctionToExecuteMultipleTimes’);
    $oMultiProc->run(20); // run 20 processes on the same time

    Cheers,
    Salvatore

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Hello! I’m a self-taught freelance web developer, and this is my blog. Here, I document problems and solutions in PHP, MySQL, and Javascript that I encounter along the way. Enjoy!