aphylogeo.multiProcessor

class aphylogeo.multiProcessor.Multi(args, function)

Bases: object

Class that harnesses the power of “multiprocess” and wraps it, for ease of use, as a single callable line. It is mainly aimed at very many large tasks that could be run on supercomputers with an ridiculous amount of RAM.

executeOnce(arg)

The method that is ran as a single process

Parameters:

method (args list The list of arguments given to the)

Returns:

Nothing None But the return value of the executed method

is passed to self.resultList

processingLargeData()

Method for executing mutliprocess on tasks that demand a LARGE amount of individual memory. Will, first, run a single process then, will start as many child processes as the available RAM permits, starting new ones as the RAM is freed.

Variables:

p (Process) Representes a single child process

Returns:

The multiprocess-friendly list, that was updated by each child

Errors:

If other application reduce the avalable RAM mid-execution, Multiprocess outputs “Killed” and kills the child.

buttler(memBloc)

Ran as a child process, the buttler will constantly run other methods forever.

In this case, it:

updates de memory capacity and prints updates on the terminal.

It exists so not to bottleneck the main thread.

Uses timers to execute it’s methods because time.sleep() it processor hungry if constantly called

memUpdate()

Method that sets the baseline for memory calculation + output some information to the terminal All memory values are in bytes

This method is ran from the buttler() and updates every second

Variables:

memBuffer double %Amount of bytes to substract from the

available RAM for safety purposes

mem double Amount of bytes

terminalUpdate(memBlock)

Method that constantly updates the user about the currently run tasks

This method is ran from the buttler() and updates every 0.1 second

executeSmall(arg)

The method executed by processingSmallData

Returns:

Nothing, but the return value of the executed method is passed to a global multiprocessing-friendly list

processingSmallData()

Method for executing mutliprocess on tasks that demand little to no memory. Will immedialty start all the child processes. The packaging causes some marginal time lost; Only use for methods that take at least a second to run : below that, a for loop is likely much faster

Variables:

p Process Representes a single child process a None Exists only to permit the for loop

Returns:

The multiprocess-friendly list, that was updated by each child

Errors:

If other application reduce the avalable RAM mid-execution, Multiprocess outputs “Killed” and kills the child.