Wednesday 12 February 2014

Python multi-threading/multi-processing

Python multi-threading does not actually run on multiple CPUs due to Global Interpreter Lock, so it is useless. We can only use python multi-processing, which also has some bugs.

# 1. include package
from multiprocessing import Process, Manager

# 2. only variables created using manager can be shared across difference processes
manager=Manager()
arr=manager.list()

# 3. create process
p=Process(target=filterRuleTable, args=(arr, i, inputSentences, N, options,))

# 4. start running
p.start()

# 5. wait for termination
p.join()