Asked By – ilovecp3
For C++, we can use OpenMP to do parallel programming; however, OpenMP will not work for Python. What should I do if I want to parallel some parts of my python program?
The structure of the code may be considered as:
solve2 are two independent function. How to run this kind of code in parallel instead of in sequence in order to reduce the running time?
The code is:
def solve(Q, G, n): i = 0 tol = 10 ** -4 while i < 1000: inneropt, partition, x = setinner(Q, G, n) outeropt = setouter(Q, G, n) if (outeropt - inneropt) / (1 + abs(outeropt) + abs(inneropt)) < tol: break node1 = partition node2 = partition G = updateGraph(G, node1, node2) if i == 999: print "Maximum iteration reaches" print inneropt
setouter are two independent functions. That’s where I want to parallel…
Now we will see solution for issue: How to do parallel programming in Python?
You can use the multiprocessing module. For this case I might use a processing pool:
from multiprocessing import Pool pool = Pool() result1 = pool.apply_async(solve1, [A]) # evaluate "solve1(A)" asynchronously result2 = pool.apply_async(solve2, [B]) # evaluate "solve2(B)" asynchronously answer1 = result1.get(timeout=10) answer2 = result2.get(timeout=10)
This will spawn processes that can do generic work for you. Since we did not pass
processes, it will spawn one process for each CPU core on your machine. Each CPU core can execute one process simultaneously.
If you want to map a list to a single function you would do this:
args = [A, B] results = pool.map(solve1, args)
Don’t use threads because the GIL locks any operations on python objects.