Fix Python – Multiprocessing example giving AttributeError

Question 1

I am trying to implement multiprocessing in my code, and so, I thought that I would start my learning with some examples. I used the first example found in this documentation.

from multiprocessing import Pool
def f(x):
    return x*x

if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(f, [1, 2, 3]))

When I run the above code I get an AttributeError: can't get attribute 'f' on <module '__main__' (built-in)>. I do not know why I am getting this error. I am also using Python 3.5 if that helps.

Question 2

This problem seems to be a design feature of multiprocessing.Pool. See https://bugs.python.org/issue25053. For some reason Pool does not always work with objects not defined in an imported module. So you have to write your function into a different file and import the module.

File: defs.py

def f(x):
    return x*x

File: run.py

from multiprocessing import Pool
import defs

 if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(defs.f, [1, 2, 3]))

If you use print or a different built-in function, the example should work. If this is not a bug (according to the link), the given example is chosen badly.

Question

Answer