Asked By – ventolin
I’m trying to use
Pool.map() function to divide out work simultaneously. When I use the following code, it works fine:
import multiprocessing def f(x): return x*x def go(): pool = multiprocessing.Pool(processes=4) print pool.map(f, range(10)) if __name__== '__main__' : go()
However, when I use it in a more object-oriented approach, it doesn’t work. The error message it gives is:
PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed
This occurs when the following is my main program:
import someClass if __name__== '__main__' : sc = someClass.someClass() sc.go()
and the following is my
import multiprocessing class someClass(object): def __init__(self): pass def f(self, x): return x*x def go(self): pool = multiprocessing.Pool(processes=4) print pool.map(self.f, range(10))
Anyone know what the problem could be, or an easy way around it?
Now we will see solution for issue: Can’t pickle
The problem is that multiprocessing must pickle things to sling them among processes, and bound methods are not picklable. The workaround (whether you consider it “easy” or not;-) is to add the infrastructure to your program to allow such methods to be pickled, registering it with the copy_reg standard library method.
For example, Steven Bethard’s contribution to this thread (towards the end of the thread) shows one perfectly workable approach to allow method pickling/unpickling via