Asked By – alexis
The documentation for the
multiprocessing module shows how to pass a queue to a process started with
multiprocessing.Process. But how can I share a queue with asynchronous worker processes started with
apply_async? I don’t need dynamic joining or anything else, just a way for the workers to (repeatedly) report their results back to base.
import multiprocessing def worker(name, que): que.put("%d is done" % name) if __name__ == '__main__': pool = multiprocessing.Pool(processes=3) q = multiprocessing.Queue() workers = pool.apply_async(worker, (33, q))
This fails with:
RuntimeError: Queue objects should only be shared between processes through inheritance.
I understand what this means, and I understand the advice to inherit rather than require pickling/unpickling (and all the special Windows restrictions). But how do I pass the queue in a way that works? I can’t find an example, and I’ve tried several alternatives that failed in various ways. Help please?
Now we will see solution for issue: Sharing a result queue among several processes
Try using multiprocessing.Manager to manage your queue and to also make it accessible to different workers.
import multiprocessing def worker(name, que): que.put("%d is done" % name) if __name__ == '__main__': pool = multiprocessing.Pool(processes=3) m = multiprocessing.Manager() q = m.Queue() workers = pool.apply_async(worker, (33, q))
This question is answered By – enderskill
This answer is collected from stackoverflow and reviewed by FixPython community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0