Fix Python – Multiprocessing : use tqdm to display a progress bar

Question

Asked By – SciPy

To make my code more “pythonic” and faster, I use `multiprocessing` and a map function to send it a) the function and b) the range of iterations.

The implanted solution (i.e., calling `tqdm` directly on the range `tqdm.tqdm(range(0, 30))`) does not work with multiprocessing (as formulated in the code below).

The progress bar is displayed from 0 to 100% (when python reads the code?) but it does not indicate the actual progress of the map function.

How can one display a progress bar that indicates at which step the ‘map’ function is ?

``````from multiprocessing import Pool
import tqdm
import time

def _foo(my_number):
square = my_number * my_number
time.sleep(1)
return square

if __name__ == '__main__':
p = Pool(2)
r = p.map(_foo, tqdm.tqdm(range(0, 30)))
p.close()
p.join()
``````

Any help or suggestions are welcome…

Now we will see solution for issue: Multiprocessing : use tqdm to display a progress bar

Solution found. Be careful! Due to multiprocessing, the estimation time (iteration per loop, total time, etc.) could be unstable, but the progress bar works perfectly.

Note: Context manager for `Pool` is only available in Python 3.3+.

``````from multiprocessing import Pool
import time
from tqdm import *

def _foo(my_number):
square = my_number * my_number
time.sleep(1)
return square

if __name__ == '__main__':
with Pool(processes=2) as p:
max_ = 30
with tqdm(total=max_) as pbar:
for _ in p.imap_unordered(_foo, range(0, max_)):
pbar.update()
``````

This question is answered By – SciPy

This answer is collected from stackoverflow and reviewed by FixPython community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0