Asked By – SciPy
To make my code more “pythonic” and faster, I use
multiprocessing and a map function to send it a) the function and b) the range of iterations.
The implanted solution (i.e., calling
tqdm directly on the range
tqdm.tqdm(range(0, 30))) does not work with multiprocessing (as formulated in the code below).
The progress bar is displayed from 0 to 100% (when python reads the code?) but it does not indicate the actual progress of the map function.
How can one display a progress bar that indicates at which step the ‘map’ function is ?
from multiprocessing import Pool import tqdm import time def _foo(my_number): square = my_number * my_number time.sleep(1) return square if __name__ == '__main__': p = Pool(2) r = p.map(_foo, tqdm.tqdm(range(0, 30))) p.close() p.join()
Any help or suggestions are welcome…
Now we will see solution for issue: Multiprocessing : use tqdm to display a progress bar
Solution found. Be careful! Due to multiprocessing, the estimation time (iteration per loop, total time, etc.) could be unstable, but the progress bar works perfectly.
Note: Context manager for
Pool is only available in Python 3.3+.
from multiprocessing import Pool import time from tqdm import * def _foo(my_number): square = my_number * my_number time.sleep(1) return square if __name__ == '__main__': with Pool(processes=2) as p: max_ = 30 with tqdm(total=max_) as pbar: for _ in p.imap_unordered(_foo, range(0, max_)): pbar.update()