Fix Python – Getting realtime output using subprocess


Asked By – Chris Lieb

I am trying to write a wrapper script for a command line program (svnadmin verify) that will display a nice progress indicator for the operation. This requires me to be able to see each line of output from the wrapped program as soon as it is output.

I figured that I’d just execute the program using subprocess.Popen, use stdout=PIPE, then read each line as it came in and act on it accordingly. However, when I ran the following code, the output appeared to be buffered somewhere, causing it to appear in two chunks, lines 1 through 332, then 333 through 439 (the last line of output)

from subprocess import Popen, PIPE, STDOUT

p = Popen('svnadmin verify /var/svn/repos/config', stdout = PIPE, 
        stderr = STDOUT, shell = True)
for line in p.stdout:
    print line.replace('\n', '')

After looking at the documentation on subprocess a little, I discovered the bufsize parameter to Popen, so I tried setting bufsize to 1 (buffer each line) and 0 (no buffer), but neither value seemed to change the way the lines were being delivered.

At this point I was starting to grasp for straws, so I wrote the following output loop:

while True:
        print'\n', '')
    except StopIteration:

but got the same result.

Is it possible to get ‘realtime’ program output of a program executed using subprocess? Is there some other option in Python that is forward-compatible (not exec*)?

Now we will see solution for issue: Getting realtime output using subprocess


I tried this, and for some reason while the code

for line in p.stdout:

buffers aggressively, the variant

while True:
  line = p.stdout.readline()
  if not line: break

does not. Apparently this is a known bug: (The issue is now “Closed” as of Aug 29, 2018)

This question is answered By – Dave

This answer is collected from stackoverflow and reviewed by FixPython community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0