Fix Python – CSV in Python adding an extra carriage return, on Windows

Question

Asked By – apalopohapa

import csv

with open('test.csv', 'w') as outfile:
    writer = csv.writer(outfile, delimiter=',', quoting=csv.QUOTE_MINIMAL)
    writer.writerow(['hi', 'dude'])
    writer.writerow(['hi2', 'dude2'])

The above code generates a file, test.csv, with an extra \r at each row, like so:

hi,dude\r\r\nhi2,dude2\r\r\n

instead of the expected

hi,dude\r\nhi2,dude2\r\n

Why is this happening, or is this actually the desired behavior?

Now we will see solution for issue: CSV in Python adding an extra carriage return, on Windows


Answer

Python 3:

The official csv documentation recommends opening the file with newline='' on all platforms to disable universal newlines translation:

with open('output.csv', 'w', newline='', encoding='utf-8') as f:
    writer = csv.writer(f)
    ...

The CSV writer terminates each line with the lineterminator of the dialect, which is '\r\n' for the default excel dialect on all platforms because that’s what RFC 4180 recommends.


Python 2:

On Windows, always open your files in binary mode ("rb" or "wb"), before passing them to csv.reader or csv.writer.

Although the file is a text file, CSV is regarded a binary format by the libraries involved, with \r\n separating records. If that separator is written in text mode, the Python runtime replaces the \n with \r\n, hence the \r\r\n observed in the file.

See this previous answer.

This question is answered By – John Machin

This answer is collected from stackoverflow and reviewed by FixPython community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0