Fix Python – Python, Unicode, and the Windows console

Question

Asked By – James Sulak

When I try to print a Unicode string in a Windows console, I get an error .

UnicodeEncodeError: 'charmap' codec can't encode character ....

I assume this is because the Windows console does not accept Unicode-only characters. What’s the best way around this?
Is there any way I can make Python automatically print a ? instead of failing in this situation?

Edit: I’m using Python 2.5.


Note: @LasseV.Karlsen answer with the checkmark is sort of outdated (from 2008). Please use the solutions/answers/suggestions below with care!!

@JFSebastian answer is more relevant as of today (6 Jan 2016).

Now we will see solution for issue: Python, Unicode, and the Windows console


Answer

Note: This answer is sort of outdated (from 2008). Please use the solution below with care!!


Here is a page that details the problem and a solution (search the page for the text Wrapping sys.stdout into an instance):

PrintFails – Python Wiki

Here’s a code excerpt from that page:

$ python -c 'import sys, codecs, locale; print sys.stdout.encoding; \
    sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout); \
    line = u"\u0411\n"; print type(line), len(line); \
    sys.stdout.write(line); print line'
  UTF-8
  <type 'unicode'> 2
  Б
  Б

  $ python -c 'import sys, codecs, locale; print sys.stdout.encoding; \
    sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout); \
    line = u"\u0411\n"; print type(line), len(line); \
    sys.stdout.write(line); print line' | cat
  None
  <type 'unicode'> 2
  Б
  Б

There’s some more information on that page, well worth a read.

This question is answered By – Lasse V. Karlsen

This answer is collected from stackoverflow and reviewed by FixPython community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0