`extract -o FILE` crashes on Windows with Unicode output, while stdout redirection works

[thisIsMikeKane]

Summary

On Windows, beangulp extract -o tmp.bean ... crashes with a UnicodeEncodeError if any extracted transaction contains non-ASCII text. The same extraction works if I do not use -o and instead redirect stdout:

beangulp extract -e "main.bean" "pending/" > "tmp.bean"

Environment

• OS: Windows
• Python: 3.12.3
• beangulp: 0.2.0
• beancount: 3.2.0
• click: 8.3.1

Reproduction

I have an input under pending/ that produces extracted text containing Japanese characters in the narration/payee, for example:

Payment from 管財人

This command crashes:

beangulp extract -e "main.bean" -o "tmp.bean" "pending/"

This command works:

beangulp extract -e "main.bean" "pending/" > "tmp.bean"

Actual behavior

With -o, extraction fails with:

UnicodeEncodeError: 'charmap' codec can't encode characters in position 32-43: character maps to <undefined>

Traceback ends here:

File "...\\site-packages\\beangulp\\__init__.py", line 108, in _extract extract.print_extracted_entries(extracted, output) File "...\\site-packages\\beangulp\\extract.py", line 232, in print_extracted_entries output.write(string) File "...\\encodings\\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode characters ...

Expected behavior

-o tmp.bean should behave the same as stdout redirection and write the extracted Beancount text successfully, including non-ASCII characters.

Suspected cause

It looks like the -o/--output option uses click.File('w'), which opens the file with the platform default text encoding on Windows (cp1252 in my case).

By contrast, shell redirection appears to produce an output stream that can handle the Unicode text, so the extraction succeeds.

The likely fix seems to be opening the -o file in UTF-8 explicitly, or otherwise allowing the output encoding to be configured.

Relevant code path

In beangulp.__init__:

@click.option('--output', '-o', type=click.File('w'), default='-', help='Output file.')

That seems to be where the locale-default encoding is coming from on Windows.