Skip to content

Fix UnicodeEncodeError in CLI docs on non-UTF consoles#2362

Open
RedZapdos123 wants to merge 1 commit intojoke2k:masterfrom
RedZapdos123:fix-cli-doc-unicode-2361
Open

Fix UnicodeEncodeError in CLI docs on non-UTF consoles#2362
RedZapdos123 wants to merge 1 commit intojoke2k:masterfrom
RedZapdos123:fix-cli-doc-unicode-2361

Conversation

@RedZapdos123
Copy link
Copy Markdown

@RedZapdos123 RedZapdos123 commented Apr 7, 2026

Changes done:

Fixes a CLI crash when printing provider docs to terminals using non-UTF encodings (for example cp950 on Windows).

Problem Explaination:

python -m faker can raise UnicodeEncodeError while printing provider examples that include characters unsupported by the output encoding (such as emoji on cp950).
This happens in print_provider() when writing formatted documentation lines.

Fix:

  • Adds _encode_for_output() in faker/cli.py to safely encode output strings for the active stream encoding.
  • Falls back to backslashreplace when characters are not encodable, so output remains printable and the CLI does not crash.
  • Uses this safe encoding path for provider headings and example lines printed by print_provider().
  • Adds regression test test_print_provider_handles_non_utf_output to verify cp950 output no longer crashes and preserves escaped content.

Fixes #2361

AI Assistance Disclosure (as REQUIRED):

  • No AI tools were used in preparing this PR.
  • If AI tools were used, I have disclosed which ones, and fully reviewed and verified their output.

AI tool used: GitHub Copilot (GPT-5.3-Codex), used for code drafting and test drafting. All changes were manually reviewed and validated by running tests.

Checklist:

  • I have read the documentation about CONTRIBUTING.
  • I have read the documentation about Coding style.
  • I have run make lint.

The screenshots of the tests run, for validation (ran on WSL):

image

Signed-off-by: Mridankan Mandal <xerontitan90@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants