Using strace, we can see that iterating over lines with this code it reads 1 byte at a time: file_in = codecs.iterdecode(sys.stdin.buffer.raw, 'utf_8', errors='replace') This fixes it to use a read buffer: file_in = io.TextIOWrapper(sys.stdin.buffer.raw, 'utf_8', errors='replace')
Using sys.stdin.buffer instead of sys.stdin.buffer.raw also solves the problem: file_in = codecs.iterdecode(sys.stdin.buffer, 'utf_8', errors='replace')
Patch posted for review: https://archives.gentoo.org/gentoo-portage-dev/message/5dcbb57f7b7fe8717ef9843909b296ba https://github.com/gentoo/portage/pull/256
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=f1367a2eeb0c911a743b0da5f37a7e9f7de76488 commit f1367a2eeb0c911a743b0da5f37a7e9f7de76488 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2018-02-14 20:22:03 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2018-02-17 18:25:45 +0000 filter-bash-environment.py: use buffered input, raw bytes (bug 647654) Use sys.stdin.buffer instead of sys.stdin.buffer.raw, for buffered input. Also use raw bytes instead of unicode strings, in order to avoid making assumptions about character encodings, and also to avoid overhead from unicode decoding/encoding. Since the % operator does not support bytes operands in python3.4, use the + operator to format strings of bytes. Bug: https://bugs.gentoo.org/647654 bin/filter-bash-environment.py | 47 +++++------ pym/portage/tests/bin/test_filter_bash_env.py | 115 ++++++++++++++++++++++++++ 2 files changed, 137 insertions(+), 25 deletions(-)}
Fixed in portage-2.3.40-r1.
