Summary: | sys-devel/gcc-10 miscompiles dev-lang/python-3.9.0_beta3 on HPPA - src_compile(): Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Jeroen Roovers (RETIRED) <jer> |
Component: | Current packages | Assignee: | Gentoo Toolchain Maintainers <toolchain> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | hppa, mgorny, python, slyfox |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | HPPA | ||
OS: | Linux | ||
URL: | https://gcc.gnu.org/PR96015 | ||
See Also: |
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96015 https://bugs.gentoo.org/show_bug.cgi?id=706426 |
||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 915000 | ||
Attachments: |
dev-lang:python-3.9.0_beta3:20200624-205627.log
0001-Disable-fdelayed-branch-by-default.patch |
Description
Jeroen Roovers (RETIRED)
2020-06-25 08:15:58 UTC
3.9.0_beta1 also has this problem. 3.9.0_alpha6 too With -O0 the problem goes away. (In reply to Jeroen Roovers from comment #3) > With -O0 the problem goes away. -O1 is good, too. Maybe a compiler bug? Could you try with an older gcc? When I compile with =sys-devel/gcc-9.3.0 then everything seems all right. Reproduced on hake. Will debug. While poking more at hppa's failure valgrind claims it sees use-after-free at python initialization time when I'm running the same command: """ $ LD_LIBRARY_PATH=/tmp/portage/dev-lang/python-3.9.0_beta3/work/Python-3.9.0b3 valgrind <...> --log-file=lf ./python -E -S -m sysconfig --generate-posix-vars ... ==418994== Invalid read of size 4 ==418994== at 0x49332E3: pymalloc_free (obmalloc.c:1866) ==418994== by 0x49332E3: _PyObject_Free (obmalloc.c:1920) ==418994== by 0x491C421: dictresize (dictobject.c:1285) ==418994== by 0x4921F47: insertion_resize (dictobject.c:1040) ==418994== by 0x4921F47: PyDict_SetDefault (dictobject.c:2964) ==418994== by 0x497977F: PyUnicode_InternInPlace (unicodeobject.c:15586) ==418994== by 0x49799C9: PyUnicode_InternFromString (unicodeobject.c:15623) ==418994== by 0x48F92FD: descr_new (descrobject.c:890) ==418994== by 0x48F92FD: PyDescr_NewMethod (descrobject.c:939) ==418994== by 0x4943292: add_methods (typeobject.c:4989) ==418994== by 0x4943292: PyType_Ready (typeobject.c:5454) ==418994== by 0x492D96B: _PyTypes_Init (object.c:1748) ==418994== by 0x49F0377: pycore_init_types (pylifecycle.c:589) ==418994== by 0x49F0377: pycore_interp_init (pylifecycle.c:726) ==418994== by 0x49F1DC8: pyinit_config (pylifecycle.c:767) ==418994== by 0x49F1DC8: pyinit_core.constprop.0 (pylifecycle.c:928) ==418994== by 0x49F35E0: Py_InitializeFromConfig (pylifecycle.c:1138) ==418994== by 0x49F35E0: Py_InitializeFromConfig (pylifecycle.c:1123) ==418994== by 0x4A13269: pymain_init (main.c:66) ==418994== by 0x4A14343: pymain_main (main.c:653) ==418994== by 0x4A14343: Py_BytesMain (main.c:686) ==418994== by 0x4C3DD49: (below main) (libc-start.c:308) ==418994== Address 0x4f4b020 is 128 bytes inside a block of size 1,112 free'd ==418994== at 0x48389CB: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==418994== by 0x493331B: _PyObject_Free (obmalloc.c:1922) ==418994== by 0x493331B: _PyObject_Free (obmalloc.c:1913) ==418994== by 0x491C421: dictresize (dictobject.c:1285) ==418994== by 0x4921F47: insertion_resize (dictobject.c:1040) ==418994== by 0x4921F47: PyDict_SetDefault (dictobject.c:2964) ==418994== by 0x497977F: PyUnicode_InternInPlace (unicodeobject.c:15586) ==418994== by 0x49799C9: PyUnicode_InternFromString (unicodeobject.c:15623) ==418994== by 0x494D077: _PyTypes_InitSlotDefs (typeobject.c:7532) ==418994== by 0x492D920: _PyTypes_Init (object.c:1736) ==418994== by 0x49F0377: pycore_init_types (pylifecycle.c:589) ==418994== by 0x49F0377: pycore_interp_init (pylifecycle.c:726) ==418994== by 0x49F1DC8: pyinit_config (pylifecycle.c:767) ==418994== by 0x49F1DC8: pyinit_core.constprop.0 (pylifecycle.c:928) ==418994== by 0x49F35E0: Py_InitializeFromConfig (pylifecycle.c:1138) ==418994== by 0x49F35E0: Py_InitializeFromConfig (pylifecycle.c:1123) ==418994== by 0x4A13269: pymain_init (main.c:66) ==418994== by 0x4A14343: pymain_main (main.c:653) ==418994== by 0x4A14343: Py_BytesMain (main.c:686) ==418994== by 0x4C3DD49: (below main) (libc-start.c:308) ==418994== Block was alloc'd at ==418994== at 0x483779F: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==418994== by 0x4933E27: _PyObject_Malloc (obmalloc.c:1645) ==418994== by 0x4933E27: _PyObject_Malloc (obmalloc.c:1638) ==418994== by 0x491BB3E: new_keys_object (dictobject.c:570) ==418994== by 0x491C17E: dictresize (dictobject.c:1223) ==418994== by 0x4921F47: insertion_resize (dictobject.c:1040) ==418994== by 0x4921F47: PyDict_SetDefault (dictobject.c:2964) ==418994== by 0x497977F: PyUnicode_InternInPlace (unicodeobject.c:15586) ==418994== by 0x49799C9: PyUnicode_InternFromString (unicodeobject.c:15623) ==418994== by 0x494D077: _PyTypes_InitSlotDefs (typeobject.c:7532) ==418994== by 0x492D920: _PyTypes_Init (object.c:1736) ==418994== by 0x49F0377: pycore_init_types (pylifecycle.c:589) ==418994== by 0x49F0377: pycore_interp_init (pylifecycle.c:726) ==418994== by 0x49F1DC8: pyinit_config (pylifecycle.c:767) ==418994== by 0x49F1DC8: pyinit_core.constprop.0 (pylifecycle.c:928) ==418994== by 0x49F35E0: Py_InitializeFromConfig (pylifecycle.c:1138) ==418994== by 0x49F35E0: Py_InitializeFromConfig (pylifecycle.c:1123) ==418994== by 0x4A13269: pymain_init (main.c:66) ==418994== by 0x4A14343: pymain_main (main.c:653) ==418994== by 0x4A14343: Py_BytesMain (main.c:686) ==418994== by 0x4C3DD49: (below main) (libc-start.c:308) """ Looks like missing refcounts somewhere? Are you talking with gcc9 or 10? i.e. is this confirmed to be a real bug in code, and not compiler issue? valgrind reports the same use-after-free error on gcc-9.3.0 on amd64. I don't know if it's related to hppa failure or not. The bug happens at the very least due to invalid generated bytecode. gcc-10.1.0 generates invalid code for 'long_richcompare()' and breaks many things including bootstrap. Filed as https://gcc.gnu.org/PR96015. Created attachment 647498 [details, diff]
0001-Disable-fdelayed-branch-by-default.patch
The error seems to come from "dbr" (delayed branch) pass in gcc/reorg.c.
At least sh has exactly the same problem, sparc and mips are probably also affected in some form.
Let's disable -fdelayed-branch on arches as 0001-Disable-fdelayed-branch-by-default.patch until we have better fix upstream. It will pessimize performance a bit. And also might uncover more corner cases.
Queued the patches for upcoming gcc-10 patchset. Might cut it this weekend. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=9d1b439e8c78ceaa561094657a0d42845611ab63 commit 9d1b439e8c78ceaa561094657a0d42845611ab63 Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2020-07-04 07:37:03 +0000 Commit: Sergei Trofimovich <slyfox@gentoo.org> CommitDate: 2020-07-04 07:37:03 +0000 11.0.0: cut 2 patchset Two new patches: + 29_all_plugin-objdump.patch: detect tool-prefixed objdump + 30_all_fno-delayed-branch.patch: disable -fdelayed-branch to to fix python mis-compilation. Bug: https://gcc.gnu.org/PR95648 Reported-by: Jeroen Roovers Bug: https://gcc.gnu.org/PR96015 Bug: https://bugs.gentoo.org/729570 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> 11.0.0/gentoo/README.history | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ed08195b77a35057668ea47c9300861828c2b779 commit ed08195b77a35057668ea47c9300861828c2b779 Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2020-07-04 07:43:58 +0000 Commit: Sergei Trofimovich <slyfox@gentoo.org> CommitDate: 2020-07-04 08:09:14 +0000 sys-devel/gcc: 11.0.0: cut 2 patchset Two new patches: + 29_all_plugin-objdump.patch: detect tool-prefixed objdump + 30_all_fno-delayed-branch.patch: disable -fdelayed-branch to to fix python mis-compilation. Bug: https://gcc.gnu.org/PR95648 Reported-by: Jeroen Roovers Bug: https://gcc.gnu.org/PR96015 Bug: https://bugs.gentoo.org/729570 Package-Manager: Portage-2.3.103, Repoman-2.3.23 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> sys-devel/gcc/Manifest | 2 +- sys-devel/gcc/gcc-11.0.0_pre9999.ebuild | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) Should be fixed by gcc-10.1.0-r2 in https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9a478857e97c4a4d2769ded41b859c5dcfc3cbf9 |