Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 723370 - sys-devel/gcc-10.1.0[lto] on musl/hardened: undefined reference to `libintl_dgettext'
Summary: sys-devel/gcc-10.1.0[lto] on musl/hardened: undefined reference to `libintl_d...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL: https://gcc.gnu.org/PR95194
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-05-16 11:35 UTC by Andrew Savchenko
Modified: 2020-06-11 23:05 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log.xz (build.log.xz,71.57 KB, application/octet-stream)
2020-05-16 11:35 UTC, Andrew Savchenko
Details
environment (environment,130.08 KB, text/plain)
2020-05-16 11:36 UTC, Andrew Savchenko
Details
emerge --info (emerge.info,8.18 KB, text/plain)
2020-05-16 11:37 UTC, Andrew Savchenko
Details
gcc-10.1.0-intl-no-asm-alias.patch (gcc-10.1.0-intl-no-asm-alias.patch,544 bytes, patch)
2020-05-18 22:01 UTC, Sergei Trofimovich (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Savchenko gentoo-dev 2020-05-16 11:35:27 UTC
Created attachment 639616 [details]
build.log.xz

Hi!

On musl/hardened gcc-10 fails to build with lto. Without lto it builds and works

/usr/x86_64-gentoo-linux-musl/bin/ld: /var/tmp/portage/sys-devel/gcc-10.1.0/temp/genmatch.NlicMk.ltrans2.ltrans.o: in function `main':
<artificial>:(.text.startup+0x94ba): undefined reference to `libintl_dgettext'
/usr/x86_64-gentoo-linux-musl/bin/ld: /var/tmp/portage/sys-devel/gcc-10.1.0/temp/genmatch.NlicMk.ltrans3.ltrans.o: in function `cpp_diagnostic(cpp_reader*, cpp_diagnostic_level, cpp_warning_reason, char const*, __va_list_tag (*) [1])':
<artificial>:(.text+0x2732): undefined reference to `libintl_dgettext'
/usr/x86_64-gentoo-linux-musl/bin/ld: /var/tmp/portage/sys-devel/gcc-10.1.0/temp/genmatch.NlicMk.ltrans3.ltrans.o: in function `cpp_diagnostic_with_line(cpp_reader*, cpp_diagnostic_level, cpp_warning_reason, unsigned int, unsigned int, char const*, __va_list_tag (*) [1]) [clone .lto_priv.0]':
<artificial>:(.text+0x4a2b): undefined reference to `libintl_dgettext'
/usr/x86_64-gentoo-linux-musl/bin/ld: /var/tmp/portage/sys-devel/gcc-10.1.0/temp/genmatch.NlicMk.ltrans3.ltrans.o: in function `cpp_error_at(cpp_reader*, cpp_diagnostic_level, unsigned int, char const*, ...)':
<artificial>:(.text+0x9b65): undefined reference to `libintl_dgettext'
/usr/x86_64-gentoo-linux-musl/bin/ld: /var/tmp/portage/sys-devel/gcc-10.1.0/temp/genmatch.NlicMk.ltrans5.ltrans.o: in function `cpp_error_at(cpp_reader*, cpp_diagnostic_level, rich_location*, char const*, ...) [clone .constprop.0]':
<artificial>:(.text+0x671f): undefined reference to `libintl_dgettext'
/usr/x86_64-gentoo-linux-musl/bin/ld: /var/tmp/portage/sys-devel/gcc-10.1.0/temp/genmatch.NlicMk.ltrans5.ltrans.o:<artificial>:(.text+0x98f3): more undefined references to `libintl_dgettext' follow
collect2: error: ld returned 1 exit status
Comment 1 Andrew Savchenko gentoo-dev 2020-05-16 11:36:46 UTC
Created attachment 639620 [details]
environment
Comment 2 Andrew Savchenko gentoo-dev 2020-05-16 11:37:34 UTC
Created attachment 639622 [details]
emerge --info
Comment 3 Sergei Trofimovich (RETIRED) gentoo-dev 2020-05-16 20:22:20 UTC
'libintl_dgettext' should probably be provided by ./../intl/libintl.a:dgettext.o.

But lto-dump says there is no such entry:

$ lto-dump -list build/intl/gettext.o
Type   Visibility  Size  Name
function  default     0  libintl_dcgettext
function  default     3  libintl_gettext

Only libintl_gettext is present.

$ lto-dump -dump-body=libintl_gettext build/intl/gettext.o
Gimple Body of Function: libintl_gettext
libintl_gettext (const char * msgid)
{
  <bb 2> [local count: 1073741824]:
  _4 = libintl_dcgettext (0B, msgid_2(D), 5);
  return _4;

}

I think it's a fancy LTO streaming bug. Only musl (and not glibc) is affected because it does not implement enough of gettext() interface and falls back to bundled intl implementation.

As a workaround you can try USE=-nls on gcc.

I'll check why LTO does not provide libintl_gettext object. Probably has something to do with K&R-style function declarations.
Comment 4 Georgy Yakovlev archtester gentoo-dev 2020-05-17 01:29:28 UTC
you should be using gcc from ::musl overlay btw, it contains some patches that are required for proper libdir(lib) support.
Comment 5 Andrew Savchenko gentoo-dev 2020-05-17 06:53:42 UTC
(In reply to Georgy Yakovlev from comment #4)
> you should be using gcc from ::musl overlay btw, it contains some patches
> that are required for proper libdir(lib) support.

Absolutely no. Because the only reason I'm using musl right now is a special container to test my own packages with musl, since I received a bugreport and later found more problems. Supporting musl overlay for my stuff may mask problems with the musl profile in the main tree.

If you have important fixes in the musl overlay, please port them to the main tree using musl profile.
Comment 6 Larry the Git Cow gentoo-dev 2020-05-17 19:48:17 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d11212bacfcd0359b65838e033292bb8e68ca4fb

commit d11212bacfcd0359b65838e033292bb8e68ca4fb
Author:     Jory Pratt <anarchy@gentoo.org>
AuthorDate: 2020-05-17 19:45:58 +0000
Commit:     Jory Pratt <anarchy@gentoo.org>
CommitDate: 2020-05-17 19:48:10 +0000

    profiles/features/musl/package.use.mask: mask nls useflag on gcc
    
    Closes: https://bugs.gentoo.org/723370
    
    Signed-off-by: Jory Pratt <anarchy@gentoo.org>

 profiles/features/musl/package.use.mask | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Comment 7 Jory A. Pratt gentoo-dev 2020-05-17 19:50:40 UTC
(In reply to Andrew Savchenko from comment #5)
> (In reply to Georgy Yakovlev from comment #4)
> > you should be using gcc from ::musl overlay btw, it contains some patches
> > that are required for proper libdir(lib) support.
> 
> Absolutely no. Because the only reason I'm using musl right now is a special
> container to test my own packages with musl, since I received a bugreport
> and later found more problems. Supporting musl overlay for my stuff may mask
> problems with the musl profile in the main tree.
> 
> If you have important fixes in the musl overlay, please port them to the
> main tree using musl profile.

Saying to port stuff to tree is easy. We have tried to work with toolchain to add support in packages like gcc/binutils/lld. The profile are maintained by blueness and myself, the overlay just ensures your toolchain for musl is what is expected for us to offer support.
Comment 8 Sergei Trofimovich (RETIRED) gentoo-dev 2020-05-18 08:27:13 UTC
(In reply to Sergei Trofimovich from comment #3)
> 'libintl_dgettext' should probably be provided by
> ./../intl/libintl.a:dgettext.o.

Now I somewhat get it. When stage2 is built we have 2 libintl.a files:
- intl/libintl.a: non-LTO objects, used by host gcc to build stage1
- prev-intl/libintl.a: lTO-only objects, user by host gcc to build stage2

When stage2 is linked it uses intl/libintl.a, which should work but does not.

If I use prev-intl/libintl.a explicitly linkage works fine.
Comment 9 Sergei Trofimovich (RETIRED) gentoo-dev 2020-05-18 21:43:21 UTC
(In reply to Sergei Trofimovich from comment #8)
> (In reply to Sergei Trofimovich from comment #3)
> > 'libintl_dgettext' should probably be provided by
> > ./../intl/libintl.a:dgettext.o.
> 
> Now I somewhat get it. When stage2 is built we have 2 libintl.a files:
> - intl/libintl.a: non-LTO objects, used by host gcc to build stage1
> - prev-intl/libintl.a: lTO-only objects, user by host gcc to build stage2
> 
> When stage2 is linked it uses intl/libintl.a, which should work but does not.
> 
> If I use prev-intl/libintl.a explicitly linkage works fine.

Heh, it was the other way around:
- prev-intl/libintl.a was non-LTO build on a previous stage run
- intl/libintl.a is an LTO-only build

Both should be fine to use. The problem was in the caller, not in this archive. Specifically:

"""
ld: genmatch.ltrans2.ltrans.o: in function `main':
gcc-10.1.0/libcpp/files.c:1439: undefined reference to `libintl_dgettext'
ld: genmatch.ltrans3.ltrans.o: in function `cpp_diagnostic(cpp_reader*, cpp_diagnostic_level, cpp_warning_reason, char const*, __va_list_tag (*) [1])':
gcc-10.1.0/work/gcc-10.1.0/libcpp/errors.c:42: undefined reference to `libintl_dgettext'
"""

Here 'errors.c' calls out 'libintl_dgettext'. That symbol has unusual definition:

$ ../prev-gcc/lto-dump -list errors.o | fgrep dgettext
function  default     0  *libintl_dgettext
$ ../prev-gcc/lto-dump -demangle -list errors.o | fgrep dgettext
function  default     0  dgettext

It's an asm 'libintl_dgettext' alias of C 'dgettext' function defined as:

"""
extern char *dgettext (const char *__domainname, const char *__msgid)
       __asm__ ("" "libintl_dgettext")
       __attribute__ ((__format_arg__ (2)));
"""

If we loot at LTO's resolution file (generated with '-save-temps' extra gcc flag) we see that 'libintl_gettext' does not get retained. That is slightly confusing.

That is another case of incompatibility of LTO and asm aliases: https://gcc.gnu.org/PR48200
Comment 10 Sergei Trofimovich (RETIRED) gentoo-dev 2020-05-18 22:01:53 UTC
Created attachment 640238 [details, diff]
gcc-10.1.0-intl-no-asm-alias.patch

If you are brave try gcc-10.1.0-intl-no-asm-alias.patch patch. It should fall back to macros and inline functions instead of using asm aliases for gettext.
Comment 11 Sergei Trofimovich (RETIRED) gentoo-dev 2020-05-18 22:39:17 UTC
Filed https://gcc.gnu.org/PR95194 to consider enabling more portable wrappers.
Comment 12 Andrew Savchenko gentoo-dev 2020-05-19 13:26:03 UTC
(In reply to Sergei Trofimovich from comment #10)
> Created attachment 640238 [details, diff] [details, diff]
> gcc-10.1.0-intl-no-asm-alias.patch
> 
> If you are brave try gcc-10.1.0-intl-no-asm-alias.patch patch. It should
> fall back to macros and inline functions instead of using asm aliases for
> gettext.

Thanks for the investigation and the patch! It works fine even with PGO, so now with USE="lto pgo" gcc builds and works fine.
Comment 13 Larry the Git Cow gentoo-dev 2020-05-19 22:02:53 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=0364868c119730d972ce101909856db7385513b6

commit 0364868c119730d972ce101909856db7385513b6
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2020-05-19 22:02:16 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2020-05-19 22:02:16 +0000

    10.1.0: Trick libintl not to use '_INTL_REDIRECT_ASM' mode as it's
    incompatible with LTO builds.
    
    glibc does not normally use libintl implementations and uses
    it's own primitives. But musl ond others do fall back to libintl.
    
    Reported-by: Andrew Savchenko
    Bug: https://bugs.gentoo.org/723370
    Bug: https://gcc.gnu.org/PR95194
    Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

 .../gentoo/30_all_lto-intl-workaround-PR95194.patch  | 20 ++++++++++++++++++++
 10.1.0/gentoo/README.history                         |  1 +
 2 files changed, 21 insertions(+)
Comment 14 Sergei Trofimovich (RETIRED) gentoo-dev 2020-05-19 22:03:36 UTC
Queued for next 10.1.0 patchset. Will take some time to cut it.

Meanwhile you can drop a patch to /etc/portage/patches/.
Comment 15 Andrew Savchenko gentoo-dev 2020-05-21 08:15:19 UTC
(In reply to Sergei Trofimovich from comment #14)
> Meanwhile you can drop a patch to /etc/portage/patches/.

This is exactly how I tested it ;)
Comment 16 Larry the Git Cow gentoo-dev 2020-06-11 23:00:26 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/gcc-patches.git/commit/?id=3c29cc8edec29f4cc48ffc77004cfd9d06b220c7

commit 3c29cc8edec29f4cc48ffc77004cfd9d06b220c7
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2020-06-11 22:52:51 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2020-06-11 22:52:51 +0000

    10.1.0: cut 2 patchset
    
    Four new patches:
    + 29_all_fix-float-hang-PR95118.patch: fix looping in printing real numbers
    + 30_all_lto-intl-workaround-PR95194.patch: add a workaround to fix gettext LTO
    + 31_all_ctor-range-PR95241.patch: fix constexpr constructors with ranges
    + 32_all_plugin-objdump.patch: user tuple-prefixed objdump
    
    Bug: https://gcc.gnu.org/PR95118
    Bug: https://bugs.gentoo.org/722774
    Reported-by: Andrew Savchenko
    Bug: https://gcc.gnu.org/PR95194
    Bug: https://bugs.gentoo.org/723370
    Bug: https://gcc.gnu.org/PR95241
    Bug: https://bugs.gentoo.org/726644
    Bug: https://gcc.gnu.org/PR95648
    Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

 10.1.0/gentoo/README.history | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 17 Larry the Git Cow gentoo-dev 2020-06-11 23:05:20 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=376f4a461521d7d7482c6ba4bc34a798bca0a120

commit 376f4a461521d7d7482c6ba4bc34a798bca0a120
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2020-06-11 23:01:24 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2020-06-11 23:05:10 +0000

    sys-devel/gcc: 10.1.0: cut 2 patchset
    
    Four new patches:
    + 29_all_fix-float-hang-PR95118.patch: fix looping in printing real numbers
    + 30_all_lto-intl-workaround-PR95194.patch: add a workaround to fix gettext LTO
    + 31_all_ctor-range-PR95241.patch: fix constexpr constructors with ranges
    + 32_all_plugin-objdump.patch: user tuple-prefixed objdump
    
    Bug: https://gcc.gnu.org/PR95118
    Closes: https://bugs.gentoo.org/722774
    Reported-by: Andrew Savchenko
    Bug: https://gcc.gnu.org/PR95194
    Closes: https://bugs.gentoo.org/723370
    Bug: https://gcc.gnu.org/PR95241
    Closes: https://bugs.gentoo.org/726644
    Bug: https://gcc.gnu.org/PR95648
    Package-Manager: Portage-2.3.100, Repoman-2.3.22
    Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

 sys-devel/gcc/Manifest             |  1 +
 sys-devel/gcc/gcc-10.1.0-r1.ebuild | 13 +++++++++++++
 2 files changed, 14 insertions(+)