Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 164504 - media-sound/mpg123 TEXTRELs
Summary: media-sound/mpg123 TEXTRELs
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: x86 Linux
: High QA (vote)
Assignee: Gentoo Sound Team
URL:
Whiteboard:
Keywords:
: 385437 475244 (view as bug list)
Depends on:
Blocks: 298512
  Show dependency tree
 
Reported: 2007-01-30 07:44 UTC by denis
Modified: 2014-02-09 18:47 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description denis 2007-01-30 07:44:09 UTC
man:
gzipping man page: mpg123.1
strip: i686-pc-linux-gnu-strip --strip-unneeded
   usr/bin/mpg123-generic
   usr/bin/mpg123-i486
   usr/bin/mpg123-esd
   usr/bin/mpg123-mmx

QA Notice: the following files contain executable stacks
 Files with executable stacks will not work properly (or at all!)
 on some architectures/operating systems.  A bug should be filed
 at http://bugs.gentoo.org/ to make sure the file is fixed.
 For more information, see http://hardened.gentoo.org/gnu-stack.xml
 Please include this file in your report:
 /var/tmp/portage/mpg123-0.59s-r11/temp/scanelf-execstack.log
RWX --- --- usr/bin/mpg123-i486
RWX --- --- usr/bin/mpg123-esd
RWX --- --- usr/bin/mpg123-mmx

/var/tmp/portage/mpg123-0.59s-r11/temp/scanelf-execstack.log:
!WX --- --- work/mpg123/dct64_MMX.o
!WX --- --- work/mpg123/tabinit_MMX.o
!WX --- --- work/mpg123/decode_MMX.o
RWX --- --- work/mpg123/gentoo-bin/mpg123-i486
RWX --- --- work/mpg123/gentoo-bin/mpg123-esd
RWX --- --- work/mpg123/gentoo-bin/mpg123-mmx
!WX --- --- work/mpg123/precompiled/linux-i386/dct64_3dnow.o
!WX --- --- work/mpg123/precompiled/linux-i386/decode_3dnow.o
RWX --- --- image/usr/bin/mpg123-i486
RWX --- --- image/usr/bin/mpg123-esd
RWX --- --- image/usr/bin/mpg123-mmx

Reproducible: Always

Steps to Reproduce:
1.emerge mpg123
2.
3.
Comment 1 Natanael Copa 2007-03-09 13:09:27 UTC
media-sound/mpg123-0.59s-r11 segfaults on hardened uclibc. 0.65 works just fine. Can we mark 0.65 as stable?


Comment 2 Björn Bylander 2007-06-03 20:24:52 UTC
Bug exists in 0.65 too:

strip: i686-pc-linux-gnu-strip --strip-unneeded
   usr/bin/mpg123

  QA Notice: The following files contain executable stacks
   Files with executable stacks will not work properly (or at all!)
   on some architectures/operating systems.  A bug should be filed
   at http://bugs.gentoo.org/ to make sure the file is fixed.
   For more information, see http://hardened.gentoo.org/gnu-stack.xml
   Please include this file in your report:
   /var/tmp/portage/media-sound/mpg123-0.65/temp/scanelf-execstack.log
  RWX --- --- usr/bin/mpg123

Couldn't find /var/tmp/portage/media-sound/mpg123-0.65/temp/scanelf-execstack.log however.
Comment 3 Luis Medinas (RETIRED) gentoo-dev 2007-08-09 04:50:31 UTC
add hardened team to take a look at this one.
Comment 4 Samuli Suominen gentoo-dev 2007-09-20 15:02:49 UTC
Still present in 0.67.
Comment 5 Markus Meier gentoo-dev 2007-12-10 21:35:31 UTC
>>> Completed installing mpg123-1.0_rc2 into /var/tmp/portage/media-sound/mpg123-1.0_rc2/image/

ecompressdir: bzip2 -9 usr/share/man
strip: i686-pc-linux-gnu-strip --strip-unneeded -R .comment
   usr/lib/mpg123/output_alsa.so
   usr/lib/mpg123/output_oss.so
   usr/lib/mpg123/output_pulse.so
   usr/lib/mpg123/output_sdl.so
   usr/lib/mpg123/output_dummy.so
   usr/lib/libmpg123.so.0.0.0
   usr/bin/mpg123

 * QA Notice: The following files contain runtime text relocations
 * TEXTREL usr/lib/libmpg123.so.0.0.0

 * QA Notice: The following files contain executable stacks
 * RWX --- --- usr/lib/libmpg123.so.0.0.0
Comment 6 Thomas Orgis 2008-05-31 08:39:04 UTC
Could someone inform (explain) the mpg123 project (me;-) about this issue and how to fix it?
It wouldn't magically disappear in new versions when I'm not aware of it.
Comment 7 Samuli Suominen gentoo-dev 2008-06-08 09:39:27 UTC
(In reply to comment #6)
> Could someone inform (explain) the mpg123 project (me;-) about this issue and
> how to fix it?
> It wouldn't magically disappear in new versions when I'm not aware of it.
> 

These guides explain how to find and fix,

execstacks, http://www.gentoo.org/proj/en/hardened/gnu-stack.xml
textrels, http://www.gentoo.org/proj/en/hardened/pic-fix-guide.xml
Comment 8 Thomas Orgis 2008-08-03 21:23:06 UTC
The executable stacks are fixed in mpg123 svn.
Textrels require more thought/work.
Comment 9 Thomas Orgis 2008-08-03 22:07:12 UTC
Ok, having a look at the textrel issue... this is going to be a major PITA.
It's basically all of the plain assembly routines. They have local data. They use global data.
Also, they use registers... heck, and we all know that x86 has basically _none_ of these to use, much less to spare.
This code was not designed for shared libs with PIC... too bad it just works despite that, isn't it? ;-)

In practice, I'll have to wonder about accessing the stuff only through the stack or really turn all the stuff into gcc inline asm.
But I don't fancy the latter since this ties the code to a specific compiler (and I had that hope that generic AT&T assembly is a bit more portable than a gnu extension).
Also, it looks ugly. I really does. I may be being impractical there, though.

Well, here is a short-term solution, of course: Don't compile any of the assembly code into libmpg123.
./configure --with-cpu=i386 (or generic_fpu) achieves that...
Yet another one, to preserve the optimizations, would be ditching the dynamic library; just installing libmpg123.a (or installing the full-featured .a plus the non-asm .so library).

I'm mildly open to other suggestions...
Comment 10 Samuli Suominen gentoo-dev 2009-04-30 13:38:16 UTC
(In reply to comment #8)
> The executable stacks are fixed in mpg123 svn.
> Textrels require more thought/work.
> 

Thanks. The way it works for us is that we don't allow those optimizations
to be enabled if they cause TEXTRELs for our hardened branch.. but it's
only a workaround.
Comment 11 Samuli Suominen gentoo-dev 2009-04-30 13:40:08 UTC
Also using static archives is overhead..
Comment 12 Thomas Orgis 2010-05-20 09:14:35 UTC
I wonder if there's a relation of this topic to that bug in debian:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=580095

the -pie linker flag causing corruption... can you reproduce something like that on hardened gentoo?
Comment 13 Tim Harder gentoo-dev 2011-07-18 07:03:20 UTC
Is this still not fixed in the mpg123 versions currently in the tree?
Comment 14 Francisco Blas Izquierdo Riera gentoo-dev 2011-10-03 13:30:32 UTC
*** Bug 385437 has been marked as a duplicate of this bug. ***
Comment 15 Maxim Kammerer 2012-02-15 20:41:43 UTC
(In reply to comment #10)
> Thanks. The way it works for us is that we don't allow those optimizations
> to be enabled if they cause TEXTRELs for our hardened branch.. but it's
> only a workaround.

Perhaps it's time to put the workaround in the ebuild? As it is, mpg123 and audacious require paxctl -m, otherwise (on hardened kernel):

error while loading shared libraries: /usr/lib/libmpg123.so.0: cannot make segment writable for relocation: Permission denied

It seems that masking mmx, sse, 3dnow, 3dnowext and altivec(?) USE flags for mpg123 on hardened gets rid of TEXTRELs in the shared library.
Comment 16 Magnus Granberg gentoo-dev 2012-02-16 17:16:02 UTC
Have disable mmx, sse, 3dnow and 3dnowext for hardened on x86.
This is done on 1.13.4
Look like the asm files for the lib need alot of work to make
it work with -fPIC.
scanelf -qT lib givs a long list to fix and register ebx is uset allover the code.
Comment 17 Thomas Orgis 2012-02-16 18:29:01 UTC
As mpg123 upstream I just want to confirm what is apparent here:

1. Disabling the offending ASM optimizations on x86 is the way to go.
2. Rewriting of the routines to make them PIC-friendly won't happen.

About point 2: I don't see me having the time to invest into that endeavour and also I fear that it will impact the efficiency of said code. There is hope that the gains from SIMD computations will dwarf any negative effect, but you don't know for sure. I'm still trying to make sense out of decoding speed comparisons with MPlayer's mpg123 binding. Performance on x86 is so flaky, dependent on the finest details and compiler oddities.

Hardened gentoo is about playing it safe, and that may be an independent reason to use the plain C code anyway, as enabling the assembly adds more functionally redundant code, more target area for attacks. The SSE opt on x86-64 is still a no-brainer, I think, but you could apply such a rule there, too, of course.

Oh, by the way: The i586 assembly is fine, or is it just disabled anyway on gentoo?
Comment 18 Maxim Kammerer 2012-02-16 18:49:24 UTC
(In reply to comment #17)
> Oh, by the way: The i586 assembly is fine, or is it just disabled anyway on
> gentoo?

Here is the gist of the relevant part of the ebuild:

  _cpu=generic_fpu

  use altivec && _cpu=altivec

  if arch == amd64
      use sse      && _cpu=x86-64
  else
      use mmx      && _cpu=mmx
      use 3dnow    && _cpu=3dnow
      use sse      && _cpu=x86
      use 3dnowext && _cpu=x86

Later assignments take precedence. configure then gets a --with-cpu=_cpu parameter.

So with mmx/sse/3dnow/3dnowext disabled on x86/amd64, the configured cpu is generic_fpu, which disables assembly altogether, if I understood you correctly.
Comment 19 Thomas Orgis 2012-02-16 19:00:45 UTC
Ah, funny selection you have there. So you make mmx and 3dnow exclusive, while sse and 3dnowext result in a build that features all available x86 optimizations.

Well, I don't know exactly how USE flags shall be interpreted on gentoo (if mmx code is to be forbidden when the mmx flag is not set, or just not enabled explicitly), so I won't discuss that logic. I can only confirm: generic_fpu won't use any custom assembly routines, so should be PIC-safe.
Comment 20 Thomas Orgis 2012-02-16 19:02:12 UTC
Oh, and just for fun: You could try comparing generic_fpu with i386_fpu ... both use plain C code, just the latter tries to cater more for the limitations of i386. The effect of that has to be analyzed on your specific CPU with your specific compiler:-/
Comment 21 Magnus Granberg gentoo-dev 2012-02-16 21:20:23 UTC
(In reply to comment #17)
> As mpg123 upstream I just want to confirm what is apparent here:
> 
> 1. Disabling the offending ASM optimizations on x86 is the way to go.
> 2. Rewriting of the routines to make them PIC-friendly won't happen.
> 
> About point 2: I don't see me having the time to invest into that endeavour and
> also I fear that it will impact the efficiency of said code. There is hope that
> the gains from SIMD computations will dwarf any negative effect, but you don't
> know for sure. I'm still trying to make sense out of decoding speed comparisons
> with MPlayer's mpg123 binding. Performance on x86 is so flaky, dependent on the
> finest details and compiler oddities.
> 
> Hardened gentoo is about playing it safe, and that may be an independent reason
> to use the plain C code anyway, as enabling the assembly adds more functionally
> redundant code, more target area for attacks. The SSE opt on x86-64 is still a
> no-brainer, I think, but you could apply such a rule there, too, of course.
> 
> Oh, by the way: The i586 assembly is fine, or is it just disabled anyway on
> gentoo?

1. If 2 won't happen, i need to do some thing and to disable MPROTECT is not the option to do.
2. To make the asm code PIC friendly will take alot of time that i don't have now. We don't have this prob on x86_64
Comment 22 Thomas Orgis 2012-02-16 21:38:56 UTC
I'm confused, what thing needs to be done here? I though it is settled now that for hardened systems, the optimizations will be disabled.

Are you talking about a normal gentoo install? Aren't linker relocations exempt from MPROTECT?
Comment 23 Magnus Granberg gentoo-dev 2012-02-17 17:29:01 UTC
(In reply to comment #22)
> I'm confused, what thing needs to be done here? I though it is settled now that
> for hardened systems, the optimizations will be disabled.
> 
> Are you talking about a normal gentoo install? Aren't linker relocations exempt
> from MPROTECT?

yes it is settled now that we disable the optimization.
Comment 24 David 2012-04-04 15:10:07 UTC
(In reply to comment #23)
> yes it is settled now that we disable the optimization.


Does this mean doing the same ('paxctl -m') thing as what other packages do...


 * Messages for package app-office/libreoffice-3.5.0.3:

 * Fallback PaX marking -m
 *      /usr/lib/libreoffice/program/soffice.bin
Comment 25 Maxim Kammerer 2012-07-08 20:44:44 UTC
(In reply to comment #23)
> yes it is settled now that we disable the optimization.

It is now in stable mpg123 -- bug can be closed?
Comment 26 Samuli Suominen gentoo-dev 2012-07-09 09:56:54 UTC
(In reply to comment #25)
> (In reply to comment #23)
> > yes it is settled now that we disable the optimization.
> 
> It is now in stable mpg123 -- bug can be closed?

it only means it's not a problem for hardened anymore as they disable the code causing text relocations, it doesn't mean the bug is solved in any way as it's possible to rewrite the code to not cause textrels in the first place

the bug should stay open for long as the workarounds are in tree
Comment 27 Maxim Kammerer 2012-07-09 10:01:43 UTC
(In reply to comment #26)
> it only means it's not a problem for hardened anymore as they disable the
> code causing text relocations

It's the standard practice in hardened -- take a look at ffmpeg, libav, or similar packages with included assembly code.

> it doesn't mean the bug is solved in any way
> as it's possible to rewrite the code to not cause textrels in the first place

Upstream made clear it's not going to happen, I think.

> the bug should stay open for long as the workarounds are in tree

Why not resolve it upstream?
Comment 28 Thomas Orgis 2012-07-09 20:15:48 UTC
Concerning rewriting the code to fix this: The assembly code is fully redundant with the plain C version; it just is a workaround for compilers that cannot be coerced into doing those optimizations themselves. But apart from speed differences, which might be alleviated by future compilers (as it happened for the i586 optimization, apparently --- would need to check an actual i586 CPU), you can consider the plain C code as the rewritten safe version of the assembly code.

So, you could consider the bug fixed in that respect.

I don't say that upstream mpg123 will never feature textrel-clean assembly, I'm just giving the realistic prospect that there won't be developer time invested in that particular goal.

Nevertheless, keeping this bug open doesn't hurt anyone. Perhaps it can be closed properly in 5 years or so;-)
Comment 29 Jason McGuiness 2012-12-10 12:42:50 UTC
I have a non-hardened Gentoo system, so don't mind the optimisations, even if they may be imperfect. Unfortunately on my system when I emerge I get:

[ebuild     U  ] media-sound/mpg123-1.14.4 [1.14.2] USE="alsa ipv6 sdl sse (-3dnow) (-3dnowext) (-altivec) (-coreaudio) -int-quality% -jack (-mmx) -nas -oss -portaudio -pulseaudio" 779 kB

In my "make.conf" I have "3dnow", "3dnowext" & "mmx" enabled on my opteron system. The man page for emerge indicates that these USE flags have been "forced, masked or removed". Might the masking of these flags for hardened systems have accidentally leaked into non-hardened systems?

How can I get my optimisations back, please?
Comment 30 Jason McGuiness 2012-12-10 12:45:37 UTC
(In reply to comment #29)
> I have a non-hardened Gentoo system, so don't mind the optimisations, even
> if they may be imperfect. Unfortunately on my system when I emerge I get:
> 
> [ebuild     U  ] media-sound/mpg123-1.14.4 [1.14.2] USE="alsa ipv6 sdl sse
> (-3dnow) (-3dnowext) (-altivec) (-coreaudio) -int-quality% -jack (-mmx) -nas
> -oss -portaudio -pulseaudio" 779 kB
> 
> In my "make.conf" I have "3dnow", "3dnowext" & "mmx" enabled on my opteron
> system. The man page for emerge indicates that these USE flags have been
> "forced, masked or removed". Might the masking of these flags for hardened
> systems have accidentally leaked into non-hardened systems?
> 
> How can I get my optimisations back, please?

Gah. Very sorry. Re-read the comments above, it answers my queries. Please ignore my post. If I can delete my comment I will.
Comment 31 Alexis Ballier gentoo-dev 2013-08-02 16:52:57 UTC
*** Bug 475244 has been marked as a duplicate of this bug. ***