| Summary: | media-sound/mpg123 TEXTRELs | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | denis <denisr> |
| Component: | Current packages | Assignee: | Gentoo Sound Team <sound> |
| Status: | RESOLVED FIXED | ||
| Severity: | QA | CC: | esigra, fordfrog, gentoo-bugs, hardened, josan_pansa, jstein, pacho, pageexec, thomas-forum, write2David |
| Priority: | High | ||
| Version: | unspecified | ||
| Hardware: | x86 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 298512 | ||
|
Description
denis
2007-01-30 07:44:09 UTC
media-sound/mpg123-0.59s-r11 segfaults on hardened uclibc. 0.65 works just fine. Can we mark 0.65 as stable? Bug exists in 0.65 too: strip: i686-pc-linux-gnu-strip --strip-unneeded usr/bin/mpg123 QA Notice: The following files contain executable stacks Files with executable stacks will not work properly (or at all!) on some architectures/operating systems. A bug should be filed at http://bugs.gentoo.org/ to make sure the file is fixed. For more information, see http://hardened.gentoo.org/gnu-stack.xml Please include this file in your report: /var/tmp/portage/media-sound/mpg123-0.65/temp/scanelf-execstack.log RWX --- --- usr/bin/mpg123 Couldn't find /var/tmp/portage/media-sound/mpg123-0.65/temp/scanelf-execstack.log however. add hardened team to take a look at this one. Still present in 0.67. >>> Completed installing mpg123-1.0_rc2 into /var/tmp/portage/media-sound/mpg123-1.0_rc2/image/
ecompressdir: bzip2 -9 usr/share/man
strip: i686-pc-linux-gnu-strip --strip-unneeded -R .comment
usr/lib/mpg123/output_alsa.so
usr/lib/mpg123/output_oss.so
usr/lib/mpg123/output_pulse.so
usr/lib/mpg123/output_sdl.so
usr/lib/mpg123/output_dummy.so
usr/lib/libmpg123.so.0.0.0
usr/bin/mpg123
* QA Notice: The following files contain runtime text relocations
* TEXTREL usr/lib/libmpg123.so.0.0.0
* QA Notice: The following files contain executable stacks
* RWX --- --- usr/lib/libmpg123.so.0.0.0
Could someone inform (explain) the mpg123 project (me;-) about this issue and how to fix it? It wouldn't magically disappear in new versions when I'm not aware of it. (In reply to comment #6) > Could someone inform (explain) the mpg123 project (me;-) about this issue and > how to fix it? > It wouldn't magically disappear in new versions when I'm not aware of it. > These guides explain how to find and fix, execstacks, http://www.gentoo.org/proj/en/hardened/gnu-stack.xml textrels, http://www.gentoo.org/proj/en/hardened/pic-fix-guide.xml The executable stacks are fixed in mpg123 svn. Textrels require more thought/work. Ok, having a look at the textrel issue... this is going to be a major PITA. It's basically all of the plain assembly routines. They have local data. They use global data. Also, they use registers... heck, and we all know that x86 has basically _none_ of these to use, much less to spare. This code was not designed for shared libs with PIC... too bad it just works despite that, isn't it? ;-) In practice, I'll have to wonder about accessing the stuff only through the stack or really turn all the stuff into gcc inline asm. But I don't fancy the latter since this ties the code to a specific compiler (and I had that hope that generic AT&T assembly is a bit more portable than a gnu extension). Also, it looks ugly. I really does. I may be being impractical there, though. Well, here is a short-term solution, of course: Don't compile any of the assembly code into libmpg123. ./configure --with-cpu=i386 (or generic_fpu) achieves that... Yet another one, to preserve the optimizations, would be ditching the dynamic library; just installing libmpg123.a (or installing the full-featured .a plus the non-asm .so library). I'm mildly open to other suggestions... (In reply to comment #8) > The executable stacks are fixed in mpg123 svn. > Textrels require more thought/work. > Thanks. The way it works for us is that we don't allow those optimizations to be enabled if they cause TEXTRELs for our hardened branch.. but it's only a workaround. Also using static archives is overhead.. I wonder if there's a relation of this topic to that bug in debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=580095 the -pie linker flag causing corruption... can you reproduce something like that on hardened gentoo? Is this still not fixed in the mpg123 versions currently in the tree? *** Bug 385437 has been marked as a duplicate of this bug. *** (In reply to comment #10) > Thanks. The way it works for us is that we don't allow those optimizations > to be enabled if they cause TEXTRELs for our hardened branch.. but it's > only a workaround. Perhaps it's time to put the workaround in the ebuild? As it is, mpg123 and audacious require paxctl -m, otherwise (on hardened kernel): error while loading shared libraries: /usr/lib/libmpg123.so.0: cannot make segment writable for relocation: Permission denied It seems that masking mmx, sse, 3dnow, 3dnowext and altivec(?) USE flags for mpg123 on hardened gets rid of TEXTRELs in the shared library. Have disable mmx, sse, 3dnow and 3dnowext for hardened on x86. This is done on 1.13.4 Look like the asm files for the lib need alot of work to make it work with -fPIC. scanelf -qT lib givs a long list to fix and register ebx is uset allover the code. As mpg123 upstream I just want to confirm what is apparent here: 1. Disabling the offending ASM optimizations on x86 is the way to go. 2. Rewriting of the routines to make them PIC-friendly won't happen. About point 2: I don't see me having the time to invest into that endeavour and also I fear that it will impact the efficiency of said code. There is hope that the gains from SIMD computations will dwarf any negative effect, but you don't know for sure. I'm still trying to make sense out of decoding speed comparisons with MPlayer's mpg123 binding. Performance on x86 is so flaky, dependent on the finest details and compiler oddities. Hardened gentoo is about playing it safe, and that may be an independent reason to use the plain C code anyway, as enabling the assembly adds more functionally redundant code, more target area for attacks. The SSE opt on x86-64 is still a no-brainer, I think, but you could apply such a rule there, too, of course. Oh, by the way: The i586 assembly is fine, or is it just disabled anyway on gentoo? (In reply to comment #17) > Oh, by the way: The i586 assembly is fine, or is it just disabled anyway on > gentoo? Here is the gist of the relevant part of the ebuild: _cpu=generic_fpu use altivec && _cpu=altivec if arch == amd64 use sse && _cpu=x86-64 else use mmx && _cpu=mmx use 3dnow && _cpu=3dnow use sse && _cpu=x86 use 3dnowext && _cpu=x86 Later assignments take precedence. configure then gets a --with-cpu=_cpu parameter. So with mmx/sse/3dnow/3dnowext disabled on x86/amd64, the configured cpu is generic_fpu, which disables assembly altogether, if I understood you correctly. Ah, funny selection you have there. So you make mmx and 3dnow exclusive, while sse and 3dnowext result in a build that features all available x86 optimizations. Well, I don't know exactly how USE flags shall be interpreted on gentoo (if mmx code is to be forbidden when the mmx flag is not set, or just not enabled explicitly), so I won't discuss that logic. I can only confirm: generic_fpu won't use any custom assembly routines, so should be PIC-safe. Oh, and just for fun: You could try comparing generic_fpu with i386_fpu ... both use plain C code, just the latter tries to cater more for the limitations of i386. The effect of that has to be analyzed on your specific CPU with your specific compiler:-/ (In reply to comment #17) > As mpg123 upstream I just want to confirm what is apparent here: > > 1. Disabling the offending ASM optimizations on x86 is the way to go. > 2. Rewriting of the routines to make them PIC-friendly won't happen. > > About point 2: I don't see me having the time to invest into that endeavour and > also I fear that it will impact the efficiency of said code. There is hope that > the gains from SIMD computations will dwarf any negative effect, but you don't > know for sure. I'm still trying to make sense out of decoding speed comparisons > with MPlayer's mpg123 binding. Performance on x86 is so flaky, dependent on the > finest details and compiler oddities. > > Hardened gentoo is about playing it safe, and that may be an independent reason > to use the plain C code anyway, as enabling the assembly adds more functionally > redundant code, more target area for attacks. The SSE opt on x86-64 is still a > no-brainer, I think, but you could apply such a rule there, too, of course. > > Oh, by the way: The i586 assembly is fine, or is it just disabled anyway on > gentoo? 1. If 2 won't happen, i need to do some thing and to disable MPROTECT is not the option to do. 2. To make the asm code PIC friendly will take alot of time that i don't have now. We don't have this prob on x86_64 I'm confused, what thing needs to be done here? I though it is settled now that for hardened systems, the optimizations will be disabled. Are you talking about a normal gentoo install? Aren't linker relocations exempt from MPROTECT? (In reply to comment #22) > I'm confused, what thing needs to be done here? I though it is settled now that > for hardened systems, the optimizations will be disabled. > > Are you talking about a normal gentoo install? Aren't linker relocations exempt > from MPROTECT? yes it is settled now that we disable the optimization. (In reply to comment #23) > yes it is settled now that we disable the optimization. Does this mean doing the same ('paxctl -m') thing as what other packages do... * Messages for package app-office/libreoffice-3.5.0.3: * Fallback PaX marking -m * /usr/lib/libreoffice/program/soffice.bin (In reply to comment #23) > yes it is settled now that we disable the optimization. It is now in stable mpg123 -- bug can be closed? (In reply to comment #25) > (In reply to comment #23) > > yes it is settled now that we disable the optimization. > > It is now in stable mpg123 -- bug can be closed? it only means it's not a problem for hardened anymore as they disable the code causing text relocations, it doesn't mean the bug is solved in any way as it's possible to rewrite the code to not cause textrels in the first place the bug should stay open for long as the workarounds are in tree (In reply to comment #26) > it only means it's not a problem for hardened anymore as they disable the > code causing text relocations It's the standard practice in hardened -- take a look at ffmpeg, libav, or similar packages with included assembly code. > it doesn't mean the bug is solved in any way > as it's possible to rewrite the code to not cause textrels in the first place Upstream made clear it's not going to happen, I think. > the bug should stay open for long as the workarounds are in tree Why not resolve it upstream? Concerning rewriting the code to fix this: The assembly code is fully redundant with the plain C version; it just is a workaround for compilers that cannot be coerced into doing those optimizations themselves. But apart from speed differences, which might be alleviated by future compilers (as it happened for the i586 optimization, apparently --- would need to check an actual i586 CPU), you can consider the plain C code as the rewritten safe version of the assembly code. So, you could consider the bug fixed in that respect. I don't say that upstream mpg123 will never feature textrel-clean assembly, I'm just giving the realistic prospect that there won't be developer time invested in that particular goal. Nevertheless, keeping this bug open doesn't hurt anyone. Perhaps it can be closed properly in 5 years or so;-) I have a non-hardened Gentoo system, so don't mind the optimisations, even if they may be imperfect. Unfortunately on my system when I emerge I get: [ebuild U ] media-sound/mpg123-1.14.4 [1.14.2] USE="alsa ipv6 sdl sse (-3dnow) (-3dnowext) (-altivec) (-coreaudio) -int-quality% -jack (-mmx) -nas -oss -portaudio -pulseaudio" 779 kB In my "make.conf" I have "3dnow", "3dnowext" & "mmx" enabled on my opteron system. The man page for emerge indicates that these USE flags have been "forced, masked or removed". Might the masking of these flags for hardened systems have accidentally leaked into non-hardened systems? How can I get my optimisations back, please? (In reply to comment #29) > I have a non-hardened Gentoo system, so don't mind the optimisations, even > if they may be imperfect. Unfortunately on my system when I emerge I get: > > [ebuild U ] media-sound/mpg123-1.14.4 [1.14.2] USE="alsa ipv6 sdl sse > (-3dnow) (-3dnowext) (-altivec) (-coreaudio) -int-quality% -jack (-mmx) -nas > -oss -portaudio -pulseaudio" 779 kB > > In my "make.conf" I have "3dnow", "3dnowext" & "mmx" enabled on my opteron > system. The man page for emerge indicates that these USE flags have been > "forced, masked or removed". Might the masking of these flags for hardened > systems have accidentally leaked into non-hardened systems? > > How can I get my optimisations back, please? Gah. Very sorry. Re-read the comments above, it answers my queries. Please ignore my post. If I can delete my comment I will. *** Bug 475244 has been marked as a duplicate of this bug. *** @zorry any news? Was this bug reported/fixed upstream in the meantime? Are there patches in other distributions? Oh, this one is still alive? Maybe you can check again the situation, however relevant those 32 bit x86 assembly routines are. Digging in the history of mpg123 I see this NEWS: ------- - Silence test for artsc-config if it is not there. - Make sure -static-libgcc from LDFLAGS gets through libtool, fixing 32 bit Windows builds (depend on libgcc DLL otherwise). - Fix build with non-GNU make by using plain rm -f instead of silly $(RM) in libout123/modules makefile fragment. - Make build work on iOS, including coreaudio backend. - libmpg123: !!!!! LOOK HERE !!!!! -- Finally provide position-independent code for x86 with assembly optimisations.The textrels are gone thanks to Won Kyu Park and Taihei Momma. -- Clarify some license language in files descending from the original MMX optimisation. -- Fix return value overflow check for MPG123_BUFFERFILL. -- Introduced mpg123_getformat2() to enable the FORMAT command for the generic control not stealing MPG123_NEW_FORMAT from the main playback loop. The sequence LOADPAUSED-FORMAT-PAUSE (play) is supposed to work now. -- Enable aarch64 optimisations on *BSD by default, too. You can always override that stupid OS whitelist using --with-optimization, anyway. -- Use of the i486 decoder is now discouraged more prominently, in configure output. - out123: Fix stupid crash with verbose mode and tone generation (print the string if the pointer is non-null, not if it is null). - libout123: More consistent error messages for dynamic and legacy (built-in) modules. Namely, you get a hint how if you choose a different module than the built-in ones for a static libout123. It's been a while … Eh … I stripped the important line: This was 1.25.0: MP3 now patent-free worldwide! This was back in 2017. seems to be fixed, can't see this issue in 1.26.5. if i'm wrong please re-open. |
