Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 528200 - media-libs/x265 - /usr/lib64/libx265.a has executable stack
Summary: media-libs/x265 - /usr/lib64/libx265.a has executable stack
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal with 4 votes (vote)
Assignee: media-video herd
URL:
Whiteboard:
Keywords: PATCH
Depends on: 666486
Blocks:
  Show dependency tree
 
Reported: 2014-11-04 03:29 UTC by Andrew John Hughes
Modified: 2018-10-17 12:20 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Mark assembly files with no executable stack. (x265-execstack.patch,7.99 KB, patch)
2014-11-04 03:36 UTC, Andrew John Hughes
Details | Diff
x265-1.4-noEXEstack.patch (x265-1.4-noEXEstack.patch,8.01 KB, patch)
2015-02-13 13:10 UTC, Paolo Pedroni
Details | Diff
x265-1.5-noEXEstack.patch (x265-1.5-noEXEstack.patch,8.02 KB, patch)
2015-03-18 10:09 UTC, Paolo Pedroni
Details | Diff
x265-1.7-noEXEstack.patch (x265-1.7-noEXEstack.patch,8.14 KB, patch)
2015-05-20 08:11 UTC, Paolo Pedroni
Details | Diff
modified patch to be clean on some systems (x265-1.7-noEXEstack_dw.patch,7.18 KB, patch)
2015-05-20 22:16 UTC, Attila Tóth
Details | Diff
x265-1.8-noEXEstack.patch (x265-1.8-noEXEstack.patch,8.23 KB, patch)
2015-10-12 08:32 UTC, Paolo Pedroni
Details | Diff
x265-1.8-noEXEstack.patch modified version (x265-1.8-noEXEstack.patch,8.92 KB, patch)
2015-10-14 12:38 UTC, Attila Tóth
Details | Diff
note-GNU-stack fix (x265-note-GNU-stack.patch,526 bytes, patch)
2015-10-14 19:58 UTC, Magnus Granberg
Details | Diff
x265-1.9-noEXEstack.patch (x265-1.9-noEXEstack.patch,9.81 KB, patch)
2016-02-01 14:24 UTC, Paolo Pedroni
Details | Diff
x265-2.0-noEXEstack.patch (x265-2.0-noEXEstack.patch,865 bytes, patch)
2016-08-01 12:50 UTC, Paolo Pedroni
Details | Diff
x265-2.5-noEXEstack.patch (x265-2.5-noEXEstack.patch,485 bytes, patch)
2017-07-17 17:53 UTC, Attila Tóth
Details | Diff
Backported fix from x265 2.7 (x265-2.6-elf.patch,1.25 KB, patch)
2018-02-25 22:14 UTC, gen2dev
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew John Hughes 2014-11-04 03:29:27 UTC
* QA Notice: The following files contain writable and executable sections
 *  Files with such sections will not work properly (or at all!) on some
 *  architectures/operating systems.  A bug should be filed at
 *  http://bugs.gentoo.org/ to make sure the issue is fixed.
 *  For more information, see http://hardened.gentoo.org/gnu-stack.xml
 *  Please include the following list of files in your report:
 *  Note: Bugs should be filed for the respective maintainers
 *  of the package in question and not hardened@g.o.
 * !WX --- --- usr/lib64/libx265.a:const-a.asm.o
 * !WX --- --- usr/lib64/libx265.a:cpu-a.asm.o
 * !WX --- --- usr/lib64/libx265.a:ssd-a.asm.o
 * !WX --- --- usr/lib64/libx265.a:mc-a.asm.o
 * !WX --- --- usr/lib64/libx265.a:mc-a2.asm.o
 * !WX --- --- usr/lib64/libx265.a:pixel-util8.asm.o
 * !WX --- --- usr/lib64/libx265.a:blockcopy8.asm.o
 * !WX --- --- usr/lib64/libx265.a:pixeladd8.asm.o
 * !WX --- --- usr/lib64/libx265.a:dct8.asm.o
 * !WX --- --- usr/lib64/libx265.a:sad16-a.asm.o
 * !WX --- --- usr/lib64/libx265.a:intrapred16.asm.o
 * !WX --- --- usr/lib64/libx265.a:ipfilter16.asm.o
 * !WX --- --- usr/lib32/libx265.a:const-a.asm.o
 * !WX --- --- usr/lib32/libx265.a:cpu-a.asm.o
 * !WX --- --- usr/lib32/libx265.a:ssd-a.asm.o
 * !WX --- --- usr/lib32/libx265.a:mc-a.asm.o
 * !WX --- --- usr/lib32/libx265.a:mc-a2.asm.o
 * !WX --- --- usr/lib32/libx265.a:pixel-util8.asm.o
 * !WX --- --- usr/lib32/libx265.a:blockcopy8.asm.o
 * !WX --- --- usr/lib32/libx265.a:pixeladd8.asm.o
 * !WX --- --- usr/lib32/libx265.a:dct8.asm.o
 * !WX --- --- usr/lib32/libx265.a:sad16-a.asm.o
 * !WX --- --- usr/lib32/libx265.a:intrapred16.asm.o
 * !WX --- --- usr/lib32/libx265.a:ipfilter16.asm.o
 * !WX --- --- usr/lib32/libx265.a:pixel-32.asm.o


Reproducible: Always
Comment 1 Andrew John Hughes 2014-11-04 03:36:37 UTC
Created attachment 388496 [details, diff]
Mark assembly files with no executable stack.
Comment 2 Samuli Suominen gentoo-dev 2014-11-09 09:12:34 UTC
Any chance you could post this patch to upstream?
Comment 3 Paolo Pedroni 2015-02-13 13:10:57 UTC
Created attachment 396360 [details, diff]
x265-1.4-noEXEstack.patch

The original patch did not apply cleanly to x265-1.4. This version is rebased against that version.
Comment 4 Paolo Pedroni 2015-03-18 10:09:44 UTC
Created attachment 399182 [details, diff]
x265-1.5-noEXEstack.patch

Patch rebased against x265-1.5
Comment 5 Alexis Ballier gentoo-dev 2015-03-18 12:02:00 UTC
(In reply to Samuli Suominen from comment #2)
> Any chance you could post this patch to upstream?

please do; as far as I am concerned, I will not apply this patch until upstream merges it
Comment 6 Attila Tóth 2015-04-22 21:13:59 UTC
(In reply to Paolo Pedroni from comment #4)
> Created attachment 399182 [details, diff] [details, diff]
> x265-1.5-noEXEstack.patch
> 
> Patch rebased against x265-1.5

Seems to do the trick for me! Thanks!
Comment 7 Paolo Pedroni 2015-05-20 08:11:09 UTC
Created attachment 403650 [details, diff]
x265-1.7-noEXEstack.patch

Patch rebased against x265-1.7
Comment 8 Attila Tóth 2015-05-20 11:38:04 UTC
(In reply to Paolo Pedroni from comment #7)
> Created attachment 403650 [details, diff] [details, diff]
> x265-1.7-noEXEstack.patch
> 
> Patch rebased against x265-1.7

Thank you so much!
Any chance to push this upstream?
Comment 9 Paolo Pedroni 2015-05-20 12:01:26 UTC
(In reply to Attila Tóth from comment #8)
> (In reply to Paolo Pedroni from comment #7)
> > Created attachment 403650 [details, diff] [details, diff] [details, diff]
> > x265-1.7-noEXEstack.patch
> > 
> > Patch rebased against x265-1.7
> 
> Thank you so much!
> Any chance to push this upstream?

I have no idea if the original poster of the patch (Andrew John Hughes, see comment #1) is taking care of that.
Comment 10 Andrew John Hughes 2015-05-20 22:08:02 UTC
I haven't had the time to look into upstreaming it.
Comment 11 Attila Tóth 2015-05-20 22:15:54 UTC
(In reply to Paolo Pedroni from comment #7)
> Created attachment 403650 [details, diff] [details, diff]
> x265-1.7-noEXEstack.patch
> 
> Patch rebased against x265-1.7

The patch did not applied cleanly for me. I attach a version, which was modified a little bit to be clean on my systems.
Comment 12 Attila Tóth 2015-05-20 22:16:45 UTC
Created attachment 403700 [details, diff]
modified patch to be clean on some systems
Comment 13 Paolo Pedroni 2015-10-12 08:32:09 UTC
Created attachment 414412 [details, diff]
x265-1.8-noEXEstack.patch

Patch for x265-1.8
Comment 14 Attila Tóth 2015-10-14 12:36:44 UTC
(In reply to Paolo Pedroni from comment #13)
> Created attachment 414412 [details, diff] [details, diff]
> x265-1.8-noEXEstack.patch
> 
> Patch for x265-1.8

Thanks!
Comment 15 Attila Tóth 2015-10-14 12:38:59 UTC
Created attachment 414556 [details, diff]
x265-1.8-noEXEstack.patch modified version

I've made some cleanup and added a chunk to handle loopfilter.asm as well.
Comment 16 Attila Tóth 2015-10-14 12:41:21 UTC
(In reply to Alexis Ballier from comment #5)
> (In reply to Samuli Suominen from comment #2)
> > Any chance you could post this patch to upstream?
> 
> please do; as far as I am concerned, I will not apply this patch until
> upstream merges it

Isn't it possible to include the patch in a way making it conditional to either pic or hardened USE flags?
Comment 17 Magnus Granberg gentoo-dev 2015-10-14 19:58:52 UTC
Created attachment 414576 [details, diff]
note-GNU-stack fix

cleaner note-GNU-stack patch
Comment 18 Paolo Pedroni 2016-02-01 14:24:07 UTC
Created attachment 424412 [details, diff]
x265-1.9-noEXEstack.patch

Patch for x265-1.9
Comment 19 Attila Tóth 2016-02-03 21:58:31 UTC
(In reply to Paolo Pedroni from comment #18)
> Created attachment 424412 [details, diff] [details, diff]
> x265-1.9-noEXEstack.patch
> 
> Patch for x265-1.9

Works for me! Thanks again: Dw.
Comment 20 Paolo Pedroni 2016-08-01 12:50:06 UTC
Created attachment 442190 [details, diff]
x265-2.0-noEXEstack.patch

This is all that's really needed to fix this problem in 2.0 (it probably works in 1.8 and 1.9 as well).
Comment 21 Attila Tóth 2016-08-01 14:30:50 UTC
(In reply to Paolo Pedroni from comment #20)
> Created attachment 442190 [details, diff] [details, diff]
> x265-2.0-noEXEstack.patch
> 
> This is all that's really needed to fix this problem in 2.0 (it probably
> works in 1.8 and 1.9 as well).

Works for me (portage user patch mechanism), thx again!
Comment 22 Agostino Sarubbo gentoo-dev 2016-12-31 22:14:36 UTC
did someone forward the patch upstream?
Comment 23 Anthony Ryan 2017-01-04 01:14:55 UTC
Since nobody else has done it, I've opened a PR upstream here: https://bitbucket.org/multicoreware/x265/pull-requests/30/ensure-x86-asm-is-marked-nowrite-noexec-on/diff
Comment 24 Attila Tóth 2017-03-16 00:40:53 UTC
(In reply to Paolo Pedroni from comment #20)
> Created attachment 442190 [details, diff] [details, diff]
> x265-2.0-noEXEstack.patch
> 
> This is all that's really needed to fix this problem in 2.0 (it probably
> works in 1.8 and 1.9 as well).

Patch works for x265-2.3 as well. I used the user patch mechanism, simple. Thx!
Comment 25 Attila Tóth 2017-07-17 17:53:28 UTC
Created attachment 485360 [details, diff]
x265-2.5-noEXEstack.patch

Former patch for x265-2.0 ported to x265-2.5.
Comment 26 gen2dev 2018-02-25 22:14:07 UTC
Created attachment 521050 [details, diff]
Backported fix from x265 2.7

x265-2.5-noEXEstack.patch also applies to x265 2.6 cleanly. However it's only partially fixing the upstream bug, which is that the assembler defines either "elf32" or "elf64" but the x86 assembly source is still using "elf". If I'm reading it right the unfixed part causes some symbols that should be hidden inside the library to be visible.

Upstream fixed the bug in their 2.7 release in a slightly different way, in the same commit where they changed from the YASM assembler to NASM. here:
  https://bitbucket.org/multicoreware/x265/commits/9eabffb26dd62e4f48c5679594ae13690eb9d221

Here's a backport of that upstream 2.7 fix to 2.6. It is only the part that fixes the "elf" issue, without the other changes in the same file in that 2.7 commit.