Created attachment 399746 [details, diff] portability fixes for musl as-is, some programs have issues compiling against linux-headers with musl. examples include iproute2, potentially iptables, and now (for some reason) busybox. The specific error I was receiving with busybox is `redefinition of 'struct in6_addr'`. This fix's approach for libc-compat.h is to change `#if defined(__GLIBC__)` and instead use `#ifndef __KERNEL__` in various locations (among a few other changes in the interest of portability). all patches shamelessly stolen from alpine linux. ;) the attached patch is against hardened-dev musl branch.
There are some serious assumptions in what's being patched there. Is anyone pushing these upstream?
i upgraded from linux-headers-3.16 to linux-headers-3.18-r99 from the overlay. now =dev-lang/python-3.3.5-r1 and =dev-lang/python-2.7.9-r1 do not compile anymore. while =dev-lang/python-2.7.9-r2 seems to be fine. when downgrading to linux-headers-3.16, all dev-lang/python:2.7 and dev-lang/python:3.3 versions do compile. will try to add build.log
Created attachment 399860 [details] build.log of dev-lang/python-3.3.5-r1 with linux-headers-3.18-r99
Created attachment 399862 [details] build.log of dev-lang/python-2.7.9-r1 with linux-headers-3.18-r99
(In reply to tt_1 from comment #2) > when downgrading to linux-headers-3.16, all dev-lang/python:2.7 and > dev-lang/python:3.3 versions do compile. can you try linux-headres-3.18 and linux-headers-3.19 from the tree without any patches and see if this fixes the problem. I'm not at all surprised that those patches caused breakage elsewhere.
the python issue is completely unrelated. it's already fixed in-tree for python 2.7 by removing a gentoo patch that causes it. install the ~ masked python 2.7 to fix (with or without the patched linux headers). i've already submitted a comment on the previous bug to have the same fix applied to 3.3.x. if you're impatient, the patch i think is the one that starts with 61? edit the ebuild to add that patch to the epatch ignore list and python 3.3 will compile just fine. sorry for the lack of specifics, but i'm currently on a windows machine and not by my gentoo install. ...actually, due to a hardware failure in another machine i had to cannibalize that gentoo install to repair it. gotta have my gaming box. ;) if you give me a few hours i can get you more detailed information re:python via virtual machine. Actually, after a few mins of searching bugzilla it looks like '61_all_process_data.patch' is the culprit. it was removed from 2.7 in the ~ masked ebuild, but not the stable ebuild or from 3.3 which has the same issue. Mike Gilbert <floppym@gentoo.org> should (should...) already be aware of the problem. I think.
(In reply to Travis Tilley from comment #6) > the python issue is completely unrelated. it's already fixed in-tree for > python 2.7 by removing a gentoo patch that causes it. install the ~ masked > python 2.7 to fix (with or without the patched linux headers). i've already > submitted a comment on the previous bug to have the same fix applied to > 3.3.x. if you're impatient, the patch i think is the one that starts with > 61? edit the ebuild to add that patch to the epatch ignore list and python > 3.3 will compile just fine. sorry for the lack of specifics, but i'm > currently on a windows machine and not by my gentoo install. > > ...actually, due to a hardware failure in another machine i had to > cannibalize that gentoo install to repair it. gotta have my gaming box. ;) > if you give me a few hours i can get you more detailed information re:python > via virtual machine. > > Actually, after a few mins of searching bugzilla it looks like > '61_all_process_data.patch' is the culprit. it was removed from 2.7 in the ~ > masked ebuild, but not the stable ebuild or from 3.3 which has the same > issue. Mike Gilbert <floppym@gentoo.org> should (should...) already be aware > of the problem. I think. What are the working versions of python so I can test and unmask them on the overlay. So far this is blocking the building of new stage3's.
2.7.9-r2 works, but there isn't yet an ebuild for 3.3.5 that removes the above mentioned patch. unless one was added in the past 24 hours.
side note: some of the regulars in #musl on freenode seem to suggest that the UAPI contract is "a mess" due to glibc being... well, i'd rather not repeat the phrasing used on an official bugzilla. some interesting expletives were also used at the suggestion that defining __MUSL__ in features.h might be helpful in keeping patches simple and clean... ideas were thrown around that don't involve patching the linux headers libc-compat header, but i haven't dug too deep into it because it would mean patching potentially hundreds of applications one at a time and the idea is exhausting. one dev was of the opinion that busybox shouldn't be loading linux/<anything> there and instead netinet/in.h and if it for some reason is including both then it's its own damn fault for having conflicting definitions and it shouldn't be the responsibility of some compat header's defines to prevent the re-definition. i haven't looked more in depth at alternative methods of fixing busybox and other packages effected, but intend to. consider this bug on hold. just a heads up, however: with the current state of the tree, a musl stage3 would not build. too many core packages currently fail (not just python and busybox).
=dev-lang/python-2.7.9-r1 and =dev-lang/python-2.7.9-r2 and =dev-lang/python-3.3.5-r1 with =linux-headers-3.16 compile =dev-lang/python-2.7.9-r1 and =dev-lang/python-3.3.5-r1 with =linux-headers-3.18 fail, =dev-lang/python-2.7.9-r2 and =dev-lang/python-3.3.5-r1 with EPATCH_EXCLUDE="61_all_process_data.patch" compile =dev-lang/python-2.7.9-r1 and =dev-lang/python-3.3.5-r1 with linux-headers-3.18-r99 fail, =dev-lang/python-2.7.9-r2 and =dev-lang/python-3.3.5-r1 with EPATCH_EXCLUDE="61_all_process_data.patch" compile with =linux-headers-3.19 it is all the same as with =linux-headers-3.18. so, dropping 61_all_process_data.patch seems to fix the compile. but no idea what this eventually breaks elsewhere? by the way, linux-headers-3.19-r99 from the overlay fails to compile, due to this Failed Patch: libc-compat.h-fix-some-issues-arising-from-in6.h.patch!
(In reply to tt_1 from comment #10) tt_1 thanks! it give me a good idea what's going on here. Okay this is blocking progress with the stage3's by breaking python and potentially others. I'm masking >sys-kernel/linux-headers-3.16 on all musl profiles until this mess is sorted out. I'm also just going to remove linux-headers-3.19-r99 because its broken and doesn't add anything for testing purposes.
The changes in the linux kernel were introduced by https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit?id=66f1c44887ba4f47d617f8ae21cf8e04e1892bd7
Created attachment 416208 [details, diff] portability fixes 4.3 headers
I can confirm that the patch attached by anarchy (Jory) still applies cleanly to 4.5 headers. I have bumped the ebuild to 4.5 and I successfully recompiled the world set with it (minimal stage3 chroot). All the packages compile fine. Could you please bump the 3.18 ebuild in the overlay with the attached patch? Please note that I also had to "unlock" virtual/os-headers to require "sys-kernel/linux-headers:0", and of course unmask linux-headers with /etc/portage/profile/package.unmask.
Created attachment 429462 [details, diff] Patch from 3.18 to 4.5. Patch to bring linux-headers-3.18 to linux-headers-4.5. Note that I used the 4.5 ebuild from the main tree as base, and then integrated the needed parts. Requires addition of the 4.3 portability patch to $FILESDIR.
(In reply to Fabio Scaccabarozzi from comment #15) > Created attachment 429462 [details, diff] [details, diff] > Patch from 3.18 to 4.5. > > Patch to bring linux-headers-3.18 to linux-headers-4.5. > Note that I used the 4.5 ebuild from the main tree as base, and then > integrated the needed parts. > Requires addition of the 4.3 portability patch to $FILESDIR. i'm going to test these on a full desktop system and if they work, i will either stabilize them or backport 4.3 which is currently stable in the tree.
(In reply to Anthony Basile from comment #16) > (In reply to Fabio Scaccabarozzi from comment #15) > > Created attachment 429462 [details, diff] [details, diff] [details, diff] > > Patch from 3.18 to 4.5. > > > > Patch to bring linux-headers-3.18 to linux-headers-4.5. > > Note that I used the 4.5 ebuild from the main tree as base, and then > > integrated the needed parts. > > Requires addition of the 4.3 portability patch to $FILESDIR. > > i'm going to test these on a full desktop system and if they work, i will > either stabilize them or backport 4.3 which is currently stable in the tree. these patches are breaking strace.
(In reply to Anthony Basile from comment #17) > these patches are breaking strace. Confirmed. I was also using strace but probably compiled it on the older headers version, and used oneshot to merge it. :-/ I spotted the relevant fragment of the patch which might be responsible for this: --- a/include/uapi/linux/kernel.h 2015-11-02 12:36:00.000000000 -0600 +++ b/include/uapi/linux/kernel.h 2015-11-06 20:59:38.595400307 -0600 @@ -1,7 +1,9 @@ #ifndef _UAPI_LINUX_KERNEL_H #define _UAPI_LINUX_KERNEL_H +#ifdef __GLIBC__ #include <linux/sysinfo.h> +#endif /* * 'kernel.h' contains some often-used function prototypes etc I have drafted a patch for strace-4.9 - it merely includes the linux/sysinfo.h header - but that does not apply cleanly to strace-4.11, because the code base is very different. I think we should find a way to fix this fragment of the patch rather than fixing every missing linux/sysinfo.h, otherwise we might end up with a lot of micro-patches for headers all over the tree.
(In reply to Fabio Scaccabarozzi from comment #18) > > I have drafted a patch for strace-4.9 - it merely includes the > linux/sysinfo.h header - but that does not apply cleanly to strace-4.11, > because the code base is very different. > I think we should find a way to fix this fragment of the patch rather than > fixing every missing linux/sysinfo.h, otherwise we might end up with a lot > of micro-patches for headers all over the tree. I've got 4.11 on the overlay. the sysinfo.h issue is fixed in later versions.
Thanks, I confirm it's working.(In reply to Anthony Basile from comment #19) > (In reply to Fabio Scaccabarozzi from comment #18) > > > > > I have drafted a patch for strace-4.9 - it merely includes the > > linux/sysinfo.h header - but that does not apply cleanly to strace-4.11, > > because the code base is very different. > > I think we should find a way to fix this fragment of the patch rather than > > fixing every missing linux/sysinfo.h, otherwise we might end up with a lot > > of micro-patches for headers all over the tree. > > I've got 4.11 on the overlay. the sysinfo.h issue is fixed in later > versions. Thanks, just tested and I confirm it's working.
Dave Miller has recently merged relevant patches [1][2] into his net tree. They will probably make it into linux-4.16. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=c0bace798436bca0fdc221ff61143f1376a9c3de [2]: https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=6926e041a8920c8ec27e4e155efa760aa01551fd
(In reply to Felix Janda from comment #21) > Dave Miller has recently merged relevant patches [1][2] into his net > tree. They will probably make it into linux-4.16. > > [1]: > https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/ > ?id=c0bace798436bca0fdc221ff61143f1376a9c3de > [2]: > https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/ > ?id=6926e041a8920c8ec27e4e155efa760aa01551fd Just wanted to mention that the two patches above were merged in 4.15. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v4.15&id=6926e041a8920c8ec27e4e155efa760aa01551fd https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v4.15&id=c0bace798436bca0fdc221ff61143f1376a9c3de
I wonder if this will help the openiscsi use case... https://github.com/open-iscsi/open-iscsi/pull/75
Closing, we use headers as they are provided by kernel now. No need to keep this around any longer.
