Mozilla Firefox does not build against the musl libc. Firefox is quite portable, seeing as the same codebase is used to make builds for Linux (GNU and Android), Windows, OS X, and various BSDs (unofficially). It should not be very difficult to get it to the point where upstream builds against musl just fine. Most problems are due to the assumption that non-Android Linux is GNU and has glibc.
Created attachment 391092 [details, diff] Experimental patchset against =firefox-34.0.5-r1 A quick and dirty patchset to enable Firefox to build successfully against musl. This needs more work before it can even end up in Portage, not to mention upstream, but might be good enough to include in hardened-development::musl as a -r99.
Created attachment 391094 [details] stab.h file required to build Firefox This file was taken from http://git.alpinelinux.org/cgit/aports/tree/main/firefox/stab.h. My only modification was the addition of #define N_UNDF 0x00.
Created attachment 391096 [details, diff] firefox-34.0.5-r1.ebuild modifications This patch modifies the ebuild to: 1. Apply the patch 2. Copy the stab.h file 3. Disable jemalloc Unfortunately, mozjemalloc does not build against musl. Disabling it was easier than trying to get it to work, but it would be good to get it to work eventually.
It seems that for building firefox 34 also patches for (at least) app-text/hunspell, dev-libs/nss, dev-libs/nspr and media-libs/mesa are necessary. For nss and nspr just a version bump in the overlay is enough. For hunspell there is http://git.alpinelinux.org/cgit/aports/tree/main/hunspell/fix-includes.patch and for mesa http://git.alpinelinux.org/cgit/aports/tree/main/mesa/musl-fixes.patch .
(In reply to Felix Janda from comment #4) > It seems that for building firefox 34 also patches for (at least) > app-text/hunspell, dev-libs/nss, dev-libs/nspr and media-libs/mesa are > necessary. For nss and nspr just a version bump in the overlay is enough. > For hunspell there is > http://git.alpinelinux.org/cgit/aports/tree/main/hunspell/fix-includes.patch > and for mesa > http://git.alpinelinux.org/cgit/aports/tree/main/mesa/musl-fixes.patch . =app-text/hunspell-1.3.2-r3 and =dev-libs/nss-3.17.3 build against =sys-libs/musl-1.1.5 just fine. =dev-libs/nspr-4.10.7 indeed builds fine with the patch from the overlay. Alpine's mesa patch isn't enough to build =media-libs/mesa-10.3.4. I will submit the correct patch in a separate bug.
Comments: 1) Make this into one patch created using `git format-patch` against HEAD of hardened-dev::musl overlay. 2) The patch against the firefox codebase: Did you get it somewhere? If so then add kudos. Also, it seems to lack intelligence. Eg -#if defined(__GLIBC__) +#if 1 I know musl doesn't announce itself, but there is usually some configure script way of testing for the glibc features that are being protected there. I can add it as is, but it cannot go upstream as it is. It would be good to get it upstream because other people will want this. 3) epatch "${FILESDIR}"/musl.patch. You need a better name ofr the patch. If the patch is addressing several issues then break it up and name the patch after the issue being addressed. You can append -musl as well. Eg. fix-included-headers-musl.patch or something like that. I know it might be a pain, but take your work seriously and lets get it upstream.
Created attachment 391232 [details, diff] Additional patch necessary to build on amd64
The part of the patch for mozilla-release/media/webrtc/signaling/src/sipcc/cpr/include/cpr_threads.h does not work for amd64. It should be 'unsigned long' instead of 'unsigned'. See also: https://bugzilla.mozilla.org/show_bug.cgi?id=1010194
Following the IRC discussion with Felix about <sys/cdefs.h> (which doesn't exist on musl), here is what I found: 1. mozilla-release/media/libstagefright/system/core/include/cutils/properties.h #includes <sys/cdefs.h>. 2. The Windows port of libstagefright has a blank cdefs.h file in mozilla-release/media/libstagefright/ports/win32/include/sys/cdefs.h which gets included on Windows builds only. 3. All other instances of #include <sys/cdefs.h> are successfully skipped on Windows, through checking for not-Windows or for BSD before the #include. 4. Deleting the #include line from (1) enables a successful Linux build if <sys/cdefs.h> doesn't exist. 5. The only other place where #include <sys/cdefs.h> is done indiscriminately is mozilla-release/netwerk/sctp. This directory is a copy of sctp-refimpl <https://code.google.com/p/sctp-refimpl/>. Mozilla sync from upstream often, so upstream is the place where the fix for this could go. Seeing as this is supposed to be the portable reference implementation of the RFC, there shouldn't be too much trouble fixing this upstream and letting the change trickle into Mozilla when they sync. My patch already removes the #include, so it's possible to build sctp without it. I will write a better patch and submit it to sctp-refimpl. 6. Similarly, the correct place to submit the libstagefright fix is the stagefright upstream. I'm not sure how often Mozilla sync their copy, and whether the upstream will be receptive to such fixes, seeing as it's an Android project and Android does provide a cdefs.h file. The alternative would be to fix the Firefox build system to avoid building libstagefright except on Android, as Android seems to be the only place where the library is currently used. That said, the stagefright project does seem to have spent some time on portability, as evidenced by the existence of ports for various platforms. I will write a patch and submit it to stagefright. 7. Any remaining #includes of cdefs.h should be proceeded by either a not-Windows check or a BSD check. This is all excluding mozjemalloc, which I'm still keeping disabled. I will include the two new patches in my split patch (which should be ready tomorrow), pending their acceptance upstream. We have also discussed the patch for mozilla-release/media/webrtc/signaling/src/sipcc/cpr/include/cpr_threads.h. unsigned long might not be the most appropriate type for handleInt, although it will enable successful builds on amd64 and will work at runtime. pthread_t would be more appropriate here, but that could impact builds on Darwin and Windows. I will update my patch to use unsigned long for now, as this seems to have a greater chance of being accepted upstream, at least at this point.
Created attachment 391296 [details, diff] One more patch for amd64
Created attachment 391318 [details, diff] Split patch against hardened-dev::musl This patch adds www-client/firefox-34.0.5-r99 to hardened-dev::musl. Includes all patches in this bug so far, including the amd64 fixes from Felix.
okay its on the overlay. Enjoy! Reopen if there's more work to be done here.
Created attachment 399446 [details, diff] Update to version 36.0.1 There is some upstreaming process going on. Two patches are already fixed upstream and most of the others are waiting for review. The patches have links to the relevant upstream bugs.
Reopened @Felix's request.
I recently tried to emerge firefox, but the llvm ebuild in the hardened-development overlay cannot be installed due to * Digest verification failed: * /var/lib/layman/hardened-development/sys-devel/llvm/llvm-3.5.0-r99.ebuild * Reason: Filesize does not match recorded size * Got: 15202 * Expected: 15143
Created attachment 400540 [details, diff] Update to version 36.0.4
Created attachment 400542 [details, diff] Fix llvm Manifest
Created attachment 400718 [details, diff] Update to version 37.0.1 Now also with a workaround for musl ld not finding libmozalloc.so. (I think it is related to fact that musl ld does not support lazy binding.) @blueness: Could you commit the llvm fix and this to the overlay?
(In reply to Felix Janda from comment #18) > Created attachment 400718 [details, diff] [details, diff] > Update to version 37.0.1 > > Now also with a workaround for musl ld not finding libmozalloc.so. (I > think it is related to fact that musl ld does not support lazy binding.) > > @blueness: Could you commit the llvm fix and this to the overlay? I thought I did commit the llvm fix to the overlay? Anyhow, I'll commit this right now and point me to the llvm fix and I'll push that too.
Thanks for committing. llvm just needs its Manifest fixed.
(In reply to Felix Janda from comment #20) > Thanks for committing. llvm just needs its Manifest fixed. shoudl be fixed
I doubt there's any chance of upstreaming this stuff, but just in case, I'm switching ths to IN_PROGRESS.
firefox does require >=dev-libs/nspr-4.10.8 , so please fix in the ebuild. the in tree version of nspr does not compile, so in my local overlay I copied =nspr-4.10.7-r99 to =nspr-4.10.8-r99 without any further changes, which compiles fine. still I had to patch =nss-3.17.4 with the fix-cdefs_h.patch from alpine linux. apart from this, =firefox-37.0.1-r99 is compiling.
(In reply to tt_1 from comment #23) > firefox does require >=dev-libs/nspr-4.10.8 , so please fix in the ebuild. done > > the in tree version of nspr does not compile, so in my local overlay I > copied =nspr-4.10.7-r99 to =nspr-4.10.8-r99 without any further changes, > which compiles fine. done > > still I had to patch =nss-3.17.4 with the fix-cdefs_h.patch from alpine > linux. is this in the overlay? do i need to do anything else? > > apart from this, =firefox-37.0.1-r99 is compiling.
@blueness: 4 of the patches are already upstream and there are bug reports for many of the others. For nss-3.17.4-r99, just take nss-3.17.4.ebuild and files/nss-3.17.1-gentoo-fixups.patch from the tree and add nss-3.16-musl.patch.
Created attachment 400890 [details, diff] nss-3.17.4-r99.patch tried to create a patch against the hardened-development::musl branch for the new nss-3.17.4 ebuild. hope it is correct.
@tt_1: Thanks for the patch. It looks fine except that there is a spurious tab at the end, which breaks the Manifest. (Also we trim the KEYWORDS to have only the arch relevant for musl. Compare to the old nss ebuild.)
Created attachment 400904 [details, diff] updated keywords I updated the KEYWORDS. but don't know what you mean with the tab and where it should be? If it is about the nss-3.17.1-gentoo-fixups.patch, this is from tree so haven't changed anything in there.
tt_1: The problem with the tab is gone. However I haven't been precise about the KEYWORDS. They should match the ebuild being patched, but with all archs except for amd64, arm, mips, ppc and x86 removed. (This is the policy in the musl branch but I'm not sure for the reasons.)
Created attachment 400914 [details, diff] patch for the patch
Created attachment 400916 [details] output of emerge --info
but besides that, firefox gives the following output when executed in a terminal user / # firefox XPCOMGlueLoad error for file /usr/lib/firefox/libxul.so: Error loading shared library libmozalloc.so: No such file or directory (needed by /usr/lib/firefox/libxul.so) Couldn't load XPCOM.
Created attachment 400918 [details] my useflaggs
Created attachment 400920 [details] some weired output if I emerge firefox with +system-sqlite, the part complaining about sqlite3 is gone. please tell me if you need anything else which could be usefull here.
@blueness: Please commit https://bugs.gentoo.org/attachment.cgi?id=400904 and https://bugs.gentoo.org/attachment.cgi?id=400914 (as one commit). (In reply to tt_1 from comment #32) > but besides that, firefox gives the following output when executed in a > terminal > > user / # firefox > > XPCOMGlueLoad error for file /usr/lib/firefox/libxul.so: > Error loading shared library libmozalloc.so: No such file or directory > (needed by /usr/lib/firefox/libxul.so) > Couldn't load XPCOM. This is the problem that Comment 18 refers to. To benefit from the workaround you however need to install sys-apps/ldconfig (will go into the musl ebuild at some point). See https://bugs.gentoo.org/show_bug.cgi?id=545006 . Alternatively, add a line with "/usr/lib/firefox" to /etc/ld-musl-x86_64.path . This should also fix your sqlite3 problem. Thanks for reporting. I was not aware that there will be additional libraries in /usr/lib/firefox when less system libraries are used.
(In reply to tt_1 from comment #32) > but besides that, firefox gives the following output when executed in a > terminal > > user / # firefox > > XPCOMGlueLoad error for file /usr/lib/firefox/libxul.so: > Error loading shared library libmozalloc.so: No such file or directory > (needed by /usr/lib/firefox/libxul.so) > Couldn't load XPCOM. To work around this, I start firefox like so: LD_LIBRARY_PATH=/usr/lib/firefox firefox I have not tried Felix's ebuild from #18 yet, but it looks like it should work without needing LD_LIBRARY_PATH.
(In reply to Wiktor W Brodlo from comment #36) .. > I have not tried Felix's ebuild from #18 yet, but it looks like it should > work without needing LD_LIBRARY_PATH. That is the ebuild now in the overlay. If ldconfig is installed it will put /usr/lib/firefox into /etc/ld-musl-*.arch. You can also do that maually. It is a bit less hacky than using LD_LIBRARY_PATH. To add some detail why this is not needed on glibc: firefox loads its libraries in /usr/lib/firefox individually via dlopen with RTLD_LAZY flag. This tells the dynamic linker to load NEEDED libraries only when one of their functions is actually called. However this lazy binding is not supported on musl: http://wiki.musl-libc.org/wiki/Functional_differences_from_glibc#Lazy_bindings For firefox /usr/lib/firefox/libxul.so has a NEEDED entry for libmozalloc.so. When dlopen'ing libxul.so musl's ld will search for libmozalloc.so but will not find it unless /usr/lib/firefox is in its dynamic library search path. With glibc, libxul and libmozalloc will load fine lazily and when all of them are loaded all symbols are defined.
Created attachment 401888 [details, diff] Update to version 37.0.2 Also mozreconf-v3.eclass conveniently detects us as 'old glibc' and disables jemalloc for us. So we can remove that from the ebuild and are at the ebuild from the portage tree except for the patches and the env.d file.
Created attachment 404738 [details, diff] Update to 38.0.1 from 37.0.1 >=dev-libs/nss-3.19 also needs to be added to the overlay.
Created attachment 405144 [details, diff] Version bump to 38.0.5
Created attachment 405146 [details, diff] Patch for nss-3.19.1
the ebuild for firefox is compiling fine and everything seems to be ok. but I do not have any text at all in the browser, neither from the interface, nor from the websites which I typed in blindfooled. I have USE="minimal jemalloc3" for firefox. please do point out what I may provide as helpfull.
Created attachment 411064 [details, diff] Version bump to 40.0.3 Many of the patches have now landed upstream. Now builds and seems to work also on arm. (A new patch was necessary.)
(In reply to Felix Janda from comment #43) > Created attachment 411064 [details, diff] [details, diff] > Version bump to 40.0.3 > > Many of the patches have now landed upstream. > > Now builds and seems to work also on arm. (A new patch was necessary.) This is in the musl overlay now. Thanks for working on this! Are there anymore patches for upstream or are we done here?
There are 7 patches. 2 of them will arrive in one of the next releases. basename.patch has a mozilla bug with a (now incomplete) patch. It should not be difficult to get a complete patch upstream. updater.patch has a mozilla bug but without a patch, yet. I'm not sure how easily it would be accepted. firefox is downstream for the crashreporter. If compilation with musl is fixed upstream https://code.google.com/p/google-breakpad/issues/detail?id=631 it should not be difficult to get a fix in firefox. profiler-gettid.patch works around issues of musl and is not upstreamable. skia.patch is new and can probably make it in some way to upstream, but I haven't thought about it too much yet.
Created attachment 411090 [details] screenshot from firefox As you may see, no menues, no context menu, not possible to see what one is typing. I will make a second attachement with the output from the console.
Created attachment 411092 [details] some weired output it seems as if some symbols could not be found?
@tt_1: Other gtk+ applications have menus? Going back to 37.0.1 fixes the issue? Have you tried starting in safe mode?
Some Alpine linux users seems to be the same problem: http://bugs.alpinelinux.org/issues/4248 I cannot reproduce it myself on amd64 and arm.
Created attachment 411112 [details] output of emerge --info I am on x86, unfortunately. my useflags are USE="dbus gstreamer jemalloc3 minimal system-cairo system-icu system-jpeg system-libvpx system-sqlite -bindist -custom-cflags -custom-optimization -debug -egl -gmp-autoupdate -gstreamer-0 (-hardened) -jit (-neon) (-pgo) -pulseaudio (-selinux) -startup-notification -test -wifi"
Yes, other gtk+ applications do have menus. I tried to use the safe mode, nothing changed. The same problem occured at 37.0.1 or 38.0.5 - when not activating all the system flags, there is not even any html rendering. typing about:config does crash firefox, 38.0.5 as well as 40.0.3 [31508] ###!!! ABORT: Divide by zero: file /var/tmp/portage/www-client/firefox-40.0.3-r99/work/mozilla-release/toolkit/xre/nsSigHandlers.cpp, line 154 Segmentation fault so, this is a duplicate of #4248 from alpine.
@tt_1: Could you open a new bug with all information so far, link to the alpine bug and make it block this one?
Created attachment 415740 [details, diff] firefox-41.0.2 bumb to 41.0.2 dropped 1152176.patch, addedfix-seccomp-bpf.patch
The patch for firefox-41.0.2 seems to work also fine for arm.
(In reply to Felix Janda from comment #54) > The patch for firefox-41.0.2 seems to work also fine for arm. I added this to the overlay. FYI, the patch did not include the metadata.xml which needed updating. I just copied it from the main gentoo repo to the overlay.
Created attachment 416502 [details, diff] firefox-38.4.0-esr with patchset it is working for me, so please test.
Created attachment 416504 [details, diff] firefox-38.4.0-esr with patchset fixed a typo, sry.
Created attachment 416506 [details, diff] firefox-42.0-r99.patch I updated four of the patches in terms of that the paths had to be adjusted, upstream changed the patterns of some of the folders obiously. The libavutil patch is indeed the only change from 41.0.2, it seems to be possible to simply delete sys/sysctl in cpu.c ; alpine is using the same patch as well. But please review this carefully, I am not exactly a professional in such things. The patch assumes that firefox-38.4.0-r99 patch is applied beforehand.
Is in mozilla overlay at moment, will be moved to tree tonight. Please look for all mozilla products to support musl via overlay then moving to tree.
Created attachment 417996 [details] build.log obviously something about crashreporter. please add the available patch, thank you.
(In reply to tt_1 from comment #60) > Created attachment 417996 [details] > build.log > > obviously something about crashreporter. please add the available patch, > thank you. we will not add the patch your build is not using the updated eclass as crashreporter is completely disabled in gentoo due to fact our symbols would not line up with upstreams for the crash report.
(In reply to tt_1 from comment #60) > Created attachment 417996 [details] > build.log > > obviously something about crashreporter. please add the available patch, > thank you. Your error could be caused by ccache which is known to cause problems from time to time.
which eclass? I used the in-tree -r2 and the -r2 from the overlay, both with the same result.
Created attachment 428470 [details, diff] Version 45.0 Here is the patch that bump firefox::musl to 45.0. The ebuild has been updated with changes from ::gentoo. The following patches are not needed any more, so they're gone: basename.patch fix-seccomp-bpf.patch profiler-gettid.patch sandbox-cdefs.patch skia.patch plus a section of crashreporter.patch. No new patches are needed.
(In reply to Wiktor W Brodlo from comment #64) > Created attachment 428470 [details, diff] [details, diff] > Version 45.0 > > Here is the patch that bump firefox::musl to 45.0. The ebuild has been > updated with changes from ::gentoo. The following patches are not needed any > more, so they're gone: > > basename.patch > fix-seccomp-bpf.patch > profiler-gettid.patch > sandbox-cdefs.patch > skia.patch > plus a section of crashreporter.patch. > > No new patches are needed. pushed to overlay.
wrt comment 64: firefox-45.0::gentoo should build also on the musl profiles. Have you checked that the patches are really necessary? The other patches are no longer necessary because they are now included in gentoo's firefox patches. There shouldn't really be a firefox-45.0 in the musl overlay. If there's a problem, it should be fixed in the tree.
(In reply to Felix Janda from comment #66) > wrt comment 64: > firefox-45.0::gentoo should build also on the musl profiles. Have you > checked that the patches are really necessary? > > The other patches are no longer necessary because they are now included > in gentoo's firefox patches. There shouldn't really be a firefox-45.0 in > the musl overlay. If there's a problem, it should be fixed in the tree. i haven't been testing firefox on musl, so i'd be curious too since i'm trying to move as many pkgs back into the tree.
(In reply to Anthony Basile from comment #67) > (In reply to Felix Janda from comment #66) > > wrt comment 64: > > firefox-45.0::gentoo should build also on the musl profiles. Have you > > checked that the patches are really necessary? > > > > The other patches are no longer necessary because they are now included > > in gentoo's firefox patches. There shouldn't really be a firefox-45.0 in > > the musl overlay. If there's a problem, it should be fixed in the tree. > > i haven't been testing firefox on musl, so i'd be curious too since i'm > trying to move as many pkgs back into the tree. Anarchy's been using firefox on musl at least since firefox-44. If the in-tree ebuild doesn't suffice for musl support please let me know what additional patches are needed to support it. I would also appreciate knowing if any additional patches are needed for firefox-46.0_beta's on mozilla-overlay.
(In reply to Felix Janda from comment #66) > wrt comment 64: > firefox-45.0::gentoo should build also on the musl profiles. Have you > checked that the patches are really necessary? > > The other patches are no longer necessary because they are now included > in gentoo's firefox patches. There shouldn't really be a firefox-45.0 in > the musl overlay. If there's a problem, it should be fixed in the tree. I made the wrong assumption that firefox still needs patches without checking. Mea maxima culpa! Yes, firefox-45.0::gentoo builds and works fine on amd64.
firefox::mozilla and firefox::gentoo do build successfully on a musl profile. but there are various crashes and segfaults at runtime, most of them are connected to a bug in mesa. see https://bugs.gentoo.org/show_bug.cgi?id=576928
I had to take www-client/firefox off the overlay for QA reasons, the overlay version had missing dependencies. we either have to update it and i'll put it back on, else we have to get the fixes in the main tree.
(In reply to Anthony Basile from comment #71) > we either have to update it and i'll put it back on I don't think that's needed. The version in the main tree seems to be fine now.
(In reply to Wiktor W Brodlo from comment #72) > (In reply to Anthony Basile from comment #71) > > we either have to update it and i'll put it back on > > I don't think that's needed. The version in the main tree seems to be fine > now. okay then, we're done here! reopen if other issues come up with firefox.
Created attachment 433576 [details] something is wrong with stackwalk For musl-amd64, the in tree firefox builds fine. x86 is affected by a build failure, please see the log for further informations.
Created attachment 433578 [details, diff] patch from alpine This patch is from alpine and it allows to compile, but I am not sure how dirty it is and what the consequences may be.
Created attachment 434346 [details] the missing part of the patches I just tried to compile a debug build of firefox-38.8.0 on musl-x86 to verify if it is affected by gentoo bug #559788 or not. As thunderbird-38.x had been stabiliszed beforehand it shouldn't be much of a deal, but there are a few things which have to be fixed! first of all, the in-tree ebuild needs a bump to firefox-38.0-patches-05.tar.xz! this allows a normal build without compile errors. But due to lazy bindings I assume, /usr/bin/firefox cannot find it's libraries although they are present in /usr/lib/firefox. Thunderbird and recent Firefox suffered from the same problem, hence a maintainer added something like append-ldflags -Wl,-rpath="${MOZILLA_FIVE_HOME}" So please edit someone the ebuild accordingly, it is not a big deal. All of this is unsufficent however if it is a debug build. In this case there is some work needed for the 9011-musl-fix-netwerk.patch from the tarball, as for an unknown reason the original patch had been adjusted, or to be more precise, a part of it is missing. I have isolated the missing part, it can be found in the attached archive as firefox-38.8.0-sctp.patch Also 9013-musl-queue.patch had been edited, with the result of another compile failure. Again I have isolated the missing part, it can be found in the attached archive as firefox-40.0.3-debug-queue-h.patch
The last version of in tree support is firefox-53.0 as of firefox-54.0 rust is hard required and musl support is not available.