| Summary: | www-client/firefox-20.0.0 needs 7 gigabytes to build | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Harris Landgarten <harrisl> |
| Component: | Current packages | Assignee: | Mozilla Gentoo Team <mozilla> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | alexandre.guimaraes, jens, mrueg, Storklerk |
| Priority: | Normal | ||
| Version: | 10.1 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
Only pass -g if --enable-debug is passed
add --disable-debug-symbols to eclass |
||
|
Description
Harris Landgarten
2013-04-06 03:40:31 UTC
(In reply to comment #0) I'm not sure if this is entirely related but firefox 20 has given me some size issues also. Compared to 19, 20 will eat all 3GB of my ram, plus my 1GB(ALL, usually turned off) swap then end with: collect2: error: ld terminated with signal 9 [Killed] Seems as though FF20 has become quite fat =/ Just a thought: do you have pgo enabled by chance? Try disabling it, pgo will double the space needed, I remember the ebuild warning about this.
Running a compile with compcache does not seem to cross the 4.5G barrier, having monitored the build process with
#!/bin/bash
zmax=0 max=0
while sleep 2; do
zram=$(($(cat /sys/block/zram0/mem_used_total)/1024/1024))
ram=$(($(cat /sys/block/zram0/orig_data_size)/1024/1024))
[[ $zram -gt $zmax ]] && zmax=$zram
[[ $ram -gt $max ]] && max=$ram
echo "zram(current/max)=${zram}M/${zmax}M origfs_size(current/max)=${ram}M/${max}M"
done
no addition use flags, Just the defaults
www-client/firefox-20.0:0::gentoo (world)
Description Firefox Web Browser
Homepage http://www.mozilla.com/firefox
Herds mozilla
Use flags
USE
alsa Adds support for media-libs/alsa-lib (Advanced Linux Sound Architecture)
-bindist Disable official Firefox branding (icons, name) which are not binary-redistributable according to upstream.
-custom-cflags Build with user-specified CFLAGS (unsupported)
-custom-optimization Fine-tune custom compiler optimizations, setting this is not recommended.
dbus Enable dbus support for anything that needs it (gpsd, gnomemeeting, etc)
-debug Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see http://www.gentoo.org/proj/en/qa/backtraces.xml
gstreamer Adds support for media-libs/gstreamer (Streaming media)
jit Enable just-in-time compilation for improved performance. May prevent use of some PaX memory protection features in Gentoo Hardened.
libnotify Enable desktop notification support
minimal Prevent sdk and headers from being installed
(-pgo) Add support for profile-guided optimization using gcc-4.5, for faster binaries. This option will double the compile time.
(-selinux) !!internal use only!! Security Enhanced Linux support, this must be set by the selinux profile or breakage will occur
startup-notification Enable application startup event feedback mechanism
-system-jpeg Use the system-wide media-libs/libjpeg-turbo Use system jpeg instead of bundled.
-system-sqlite Use the system-wide dev-db/sqlite installation with secure-delete enabled
-wifi Enable wifi
Also no warning that there was not enough space from the ebuild.
Weird, I can't reproduce the results you get. I've compiled with clang, but I do not think that using gcc should make that big of a difference in space usage. If you feel adventurous, give bug #462608 a try, runs fine here. Once FF compile was done, compcache was at zram(current/max)=1951M/1951M origfs_size(current/max)=4493M/4493M Note, since compcache does not seem to release buffers once they are allocated, you'd have to reset /sys/block/zram0/ for the script above to produce correct values on a new compile. Using "du -s" on your tmpfs or compcache might be an easier way to get some stats during the build. I build in a tmp mounted on tmpfs with the following: tmpfs on /var/tmp/paludis type tmpfs (rw,size=6500M,mode=0777,nr_inodes=1M) The build failed at the end when installing the image and the build directory was completely out of space. I am using gcc-4.7.2 with glibc-2.17 and binutils-2.23.1 When the build failed I remounted it with size=7500M and then the build completed. I have been using this build directory in tmpfs for a least 2 years and have never run out of space even on libreoffice. It is also interesting that the ebuild thought it had enough space. Monsterous RAM requirements from the mozilla browsers during link was a cause for me to switch an older laptop that only got 1GB RAM to chromium.
The ebuild seems to check for diskspace:
# Ensure we have enough disk space to compile
if use pgo || use debug || use test ; then
CHECKREQS_DISK_BUILD="8G"
else
CHECKREQS_DISK_BUILD="4G"
fi
And on my system (with +debug) the ~8GB were needed, but sufficent.
What I find very strange is the difference between seamonkey and firefox:
-rw-r--r-- 1 root root 304767409 Apr 6 10:03 firefox-20.0.tbz2
-rw-r--r-- 1 root root 638497553 Apr 3 19:53 seamonkey-2.17.tbz2
The seamonkey package grows to over 600 MB, because the enormous debug infos for libxul.so get included (and installed) twice:
b8b2bdd2eaadccdd9d971cd4190fab4 /usr/lib64/debug/usr/lib64/firefox/libxul.so.debug
d30274bf63469a8153881766d0440c14 /usr/lib64/debug/usr/lib64/seamonkey/libxul.so.debug
d30274bf63469a8153881766d0440c14 /usr/lib64/debug/usr/lib64/seamonkey-devel/sdk/lib/libxul.so.debug
7d1356fd61807d9afa5bfd2c38b90eee /usr/lib64/firefox/libxul.so
290f6c1fee51e3ea18a3a0c781c1a9bb /usr/lib64/seamonkey/libxul.so
290f6c1fee51e3ea18a3a0c781c1a9bb /usr/lib64/seamonkey-devel/sdk/lib/libxul.so
Could the second, identical copy of these files be dropped?
(Or should I open a separate bug for this?)
(In reply to comment #5) > I build in a tmp mounted on tmpfs with the following: > > tmpfs on /var/tmp/paludis type tmpfs (rw,size=6500M,mode=0777,nr_inodes=1M) > > The build failed at the end when installing the image and the build > directory was completely out of space. I've used gcc-4.7.2 as well. I've rethought your issue. You might benefit from a patch I did for clang support https://462608.bugs.gentoo.org/attachment.cgi?id=344638 In particular these additions to various configure.in's of the src dir: + MOZ_DEBUG_FLAGS= fi ], - MOZ_DEBUG=) + MOZ_DEBUG= + MOZ_DEBUG_FLAGS=) It boils down to that even with the the debug use flag disabled, you'll end up having "-g" in the compile strings. You can check this in the output log of the build. Try copying the patch above to /etc/portage/patches/www-client/firefox-20.0/firefox-20.0-clang-3.3svn.patch and rebuild. In the output log you should not see gcc called with "-g" anymore. This might save you some space. Please report back if this makes a difference for you. Thanks for trying. I tested the patch from 462608 on firefox 19 and the result is amazing -- only 1.7GB is required. We should have at least a part of this patch in the main tree. Created attachment 344854 [details, diff]
Only pass -g if --enable-debug is passed
I will be adding this to the patchset shortly, for those who are wanting it earlier please throw it into /etc/portage/patches/www-client/firefox and compile
Thank You Christian Müller for the patch I will get it pushed upstream.
(In reply to comment #9) > Created attachment 344854 [details, diff] [details, diff] > Only pass -g if --enable-debug is passed > > I will be adding this to the patchset shortly, for those who are wanting it > earlier please throw it into /etc/portage/patches/www-client/firefox and > compile > > Thank You Christian Müller for the patch I will get it pushed upstream. Alright after further talk with a few fellow upstream devs, the patch is being scrap'd. I been under the impression that --enable-debug-symbols was default disable well it is default enabled. I am gonna add a debug-symbols useflag to the eclass to handle this. It will default to disable for gentoo. We will still end up with same results as with the patch. Created attachment 345044 [details, diff]
add --disable-debug-symbols to eclass
I will be landing this within the hour, if anyone has any objections or suggestions please let me know before then.
Thanks for reporting, unless doing a debug build you will now end up with --disable-debug-symbols being passed to configure :) (In reply to comment #10) > (In reply to comment #9) > > Created attachment 344854 [details, diff] [details, diff] [details, diff] > > Only pass -g if --enable-debug is passed > > > > I will be adding this to the patchset shortly, for those who are wanting it > > earlier please throw it into /etc/portage/patches/www-client/firefox and > > compile > > > > Thank You Christian Müller for the patch I will get it pushed upstream. > > Alright after further talk with a few fellow upstream devs, the patch is > being scrap'd. I been under the impression that --enable-debug-symbols was > default disable well it is default enabled. I am gonna add a debug-symbols > useflag to the eclass to handle this. It will default to disable for gentoo. > We will still end up with same results as with the patch. Although this has the same effect as if the patch was applied, it is not a good fix, if it is at all. Imho you obsoleted the wrong attachment, because essentially you have been under the right impression, namely that --disable-debug-symbols should be the default when not giving --enable-debug-symbols to configure. You can conclude this by carefully examining the output of ./configure --help [...] --disable-optimize Disable compiler optimization --enable-optimize=[OPT] Specify compiler optimization flags [OPT=-O] --enable-debug-symbols[=DBG] Enable debugging symbols (using compiler flags DBG) [...] --disable-logging Disable logging facilities [...] This output lists the configure options reverse to the defaults set. So, e.g. when optimize is enabled, help prints the reverse option, i.e. to "--disable-optimize". Hence, if "--enable-debug-symbols" is a default option, like you're lead to believe by whomever one of your "fellow" upstream devs, configure should list "--disable-debug-symbols" to well, disable debug symbols - for me and you as a user of configure. This is not the case "--disable-debug-symbols" is not part of the output of "./configure --help". Essentially this means: - disable-debug-symbols should indeed be the default, as suggested by configure - you should not need to pass this option within mozconfig eclass, if sources weren't buggy - right now, you've patched a bug by using this eclass, this is not good style - if you do this for each bug in mozilla sources, the eclass will grow over time and the fixes do not land upstream (this is why you asked me if it's ok if the original patches to this problem go upstream) - so right now you've patched a bug without a patch Imho mozilla devs should just fix it (tm) - the configure bug is obvious: A default for MOZ_DEBUG_FLAGS is set for the positive case of MOZ_DEBUG in configure. I.e. - if --enable-debug-symbols=DBG is given, MOZ_DEBUG_FLAGS are overwritten with DBG - if just --enable-debug-symbols is passed then MOZ_DEBUG_FLAGS are used - if neither one of --{enable,disable}-debug-symbols is given, then MOZ_DEBUG= is emptied in the original source of configure.ac but emptying MOZ_DEBUG_FLAGS= has simply been forgotten Now you have patched this for gentoo, ok and I do not argue about it "looking" bad. It's a simple and short addition to the eclass, but - what about other distros? - what about other users of mozilla's source? Do they need to learn from scratch each time that "debug-symbols" is apart from all other flags being passed to configure, somewhat special? Do they need to waste lots of cpu cycles too, compiling debug information when they don't want or need it? etc. pp. If this is the gentoo way I might as well relax and just refrain from reporting bugs in the future. It seems not worth the effort, researching a proper fix and watching as it is reverted to an unlucky one for personal preference. Yes, I'm complaining, freedom of information has been in a better state, my impression's we're back to "code owners" or "fellow" mozilla devs here, not OSS. Sorry, simply could not resist the rant.. Greetings If you think, the configure help should not be authoritative to this extent, look at "optimize" in the last comment, also included in the output excerpt, it lists both --disable-optimize --enable-optimize[=OPT] _because_ --enable-optimize=-O2 is the default. Hence it makes sense to list both, either to deactivate or user-overwrite with another -O? flag Again, you will not find that this is the case for debug-symbols, probably because the corresponding variable MOZ_DEBUG is unset by default.. .. and so should be the tightly woven MOZ_DEBUG_FLAGS Of course, you might fix configure's help if you still don't feel for the patch ;-) Either way, things in this regard stay broken, and this is fault is not in gentoo, but in mozilla. (In reply to comment #13) > If this is the gentoo way I might as well relax and just refrain from > reporting bugs in the future. It seems not worth the effort, researching a > proper fix and watching as it is reverted to an unlucky one for personal > preference. Yes, I'm complaining, freedom of information has been in a > better state, my impression's we're back to "code owners" or "fellow" > mozilla devs here, not OSS. Sorry, simply could not resist the rant.. Your opinion and debugging efforts are valued here; however the mozilla herd's job is to maintain the gentoo packages to the best of its ability -- we push bugs and patches upstream so as to keep our differences to a minimum, but we only have so much sway upstream in relation to actual decision making. I would highly recommend that you discuss your points directly with upstream, in mozilla's bugzilla; they would be in a much better position to tell you reasons for why they find the patch unacceptable and/or why they don't see the current situation as a bug. And, of course, we will continue to maintain our version of the "fix". (In reply to comment #15) > (In reply to comment #13) > > If this is the gentoo way I might as well relax and just refrain from > > reporting bugs in the future. It seems not worth the effort, researching a > > proper fix and watching as it is reverted to an unlucky one for personal > > preference. Yes, I'm complaining, freedom of information has been in a > > better state, my impression's we're back to "code owners" or "fellow" > > mozilla devs here, not OSS. Sorry, simply could not resist the rant.. > > > Your opinion and debugging efforts are valued here; however the mozilla > herd's job is to maintain the gentoo packages to the best of its ability -- > we push bugs and patches upstream so as to keep our differences to a > minimum, but we only have so much sway upstream in relation to actual > decision making. > > I would highly recommend that you discuss your points directly with > upstream, in mozilla's bugzilla; they would be in a much better position to > tell you reasons for why they find the patch unacceptable and/or why they > don't see the current situation as a bug. > > And, of course, we will continue to maintain our version of the "fix". Thanks for your reply, I've thought about upstream, but since there has been a story and discussion with Jory prior to this fix, I had to post this here before going any further. Patching the debug symbol thingy was not the only issue. It was part of a series (four) smaller patches that shaped mozilla sources into compiling with clang, which is a gentoo goal. None of these other issues are adressed up until now, though they are in a very similar scope and simple/small enough to maintain in gentoo until ironed out upstream. I've gone to the point to make the patch of bug #462608 "digestable" and agreed with Jory's opinion that they should be split into the single issues they are adressing whereafter they were git diffed and emailed to him, wrt master of mozilla overlay. Security issues and version bumps are yielded to, ok, but it's been a while and since it's forseeable that these issues are not resolved upstream until at least FF22 (I've tested with their trunk), it's kind of disappointing to see them more or less /silently/ rejected here. These are small issues, not worth making (so) much fuzz about, which makes it even more incomprehensible why they are still open. We're not talking about a 50k patch to review.. |
