CXX portage/keywords.o portage/keywords.cc: In static member function ‘static bool Keywords::modify_keywords(std::string&, const std::string&, const std::string&)’: portage/keywords.cc:128: error: no matching function for call to ‘find(__gnu_cxx::__normal_iterator<std::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, __gnu_cxx::__normal_iterator<std::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)’ portage/keywords.cc:135: error: no matching function for call to ‘find(__gnu_cxx::__normal_iterator<std::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, __gnu_cxx::__normal_iterator<std::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)’ make[2]: *** [portage/keywords.o] Error 1 make[2]: *** Waiting for unfinished jobs.... make[2]: Leaving directory `/Library/Gentoo/var/tmp/portage/app-portage/eix-0.18.3/work/eix-0.18.3/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/Library/Gentoo/var/tmp/portage/app-portage/eix-0.18.3/work/eix-0.18.3' make: *** [all] Error 2 * ERROR: app-portage/eix-0.18.3 failed: * emake failed If you need anything, please say so, I can't make anything out of this message. Portage 2.2.00.14813-prefix (!/Library/Gentoo/usr/portage/profiles/prefix/sunos/solaris/5.10/x86, gcc-4.4.2, unavailable, 5.10 i86pc) ================================================================= System Settings ================================================================= System uname: Solaris-2.10-i86pc-i386-32bit-ELF Timestamp of tree: Sun, 15 Nov 2009 10:10:56 +0000 app-shells/bash: 4.0_p35 dev-java/java-config: 1.3.7-r1, 2.1.9-r1 dev-lang/python: 2.4.6, 2.5.4-r3, 2.6.4 dev-python/pycrypto: 2.0.1-r8 dev-util/cmake: 2.8.0 sys-devel/autoconf: 2.13, 2.63-r01.1 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2-r00.1, 1.11 sys-devel/binutils: 2.20.51.0.2 sys-devel/gcc-config: 1.4.1-r00.2 sys-devel/libtool: 2.2.6a-r00.2 ACCEPT_KEYWORDS="~x86-solaris" CBUILD="i386-pc-solaris2.10" CFLAGS="-march=core2 -O3 -pipe" CHOST="i386-pc-solaris2.10" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="" DISTDIR="/Library/Gentoo/usr/portage/distfiles" FEATURES="assume-digests collision-protect distlocks fixpackages news nostrip preserve-libs protect-owned sfperms strict unmerge-logs unmerge-orphans userfetch" GENTOO_MIRRORS="ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" LC_ALL="en_GB.UTF-8" LDFLAGS="" MAKEOPTS="-j5" PKGDIR="/Library/Gentoo/usr/portage/packages" PORTAGE_CONFIGROOT="/Library/Gentoo/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/Library/Gentoo/var/tmp" PORTDIR="/nfs/amun/export/scratch0/gentoo/prefix-overlay-rsync/rsync1" PORTDIR_OVERLAY="/export/gentoo/prefix-tree" SYNC="null://dontuse" USE="X berkdb cracklib ipv6 modules ncurses nls prefix readline ssl unicode x86-solaris zlib" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="SunOS" INPUT_DEVICES="keyboard mouse" KERNEL="SunOS" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
let it go to the prefix herd instead, so we can track
additional information: 0.18.2 did still compile
Created attachment 210307 [details, diff] Add missing #include <algorithm> I guess, an #include <algorithm> may be missing if UNIQUE_WORKS is false - the attached patch should fix this. However, the only explanation I have for the changed behavior is that UNIQUE_WORKS has changed from eix-0.18.2 to eix-0.18.3. Please tell me: Does the ./configure output in eix-0.18.3 really contain: checking whether std::unique seems to work... no while in eix-0.18.2 it contained checking whether std::unique seems to work... yes Then the former is the actual problem - would be nice if you could find out which error configure.log contains when running the test program... As a side note: In eix-0.19.0 the whole include-philosophy was changed; instead on relying on such side effects for include's (with such unrelated errors) now everything needed is included. Since I cannot test on sun or other machines, I expect that I made several mistakes in eix-0.19.0, so I would be grateful for bug reports - once these have been fixed, these include problems should have vanished also for future eix releases.
0.18.3 config.log: configure:21705: checking whether std::unique seems to work configure:21777: i386-pc-solaris2.10-g++ -o conftest -fomit-frame-pointer - frename-registers -fstrict-aliasing -fmerge-all-constants -funsafe-loop- optimizations -finline-functions -ffast-math -fgcse-sm -fgcse-las -fgcse-after- reload -fpredictive-commoning -ftree-switch-conversion -fno-ident - fvisibility=hidden -fvisibility-inlines-hidden -fno-enforce-eh-specs -DNDEBUG - DNO_DEBUG -DG_DISABLE_ASSERT -I/Library/Gentoo/usr/include -Wl,-O1 -Wl,--relax -Wl,--hash-style=gnu -Wl,--enable-new-dtags -Wl,--as-needed -Wl,--sort-common - Wl,-z,combreloc conftest.cpp >&5 configure:21781: $? = 0 configure:21787: ./conftest terminate called after throwing an instance of 'St9bad_alloc' what(): std::bad_alloc ./configure: line 21789: 23766 Abort ./conftest$ac_exeext configure:21791: $? = 134 configure: program exited with status 134 configure: failed program was: [snip] configure:21802: result: no so std::unique indeed is reported not to work configure for 0.18.2 indeed reports: checking whether std::unique seems to work... yes
I applied your patch and it makes eix 0.18.3 compile fine now. Do you want me to apply it to the ebuild, or will you release a new version of eix soon?
(In reply to comment #5) > I applied your patch and it makes eix 0.18.3 compile fine now. Unfortunately, it only disguises the actual problem which is that even the simple test program does not run with the added CXXFLAGS/LDFLAGS: I did not expect that they break a compiler's own STL - this is of course a bug in the compiler, and it means that other breakage with them is to be expected. I think the proper solution is to mask USE=optimization and/or USE=security, depending on which one causes the wrong behavior of the std::unique test.
Well, I noticed eix injects some lunatic flags both to compiler and linker, which really break common rules in Gentoo land, which really make me wonder why they are enabled by default in the ebuild. But that's a different topic. Dropping all your LDFLAGS (which look highly unhealthy since this is Solaris, not Linux) but -O1 just results in: checking whether std::unique seems to work... yes Something like --hash-style=gnu for instance isn't going to be a good idea on a non-GNU ld.so/kernel, IMO. Dropping it, actually got me still this: checking whether std::unique seems to work... yes But please be conservative and careful, checking whether the linker accepts it doesn't mean it makes sense. I think some of the other opts are better not used on non-Linux/non-glibc as well.
(In reply to comment #7) > Well, I noticed eix injects some lunatic flags both to compiler and linker, > which really break common rules in Gentoo land I know: It was an experiment, but had some reasons, about which I had a lengthy discussion. Short summary: The admissible CXXFLAGS depend on the code, and the eix code should be written in such a way that it should work with these flags (if not, I would like to receive bugs to know about it and fix the code correspondingly - cleanly written code should have no problems with these flags IMHO. Moreover, if not the project maintainer, who else should ever decide that the code is such that certain flags should be safe to use for a project? Just letting ricers do it by guessing is not good IMHO, and since the flags exist, I think they should also be used if they make sense: For example, I took care to not do type puning, so why shouldn't the user be able to benefit from it with -fstrict-aliasing?) That's why I suggested for eix-0.18.3 to enable it by default; I didn't expect that even the compiler/linker itself wouldn't like them, since I thought I was very careful with testing: The flags are even dropped if only a warning (not even an error) shows up when using them for linking > Something like --hash-style=gnu for instance isn't going to be a good idea > on a non-GNU ld.so/kernel, IMO. Yes, I would have expected that the linker complains if it cannot link with a basic library; seems I do not only have to link but actually run a small test program, at least for the LDFLAGS to get runtime-link errors. I will do this and moreover shift --hash-style=gnu and --enable-new-dtags (which is unneeded on gentoo anyway, I know) to "strong-optimization". Probably it is also better to drop the default enabling of "optimization" and "security" again, although especially about "security" I am unsure whether it is really a good idea to not have it by default: I had found some buffer overflows which are probably harder to exploit with -fstack-protector (of course even better with PAX, but unfortunately this cannot be achieved by only adding some flags), so I would feel better if users would at least have these "hardened" flags...
Well, let's be clear: about the warnings, there's nothing wrong with that, although a bit useless for users perhaps, but that doesn't hurt anything. In this case it showed you carefully chose the CXXFLAGS, since that was not the problem at all. I doubt whether your linking/running trick is going to work, since it only showed to be a problem in a very specific case, and in general the linking went fine.
(In reply to comment #9) > although a bit useless for users perhaps I admit that this is a strange sort of perfectionism, but it gave me always a bad feeling in the stomach to know that things are not optimal - even if potential improvement in speed or memory are almost negligible. When I was playing around with autotools anyway, I just couldn't resist... > I doubt whether your linking/running trick is going to work, since it only > showed to be a problem in a very specific case, and in general the linking > went fine. That's what I really do not understand: If sun's libraries cannot be linked to a --hash-style=gnu binary, how can something runnable be produced at all? Perhaps different libraries are linked when this flag is used and perhaps in these libraries std::unique is really broken? After all, std::bad_alloc does really not sound as if a dynamic library is linked incorrectly, and the small test program uses besides std::unique nothing special which isn't used by eix anyway... If really std::unique is the only problem (which may be possible: apparently, it segfaults in several STL implementations), the problem won't exist in eix-0.19.0, since now different data types (std::set or std::map) are used throughout where the former was necessary.
(In reply to comment #10) > That's what I really do not understand: If sun's libraries cannot be linked > to a --hash-style=gnu binary, how can something runnable be produced at all? > Perhaps different libraries are linked when this flag is used and perhaps > in these libraries std::unique is really broken? > After all, std::bad_alloc does really not sound as if a dynamic library > is linked incorrectly, and the small test program uses besides std::unique > nothing special which isn't used by eix anyway... I can live with the fact that GNU-ld on Solaris performs fine in most cases, especially when not being told to do very tricky business ;) The alternative is to use Sun-ld, which of course *is* supposed to never do anything wrong, but breaks a whole lot of other things because its arguments differ so much. You can blame us for using GNU-ld on that one, but we're not going to change that anywhere near soon. > If really std::unique is the only problem (which may be possible: apparently, > it segfaults in several STL implementations), the problem won't exist in > eix-0.19.0, since now different data types (std::set or std::map) are used > throughout where the former was necessary. May very well be, I still think it's pretty scary to see that stuff as linker flags. Compiler flags I don't care about, but this is about operability with the rest of the system, which is so non-GNU. Maybe haubi has some useful input here. haubi?
(In reply to comment #11) > You can blame us for using GNU-ld on that one I don't want to blame anybody, I just want to understand why --hash-style=gnu can cause a properly working executable which works with many STL functions but only one segfaults. Really, the only explanation I have is that it links to a different lib than without this option - in which case, one of course should ask which is the more desirable lib to link to... > Compiler flags I don't care about, but this is about operability with > the rest of the system, which is so non-GNU. I already thought about introducing yet another LDFLAGS-useflag, but four useflags just for *FLAGS (debug, security, optimization, strong-optimization) are already confusing enough for a user anyway. As mentioned, I have already shifted --hash-style=gnu and --enable-new-dtags to strong-optimization (which is not default-enabled and definitely never will be, and it is documented that this can break things). About the other optimizing flags -O1 --relax --as-needed --sort-common -z,combreloc I do not care much for eix: I just added them since the mechanism to add them existed anyway... However, --as-needed can at most change things if autotools or "pkg-config --libs sqlite3" (if pkg-config exists on sun) should add some strange -l... for libs "just for the case" although they are actually not used in eix, and for the others I cannot imagine that they could do more harm than perhaps interfering with a debugger. Do you really think, I should also shift them to strong-optimization? However, what about the security relevant combination -z,now -z,relro? I would really like to keep them in security and leave that on by default to make exploits somewhat harder: After all, I guess that eix or at least eix-update is often called from root and perhaps not always with a clean environment - to which eix is very sensible. That's why I consider this so important (although by changing environment there are some "documented" ways to exploit eix, but it would prevent some "undocumented" ways like some buffer overflows).
(In reply to comment #12) > (In reply to comment #11) > > You can blame us for using GNU-ld on that one > > I don't want to blame anybody, I just want to understand why --hash-style=gnu > can cause a properly working executable which works with many STL functions > but only one segfaults. Really, the only explanation I have is that it links > to a different lib than without this option - in which case, one of course > should ask which is the more desirable lib to link to... I haven't seen a segfault, just an abort. But the reason why this exactly happens is actually above my knowledge. > About the other optimizing flags > -O1 --relax --as-needed --sort-common -z,combreloc > I do not care much for eix: I just added them since the mechanism to add > them existed anyway... However, --as-needed can at most change things if Sun-ld defaults to GNU-ld --as-needed behaviour for years > autotools or "pkg-config --libs sqlite3" (if pkg-config exists on sun) Gentoo Prefix just provides pkg-config > should add some strange -l... for libs "just for the case" although they are > actually not used in eix, and for the others I cannot imagine that they > could do more harm than perhaps interfering with a debugger. I don't think this is just up to me. I think the general consensus is that this should be left to the users. -O1 is already in the profiles, and --as-needed advertised as good candidate (and yes this thing can clean up a lot!). sorting stuff sounds like something that really is just a (knowledgable) user preference to me too, isn't it? > Do you really think, I should also shift them to strong-optimization? > However, what about the security relevant combination -z,now -z,relro? > I would really like to keep them in security and leave that on by default > to make exploits somewhat harder: After all, I guess that eix or at least > eix-update is often called from root and perhaps not always with a clean > environment - to which eix is very sensible. That's why I consider this so > important (although by changing environment there are some "documented" > ways to exploit eix, but it would prevent some "undocumented" ways like > some buffer overflows). Arguable, though, do other programs (e.g. python) that have the same privileges and are perhaps ran even more (mta) also have them? The program itself should provide the maximum security features possible.
(In reply to comment #13) > I think the general consensus is that this should be left to the users. That's also the case for CFLAGS, but users actually using them are then blamed as "ricers" - to the effect that even useful flags are simply never used in practice. So I think it is reasonable that the projects should add them where they make sense. I think that I even read somewhere that a reason for recommending only very basic flags in /etc/make.conf is that projects who could need them should add them - this is what eix is doing now. I do not see the big difference in CFLAGS and LDFLAGS in this respect. > sorting stuff sounds like something that really is just a > (knowledgable) user preference to me too, isn't it? I would agree if the flag is forced on somebody. But users can just disable the USE-flag and choose their own flags, so nobody is forced if he doesn't agree with the default. It is only the question, what is the saner default if the users specifies nothing. That which costs (minimally) more build time or that which will start the program (minimally) faster if the user specifies nothing. To me, the latter sounds more natural. > Arguable, though, do other programs (e.g. python) that have the same > privileges and are perhaps ran even more (mta) also have them? I don't know and I don't care much: If other projects think it is up to the distribution to add the flags, I do not have to repeat this policy. For binary distribution, expecting detailed knowledge fromthe packager is reasonable, and many binary distributions use these flags for practically every packet. But eix is mainly used on gentoo, and gentoo does not add these flags by defaults. Many users probably even do not know about them or how they are related to security. > The program itself should provide the maximum security features possible. The program itself should not have buffer overflows, to start with. But we do not live in an ideal world, and mistakes happen.
Your flags broke my compilation, which perfectly succeeds without. That's why Gentoo wants flags to be sane, such that any extra flag trickery is the user's fault, not the distro's. We have an insane amount of platforms to take care of, with an even more insane amount of configurations, which are sometimes highly sensitive to any "beyond normal" flags. I had to mask this, go back and forth numerous times, try multiple versions, etc. etc. etc. It's enough. We identified the issue, now it would be great if the next version would work again. Thanks.
I think it is time to end this discussion. Anyway, I would like to clarify a possible misunderstanding, since perhaps this was not clearly emphasized: I am not critizicing Gentoo's general *FLAGS policy; I only care about eix and what are its best defaults.
checking whether CXXFLAGS=-fstack-protector is known... yes it is "known" yes CXXLD eix eixTk/ansicolor.o: In function `AnsiColor::calc_string()': ansicolor.cc:(.text+0x93): undefined reference to `__stack_chk_guard' ansicolor.cc:(.text+0x13a): undefined reference to `__stack_chk_guard' ansicolor.cc:(.text+0x195): undefined reference to `__stack_chk_fail' this 30 times can we PLEASE stop with flag trickery now?
(In reply to comment #11) > The alternative is to use Sun-ld, which of course *is* supposed to never do > anything wrong, but > breaks a whole lot of other things because its arguments differ so much. You > can blame us for using GNU-ld on that one, but we're not going to change that > anywhere near soon. Well, we are (actually mduft is) doing native ld on Solaris now, as IBM Rational Purify doesn't support GNU ld. He got this working so far for the <200 Packages that we (mduft+haubi@work) should release in our stable tree around end of this year. Basically, I don't care compiler flags that much, except to enable additional warnings. For optimization, I do like to use -O2, as this is the most tested setting AFAIK, simply because I don't have time to play around with. Please note: Compiler flags have to be passed when the compiler is used for linking (either sharedlib or executable) too, as the compiler very often (especially with C++) does some intermediate compilation step too (by collect2). There's one thing to say for: > checking whether CXXFLAGS=-fstack-protector is known... yes > ansicolor.cc:(.text+0x93): undefined reference to `__stack_chk_guard' GCC's stack protection needs an additional library (shipped with gcc), and thus *requires* this flag for linking too. And for AIX: While -fstack-protector basically works with gcc, it does break some AIX ABI specification, and thus the libssp.a isn't built with gcc without an extra hack: http://gcc.gnu.org/ml/gcc-help/2007-07/msg00049.html So when a compile-test for some flag works, a link-test might fail. Result: When CC/CXX in use is GNU, turn on *warning* flags as you like (after testing if gcc version in use supports them). But for optimization, please ship with -O2 (autotools default to "-g -O2"). And don't ship with additional development-helper features enabled, just use them during development - they may change a library's ABI. my 2ct
Sorry: That the flags are still added is really a bug in eix-0.19.0 and was not my intention: Due to the previous discussion, the ebuild (of 0.19.0) was modified to *not* add/check any flags by default, and I even changed the defaults in ./configure. Unfortunately, only now I realize that I forgot to erase a line in configure.ac so that --disable-security is treated like --enable-security by mistake. This is of course broken behavior. I am really sorry about that, and I will fix this of course ASAP.
The default options in 0.19.0-r1 work now (I was seeing the same issue on amd64-linux)
