Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 526114

Summary: dev-db/mongodb-2.6.5 - mongo: segmentation fault in void v8::internal::String::WriteToFlat<unsigned short>(v8::internal::String*, unsigned short*, int, int) ()
Product: Gentoo Linux Reporter: Darko Luketic <info>
Component: Current packagesAssignee: Ultrabug <ultrabug>
Status: RESOLVED FIXED    
Severity: normal CC: bugs, proxy-maint
Priority: Normal    
Version: unspecified   
Hardware: AMD64   
OS: Linux   
URL: https://jira.mongodb.org/browse/SERVER-15920
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 540460    
Attachments: mongodb add filter-flags

Description Darko Luketic 2014-10-20 16:36:32 UTC
# mongo
MongoDB shell version: 2.6.5
connecting to: test
2014-10-20T18:32:33.467+0200 mongo got signal 11 (Segmentation fault), stack trace: 
2014-10-20T18:32:33.470+0200 0x7d1499 0x5e4e59 0x7fae9d3a9810 0x9bf6d2 0xa2ff98 0xa2ffd4 0xa3537e 0x9cdb8b 0x9d9248 0x9d9665 0x8782c0 0x99a1d8 0x93719d 0x352d1ee06362 
 mongo(_ZN5mongo15printStackTraceERSo+0x39) [0x7d1499]
 mongo(_Z12quitAbruptlyi+0x119) [0x5e4e59]
 /lib64/libc.so.6(+0x38810) [0x7fae9d3a9810]
 mongo(_ZN2v88internal6String11WriteToFlatItEEvPS1_PT_ii+0x692) [0x9bf6d2]
 mongo(_ZN2v88internal33GenericStringUtf16CharacterStream10FillBufferEjj+0x38) [0xa2ff98]
 mongo(_ZN2v88internal28BufferedUtf16CharacterStream9ReadBlockEv+0x34) [0xa2ffd4]
 mongo(_ZN2v88internal7Scanner10InitializeEPNS0_20Utf16CharacterStreamE+0x5e) [0xa3537e]
 mongo(_ZN2v88internal6Parser9ParseLazyEPNS0_20Utf16CharacterStreamEPNS0_9ZoneScopeE+0x3b) [0x9cdb8b]
 mongo(_ZN2v88internal6Parser9ParseLazyEv+0x158) [0x9d9248]
 mongo(_ZN2v88internal9ParserApi5ParseEPNS0_15CompilationInfoEi+0x205) [0x9d9665]
 mongo(_ZN2v88internal8Compiler11CompileLazyEPNS0_15CompilationInfoE+0xf0) [0x8782c0]
 mongo(_ZN2v88internal10JSFunction11CompileLazyENS0_6HandleIS1_EENS0_18ClearExceptionFlagE+0x1a8) [0x99a1d8]
 mongo(_ZN2v88internal11CallIC_MissENS0_9ArgumentsEPNS0_7IsolateE+0x27d) [0x93719d]
 [0x352d1ee06362]


# emerge --info
Portage 2.2.14 (python 2.7.7-final-0, default/linux/amd64/13.0/no-multilib, gcc-4.9.1, glibc-2.19-r1, 3.17.1-gentoo x86_64)
=================================================================
System uname: Linux-3.17.1-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E3-1245_V2_@_3.40GHz-with-gentoo-2.2
KiB Mem:    16333912 total,  10899168 free
KiB Swap:   16777208 total,  16777208 free
Timestamp of tree: Mon, 20 Oct 2014 15:30:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
app-shells/bash:          4.3_p30
dev-lang/perl:            5.20.1-r1
dev-lang/python:          2.7.7
dev-util/cmake:           3.0.2
dev-util/pkgconfig:       0.28-r2
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.13.1
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.69
sys-devel/automake:       1.13.4, 1.14.1
sys-devel/binutils:       2.24-r3
sys-devel/gcc:            4.9.1
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2-r1
sys-devel/make:           4.1
sys-kernel/linux-headers: 3.17 (virtual/os-headers)
sys-libs/glibc:           2.19-r1
Repositories: gentoo gohost docker
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--quiet-build=y --jobs=4 --autounmask-write"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://de-mirror.org/gentoo/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j8 -l8"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /var/lib/layman/docker"
USE="acl amd64 berkdb bindist bzip2 cli cracklib crypt cxx dri fortran gdbm iconv ipv6 mmx modules ncurses nls nptl openmp pam pcre readline session sse sse2 ssl systemd tcpd threads unicode zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NGINX_MODULES_HTTP="access auth_basic autoindex browser charset empty_gif fastcgi geo gzip limit_conn limit_req map memcached proxy referer rewrite scgi split_clients ssi upstream_ip_hash userid spdy" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="2.7"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC
Comment 1 Jeroen Roovers (RETIRED) gentoo-dev 2014-10-20 16:40:21 UTC
1) Please post your `emerge -vpq dev-db/mongodb' output in a comment.
2) Please attach a full gdb backtrace.
Comment 2 Darko Luketic 2014-10-20 20:43:42 UTC
1)
# emerge -vpq dev-db/mongodb
[ebuild   R   ] dev-db/mongodb-2.6.5  USE="ssl -kerberos -mms-agent -static-libs"

2)
(gdb) r
Starting program: /usr/bin/mongo 
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
MongoDB shell version: 2.6.5
[New Thread 0x7ffff4ce8700 (LWP 3515)]
[New Thread 0x7ffff7e56700 (LWP 3516)]
connecting to: test
[New Thread 0x7ffff44e7700 (LWP 3517)]
[Thread 0x7ffff44e7700 (LWP 3517) exited]

Program received signal SIGSEGV, Segmentation fault.
0x00000000009bf6d2 in void v8::internal::String::WriteToFlat<unsigned short>(v8::internal::String*, unsigned short*, int, int) ()
(gdb) bt
#0  0x00000000009bf6d2 in void v8::internal::String::WriteToFlat<unsigned short>(v8::internal::String*, unsigned short*, int, int) ()
#1  0x0000000000a2ff98 in v8::internal::GenericStringUtf16CharacterStream::FillBuffer(unsigned int, unsigned int) ()
#2  0x0000000000a2ffd4 in v8::internal::BufferedUtf16CharacterStream::ReadBlock() ()
#3  0x0000000000a3537e in v8::internal::Scanner::Initialize(v8::internal::Utf16CharacterStream*) ()
#4  0x00000000009cdb8b in v8::internal::Parser::ParseLazy(v8::internal::Utf16CharacterStream*, v8::internal::ZoneScope*) ()
#5  0x00000000009d9248 in v8::internal::Parser::ParseLazy() ()
#6  0x00000000009d9665 in v8::internal::ParserApi::Parse(v8::internal::CompilationInfo*, int) ()
#7  0x00000000008782c0 in v8::internal::Compiler::CompileLazy(v8::internal::CompilationInfo*) ()
#8  0x000000000099a1d8 in v8::internal::JSFunction::CompileLazy(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::ClearExceptionFlag) ()
#9  0x000000000093719d in v8::internal::CallIC_Miss(v8::internal::Arguments, v8::internal::Isolate*) ()
#10 0x00000190e3006362 in ?? ()
#11 0x00000190e30062c1 in ?? ()
#12 0x00007fffffffd1a0 in ?? ()
#13 0x00007fffffffd1e8 in ?? ()
#14 0x00000190e30261b5 in ?? ()
#15 0x000027a39fb284d1 in ?? ()
#16 0x0000263ebdf62721 in ?? ()
#17 0x00000190e3026121 in ?? ()
#18 0x0000000600000000 in ?? ()
#19 0x0000263ebdf0c6f9 in ?? ()
#20 0x00007fffffffd278 in ?? ()
#21 0x00000190e3049b9b in ?? ()
#22 0x0000263ebdf62721 in ?? ()
#23 0x0000000000000000 in ?? ()
(gdb)
Comment 3 Darko Luketic 2014-10-20 20:47:34 UTC
(gdb) t a a bt

Thread 3 (Thread 0x7ffff7e56700 (LWP 3590)):
#0  0x00007ffff585cafd in nanosleep () from /lib64/libc.so.6
#1  0x00007ffff58861f4 in usleep () from /lib64/libc.so.6
#2  0x00000000009dd4bb in v8::internal::SignalSender::Run() ()
#3  0x00000000009dbd96 in ?? ()
#4  0x00007ffff69bb1da in start_thread () from /lib64/libpthread.so.0
#5  0x00007ffff588cd7d in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7ffff4ce8700 (LWP 3589)):
#0  0x00007ffff69bf44c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000720535 in mongo::DeadlineMonitor<mongo::V8Scope>::deadlineMonitorThread() ()
#2  0x00007ffff74c903a in ?? () from /usr/lib64/libboost_thread.so.1.55.0
#3  0x00007ffff69bb1da in start_thread () from /lib64/libpthread.so.0
#4  0x00007ffff588cd7d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7ffff7fe3780 (LWP 3585)):
#0  0x00000000009bf6d2 in void v8::internal::String::WriteToFlat<unsigned short>(v8::internal::String*, unsigned short*, int, int) ()
#1  0x0000000000a2ff98 in v8::internal::GenericStringUtf16CharacterStream::FillBuffer(unsigned int, unsigned int) ()
#2  0x0000000000a2ffd4 in v8::internal::BufferedUtf16CharacterStream::ReadBlock() ()
#3  0x0000000000a3537e in v8::internal::Scanner::Initialize(v8::internal::Utf16CharacterStream*) ()
#4  0x00000000009cdb8b in v8::internal::Parser::ParseLazy(v8::internal::Utf16CharacterStream*, v8::internal::ZoneScope*) ()
#5  0x00000000009d9248 in v8::internal::Parser::ParseLazy() ()
#6  0x00000000009d9665 in v8::internal::ParserApi::Parse(v8::internal::CompilationInfo*, int) ()
#7  0x00000000008782c0 in v8::internal::Compiler::CompileLazy(v8::internal::CompilationInfo*) ()
#8  0x000000000099a1d8 in v8::internal::JSFunction::CompileLazy(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::ClearExceptionFlag) ()
#9  0x000000000093719d in v8::internal::CallIC_Miss(v8::internal::Arguments, v8::internal::Isolate*) ()
#10 0x00002ff9f8b06362 in ?? ()
#11 0x00002ff9f8b062c1 in ?? ()
#12 0x00007fffffffd1a0 in ?? ()
#13 0x00007fffffffd1e8 in ?? ()
#14 0x00002ff9f8b261b5 in ?? ()
#15 0x0000248e896284d9 in ?? ()
#16 0x0000295e5ff62721 in ?? ()
#17 0x00002ff9f8b26121 in ?? ()
#18 0x0000000600000000 in ?? ()
#19 0x0000295e5ff0c6f9 in ?? ()
#20 0x00007fffffffd278 in ?? ()
#21 0x00002ff9f8b49b9b in ?? ()
#22 0x0000295e5ff62721 in ?? ()
#23 0x0000000000000000 in ?? ()
Comment 4 Darko Luketic 2014-10-20 20:49:27 UTC
I hope I got it right this time

Thread 3 (Thread 0x7ffff7e56700 (LWP 3602)):
#0  0x00007ffff585cafd in nanosleep () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007ffff58861f4 in usleep () from /lib64/libc.so.6
No symbol table info available.
#2  0x00000000009dd4bb in v8::internal::SignalSender::Run() ()
No symbol table info available.
#3  0x00000000009dbd96 in ?? ()
No symbol table info available.
#4  0x00007ffff69bb1da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007ffff588cd7d in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 2 (Thread 0x7ffff4ce8700 (LWP 3601)):
#0  0x00007ffff69bf44c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x0000000000720535 in mongo::DeadlineMonitor<mongo::V8Scope>::deadlineMonitorThread() ()
No symbol table info available.
#2  0x00007ffff74c903a in ?? () from /usr/lib64/libboost_thread.so.1.55.0
No symbol table info available.
#3  0x00007ffff69bb1da in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4  0x00007ffff588cd7d in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 1 (Thread 0x7ffff7fe3780 (LWP 3597)):
#0  0x00000000009bf6d2 in void v8::internal::String::WriteToFlat<unsigned short>(v8::internal::String*, unsigned short*, int, int) ()
No symbol table info available.
#1  0x0000000000a2ff98 in v8::internal::GenericStringUtf16CharacterStream::FillBuffer(unsigned int, unsigned int) ()
No symbol table info available.
#2  0x0000000000a2ffd4 in v8::internal::BufferedUtf16CharacterStream::ReadBlock() ()
No symbol table info available.
#3  0x0000000000a3537e in v8::internal::Scanner::Initialize(v8::internal::Utf16CharacterStream*) ()
No symbol table info available.
#4  0x00000000009cdb8b in v8::internal::Parser::ParseLazy(v8::internal::Utf16CharacterStream*, v8::internal::ZoneScope*) ()
No symbol table info available.
#5  0x00000000009d9248 in v8::internal::Parser::ParseLazy() ()
No symbol table info available.
#6  0x00000000009d9665 in v8::internal::ParserApi::Parse(v8::internal::CompilationInfo*, int) ()
No symbol table info available.
#7  0x00000000008782c0 in v8::internal::Compiler::CompileLazy(v8::internal::CompilationInfo*) ()
No symbol table info available.
#8  0x000000000099a1d8 in v8::internal::JSFunction::CompileLazy(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::ClearExceptionFlag) ()
No symbol table info available.
#9  0x000000000093719d in v8::internal::CallIC_Miss(v8::internal::Arguments, v8::internal::Isolate*) ()
No symbol table info available.
#10 0x0000255eff006362 in ?? ()
No symbol table info available.
#11 0x0000255eff0062c1 in ?? ()
No symbol table info available.
#12 0x00007fffffffd1a0 in ?? ()
No symbol table info available.
#13 0x00007fffffffd1e8 in ?? ()
No symbol table info available.
#14 0x0000255eff0261b5 in ?? ()
No symbol table info available.
#15 0x00002192613284d1 in ?? ()
No symbol table info available.
#16 0x00000ae853f62721 in ?? ()
No symbol table info available.
#17 0x0000255eff026121 in ?? ()
No symbol table info available.
#18 0x0000000600000000 in ?? ()
No symbol table info available.
#19 0x00000ae853f0c6f9 in ?? ()
No symbol table info available.
#20 0x00007fffffffd278 in ?? ()
No symbol table info available.
#21 0x0000255eff049bbb in ?? ()
No symbol table info available.
#22 0x00000ae853f62721 in ?? ()
No symbol table info available.
#23 0x0000000000000000 in ?? ()
No symbol table info available.
Comment 5 Ultrabug gentoo-dev 2014-10-22 08:14:00 UTC
Did you report this upstream on a Jira ticket ? I'm not sure it's Gentoo related yet.
Comment 6 Darko Luketic 2014-10-22 13:37:18 UTC
no I didn't

It works on Archlinux

mongo
MongoDB shell version: 2.6.5
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
> 
bye


https://projects.archlinux.org/svntogit/community.git/tree/trunk?h=packages/mongodb
Comment 7 Ultrabug gentoo-dev 2014-11-05 15:27:44 UTC
We have a running JIRA ticket as per $URL
Comment 8 Ultrabug gentoo-dev 2014-11-12 10:24:21 UTC
Hi,

This bug happens on GRSEC enabled kernels. This is related to this : https://jira.mongodb.org/browse/SERVER-12991

Ebuilds have been revision bumped accordingly, the pax-marking somehow disappeared from the 2.4 to 2.6 ebuilds I'm sorry about that.

Thanks again Darko
Comment 9 Jeroen Roovers (RETIRED) gentoo-dev 2015-01-16 14:36:18 UTC
*** Bug 536760 has been marked as a duplicate of this bug. ***
Comment 10 Darko Luketic 2015-01-16 14:40:31 UTC
it's not fixed tho
Comment 11 Darko Luketic 2015-01-19 08:59:28 UTC
So what will it be?
Both bug reports are closed however the problem still remains,
even on a non-GRSEC-enabled kernel.

Continue here?
Continue on MongoDB's bugtracker?
Comment 12 Ultrabug gentoo-dev 2015-01-19 11:55:16 UTC
(In reply to Darko Luketic from comment #11)
> So what will it be?
> Both bug reports are closed however the problem still remains,
> even on a non-GRSEC-enabled kernel.
> 
> Continue here?
> Continue on MongoDB's bugtracker?

Clearly MongoDB Jira please, it's upstream related.
Comment 13 Darko Luketic 2015-01-22 13:18:20 UTC
https://jira.mongodb.org/browse/SERVER-16985?focusedCommentId=810454&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-810454

It's happening because of -march=native
or the equivalent of it, set because of distcc.
Actually -march=sandybridge or -march=ivybridge, not tried with -march=generic.
Plain "-O3 -pipe" works

There's still a bug with the ebuild and that is that distcc isn't used and in turn job setting of -j24 leads to pretty high load.

If there was a way to use a different job number setting for builds not using distcc, that would be great, but I guess that's a portage feature.

Manually building mongodb with --cc=distcc --cxx=distcc resulted in a build error.

Perhaps the ebuild should ignore CFLAGS & CXXFLAGS and use the provided.
Comment 14 Ultrabug gentoo-dev 2015-02-23 11:56:35 UTC
(In reply to Darko Luketic from comment #13)
> https://jira.mongodb.org/browse/SERVER-
> 16985?focusedCommentId=810454&page=com.atlassian.jira.plugin.system.
> issuetabpanels:comment-tabpanel#comment-810454
> 
> It's happening because of -march=native
> or the equivalent of it, set because of distcc.
> Actually -march=sandybridge or -march=ivybridge, not tried with
> -march=generic.
> Plain "-O3 -pipe" works
> 
> There's still a bug with the ebuild and that is that distcc isn't used and
> in turn job setting of -j24 leads to pretty high load.
> 
> If there was a way to use a different job number setting for builds not
> using distcc, that would be great, but I guess that's a portage feature.
> 
> Manually building mongodb with --cc=distcc --cxx=distcc resulted in a build
> error.
> 
> Perhaps the ebuild should ignore CFLAGS & CXXFLAGS and use the provided.

I'm not so sure I follow your idea, if you want to propose a patch to the ebuild, be my guest mate.
Comment 15 Darko Luketic 2015-03-01 12:37:45 UTC
(In reply to Ultrabug from comment #14)
> (In reply to Darko Luketic from comment #13)
> > https://jira.mongodb.org/browse/SERVER-
> > 16985?focusedCommentId=810454&page=com.atlassian.jira.plugin.system.
> > issuetabpanels:comment-tabpanel#comment-810454
> > 
> > It's happening because of -march=native
> > or the equivalent of it, set because of distcc.
> > Actually -march=sandybridge or -march=ivybridge, not tried with
> > -march=generic.
> > Plain "-O3 -pipe" works
> > 
> > There's still a bug with the ebuild and that is that distcc isn't used and
> > in turn job setting of -j24 leads to pretty high load.
> > 
> > If there was a way to use a different job number setting for builds not
> > using distcc, that would be great, but I guess that's a portage feature.
> > 
> > Manually building mongodb with --cc=distcc --cxx=distcc resulted in a build
> > error.
> > 
> > Perhaps the ebuild should ignore CFLAGS & CXXFLAGS and use the provided.
> 
> I'm not so sure I follow your idea, if you want to propose a patch to the
> ebuild, be my guest mate.

Hey Ultra,
Sorry for the late reply. Tooth issues, resolved and recovering ;) also restricted to mobile (slow and laggy).

What I mean is,
1. distcc
If you build with distcc, it won't be used.
So imagine you have 3 distcc hosts with 8 logical CPUs, you set -j24
but because distcc isn't used in this build (and actually leads to the build failing if being used) mongodb is compiled with -j24 instead of -j8, even if you have -l8 set, which is ignored or at least not being followed.

But imho that's a portage feature issue. When a build can't use distcc auto-reduce -j setting to ... "MAX_JOBS" value, which would be defined in make.conf, or perhaps someone has a better idea

2. -march= or CFLAGS/CXXFLAGS
There's a patch that is applied.
mongodb-2.6.2-fix-scons.patch

-    env.Append( CXXFLAGS=["-Wnon-virtual-dtor", "-Woverloaded-virtual"] )
+    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )

amongst other things it passes CXXFLAGS.

Andrew from MongoDB wrote:
"Avoid using optimization levels or codegen options (like -march=native) that result in the use of vectorized operations in v8."
as 1 of 3 options to deal with this "-march=native" issue.

Also that boost should be <1.57.

It isn't clear why mongodb-2.6.2-fix-scons.patch was introduced
If anything remove the
+    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )
line from mongodb-2.6.2-fix-scons.patch
so CXXFLAGS won't be passed to the mongodb build

What do you think?
Comment 16 Ultrabug gentoo-dev 2015-03-02 09:26:07 UTC
(In reply to Darko Luketic from comment #15)
> (In reply to Ultrabug from comment #14)
> What I mean is,
> 1. distcc
> If you build with distcc, it won't be used.
> So imagine you have 3 distcc hosts with 8 logical CPUs, you set -j24
> but because distcc isn't used in this build (and actually leads to the build
> failing if being used) mongodb is compiled with -j24 instead of -j8, even if
> you have -l8 set, which is ignored or at least not being followed.
> 
> But imho that's a portage feature issue. When a build can't use distcc
> auto-reduce -j setting to ... "MAX_JOBS" value, which would be defined in
> make.conf, or perhaps someone has a better idea
> 

Being part of the cluster herd too, AFAIK we have nobody really taking care of distcc and tbh I have no time/interest for this matter so I'm sorry I give any real insight here.

> 2. -march= or CFLAGS/CXXFLAGS
> There's a patch that is applied.
> mongodb-2.6.2-fix-scons.patch
> 
> -    env.Append( CXXFLAGS=["-Wnon-virtual-dtor", "-Woverloaded-virtual"] )
> +    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )
> 
> amongst other things it passes CXXFLAGS.
> 
> Andrew from MongoDB wrote:
> "Avoid using optimization levels or codegen options (like -march=native)
> that result in the use of vectorized operations in v8."
> as 1 of 3 options to deal with this "-march=native" issue.
> 
> Also that boost should be <1.57.
> 

Can you link me to this please ?

> It isn't clear why mongodb-2.6.2-fix-scons.patch was introduced
> If anything remove the
> +    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )
> line from mongodb-2.6.2-fix-scons.patch
> so CXXFLAGS won't be passed to the mongodb build
> 
> What do you think?

It's been introduced because one of the QA policies of Gentoo is to respect users' flags.

You can see it's been there for a long time now : mongodb-2.0-fix-scons.patch

Maybe we could bargain by filtering some known problematic flags while respecting the others tho ?
Comment 17 Darko Luketic 2015-03-03 09:42:58 UTC
(In reply to Ultrabug from comment #16)
> (In reply to Darko Luketic from comment #15)
> > (In reply to Ultrabug from comment #14)
> > What I mean is,
> > 1. distcc
> > If you build with distcc, it won't be used.
> > So imagine you have 3 distcc hosts with 8 logical CPUs, you set -j24
> > but because distcc isn't used in this build (and actually leads to the build
> > failing if being used) mongodb is compiled with -j24 instead of -j8, even if
> > you have -l8 set, which is ignored or at least not being followed.
> > 
> > But imho that's a portage feature issue. When a build can't use distcc
> > auto-reduce -j setting to ... "MAX_JOBS" value, which would be defined in
> > make.conf, or perhaps someone has a better idea
> > 
> 
> Being part of the cluster herd too, AFAIK we have nobody really taking care
> of distcc and tbh I have no time/interest for this matter so I'm sorry I
> give any real insight here.
> 
> > 2. -march= or CFLAGS/CXXFLAGS
> > There's a patch that is applied.
> > mongodb-2.6.2-fix-scons.patch
> > 
> > -    env.Append( CXXFLAGS=["-Wnon-virtual-dtor", "-Woverloaded-virtual"] )
> > +    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )
> > 
> > amongst other things it passes CXXFLAGS.
> > 
> > Andrew from MongoDB wrote:
> > "Avoid using optimization levels or codegen options (like -march=native)
> > that result in the use of vectorized operations in v8."
> > as 1 of 3 options to deal with this "-march=native" issue.
> > 
> > Also that boost should be <1.57.
> > 
> 
> Can you link me to this please ?

See Comment 13 please.

https://jira.mongodb.org/browse/SERVER-16985?focusedCommentId=812066&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-812066

I misread, he wrote that GCC-4.9 & boost-1.57 have issues, but he doesn't mention which specific GCC version.

> 
> > It isn't clear why mongodb-2.6.2-fix-scons.patch was introduced
> > If anything remove the
> > +    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )
> > line from mongodb-2.6.2-fix-scons.patch
> > so CXXFLAGS won't be passed to the mongodb build
> > 
> > What do you think?
> 
> It's been introduced because one of the QA policies of Gentoo is to respect
> users' flags.
> 
> You can see it's been there for a long time now : mongodb-2.0-fix-scons.patch
> 
> Maybe we could bargain by filtering some known problematic flags while
> respecting the others tho ?

Well if we know that -march &| -mtune and possibly also -O have issues with v8 resulting in errors... personally I would remove custom CXXFLAGS alltogether from the patch. What good is a QA policy when it collides with real world use? If every package was the same we wouldn't need ebuilds for each individual package, therefore QA policy can't apply to every package. Real world is more imporant than policy.

I would add a notice that CXXFLAGS are ignored with reference to this bug,
and modify the mongodb-2.6.2-fix-scons.patch (or rather create a new one), remove
-    env.Append( CXXFLAGS=["-Wnon-virtual-dtor", "-Woverloaded-virtual"] )
+    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )

If you'd like to go through the trouble of filtering -march -mtune -O 
...
- split CXXFLAGS string into flagsarray
- for each flag in flagsarray 
   check if begins with -m or -O "^-[mO]+"
- if it does continue else add to new array
- in the end join array elements into string
- apply fix-scons.patch, since no change is needed

but if your CXXFLAGS looks like mine...

CFLAGS="-O3 -pipe -march=ivybridge -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -maes -mno-sha -mpclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=ivybridge -fstack-protector-strong --param=ssp-buffer-size=4"

that would actually work :) but needs to be tested
Comment 18 Ultrabug gentoo-dev 2015-03-03 15:18:27 UTC
(In reply to Darko Luketic from comment #17)
> (In reply to Ultrabug from comment #16)
> > (In reply to Darko Luketic from comment #15)
> > > (In reply to Ultrabug from comment #14)
> > > What I mean is,
> > > 1. distcc
> > > If you build with distcc, it won't be used.
> > > So imagine you have 3 distcc hosts with 8 logical CPUs, you set -j24
> > > but because distcc isn't used in this build (and actually leads to the build
> > > failing if being used) mongodb is compiled with -j24 instead of -j8, even if
> > > you have -l8 set, which is ignored or at least not being followed.
> > > 
> > > But imho that's a portage feature issue. When a build can't use distcc
> > > auto-reduce -j setting to ... "MAX_JOBS" value, which would be defined in
> > > make.conf, or perhaps someone has a better idea
> > > 
> > 
> > Being part of the cluster herd too, AFAIK we have nobody really taking care
> > of distcc and tbh I have no time/interest for this matter so I'm sorry I
> > give any real insight here.
> > 
> > > 2. -march= or CFLAGS/CXXFLAGS
> > > There's a patch that is applied.
> > > mongodb-2.6.2-fix-scons.patch
> > > 
> > > -    env.Append( CXXFLAGS=["-Wnon-virtual-dtor", "-Woverloaded-virtual"] )
> > > +    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )
> > > 
> > > amongst other things it passes CXXFLAGS.
> > > 
> > > Andrew from MongoDB wrote:
> > > "Avoid using optimization levels or codegen options (like -march=native)
> > > that result in the use of vectorized operations in v8."
> > > as 1 of 3 options to deal with this "-march=native" issue.
> > > 
> > > Also that boost should be <1.57.
> > > 
> > 
> > Can you link me to this please ?
> 
> See Comment 13 please.
> 
> https://jira.mongodb.org/browse/SERVER-
> 16985?focusedCommentId=812066&page=com.atlassian.jira.plugin.system.
> issuetabpanels:comment-tabpanel#comment-812066
> 
> I misread, he wrote that GCC-4.9 & boost-1.57 have issues, but he doesn't
> mention which specific GCC version.
> 
> > 
> > > It isn't clear why mongodb-2.6.2-fix-scons.patch was introduced
> > > If anything remove the
> > > +    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )
> > > line from mongodb-2.6.2-fix-scons.patch
> > > so CXXFLAGS won't be passed to the mongodb build
> > > 
> > > What do you think?
> > 
> > It's been introduced because one of the QA policies of Gentoo is to respect
> > users' flags.
> > 
> > You can see it's been there for a long time now : mongodb-2.0-fix-scons.patch
> > 
> > Maybe we could bargain by filtering some known problematic flags while
> > respecting the others tho ?
> 
> Well if we know that -march &| -mtune and possibly also -O have issues with
> v8 resulting in errors... personally I would remove custom CXXFLAGS
> alltogether from the patch. What good is a QA policy when it collides with
> real world use? If every package was the same we wouldn't need ebuilds for
> each individual package, therefore QA policy can't apply to every package.
> Real world is more imporant than policy.
> 
> I would add a notice that CXXFLAGS are ignored with reference to this bug,
> and modify the mongodb-2.6.2-fix-scons.patch (or rather create a new one),
> remove
> -    env.Append( CXXFLAGS=["-Wnon-virtual-dtor", "-Woverloaded-virtual"] )
> +    env.Append( CXXFLAGS=os.environ['CXXFLAGS'] )
> 
> If you'd like to go through the trouble of filtering -march -mtune -O 
> ...
> - split CXXFLAGS string into flagsarray
> - for each flag in flagsarray 
>    check if begins with -m or -O "^-[mO]+"
> - if it does continue else add to new array
> - in the end join array elements into string
> - apply fix-scons.patch, since no change is needed
> 
> but if your CXXFLAGS looks like mine...
> 
> CFLAGS="-O3 -pipe -march=ivybridge -mmmx -mno-3dnow -msse -msse2 -msse3
> -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -maes -mno-sha -mpclmul -mpopcnt
> -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm
> -mavx -mno-avx2 -msse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mrdrnd
> -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt
> -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1
> --param l1-cache-size=32 --param l1-cache-line-size=64 --param
> l2-cache-size=8192 -mtune=ivybridge -fstack-protector-strong
> --param=ssp-buffer-size=4"
> 
> that would actually work :) but needs to be tested

Please check out bug #536688 mate.

I agree we can filter out problematic flags and just so you know : there is an eclass flag-o-matic for this, it's easy and we can still respect CXXFLAGS imo !
Comment 19 Darko Luketic 2015-03-08 14:04:50 UTC
Created attachment 398394 [details, diff]
mongodb add filter-flags
Comment 20 Ultrabug gentoo-dev 2015-03-12 18:04:57 UTC
(In reply to Darko Luketic from comment #19)
> Created attachment 398394 [details, diff] [details, diff]
> mongodb add filter-flags

implemented wrt bug #536688
Comment 21 Darko Luketic 2015-03-12 19:26:25 UTC
(In reply to Ultrabug from comment #20)
> (In reply to Darko Luketic from comment #19)
> > Created attachment 398394 [details, diff] [details, diff] [details, diff]
> > mongodb add filter-flags
> 
> implemented wrt bug #536688

Thanks m8. Where can I find your overlay or is it synced tomorrow?
Sorry for my ignorance :)
Comment 22 Ultrabug gentoo-dev 2015-03-13 08:55:04 UTC
(In reply to Darko Luketic from comment #21)
> (In reply to Ultrabug from comment #20)
> > (In reply to Darko Luketic from comment #19)
> > > Created attachment 398394 [details, diff] [details, diff] [details, diff] [details, diff]
> > > mongodb add filter-flags
> > 
> > implemented wrt bug #536688
> 
> Thanks m8. Where can I find your overlay or is it synced tomorrow?
> Sorry for my ignorance :)

Use layman : layman -a ultrabug