Originally seen as "cursor not moving in a cell in Jupyter Notebook". USE flags bisection in https://github.com/jupyter/notebook/issues/7336 Reproducible: Always Steps to Reproduce: 1. Compile with USE=lto 2. -> problem 3. Compile with USE=-lto 4. -> no problem gcc version 13.2.1 20240210 (Gentoo 13.2.1_p20240210 p13)
Jupyter issue description: have a Jupyter notebook, with a cell containing some code ; try to put the cursor anywhere but before the first character, and it fails.
`emerge --info`? and `emerge -pvqO firefox` to see other USE? Have you tried to reproduce with firefox-bin? Afaik (upstream) firefox-bin is built with lto too, albeit it uses clang. Given you mention gcc, I guess that means your firefox is built with USE=-clang too?
I forgot to say "using the mouse", try to put the cursor... ionen yes, that's with USE=-clang ; firefox-bin doesn't have the issue.
emerge --info still?
emerge -pvqO firefox: [ebuild R ] www-client/firefox-125.0.2 USE="X dbus geckodriver gmp-autoupdate hardened jumbo-build lto* openh264 pulseaudio system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx system-png system-webp telemetry wayland -clang -debug -eme-free -hwaccel -jack -libproxy -pgo (-selinux) -sndio (-system-python-libs) (-valgrind) -wifi" L10N="-ach -af -an -ar -ast -az -be -bg -bn -br -bs -ca -ca-valencia -cak -cs -cy -da -de -dsb -el -en-CA -en-GB -eo -es-AR -es-CL -es-ES -es-MX -et -eu -fa -ff -fi -fr -fur -fy -ga -gd -gl -gn -gu -he -hi -hr -hsb -hu -hy -ia -id -is -it -ja -ka -kab -kk -km -kn -ko -lij -lt -lv -mk -mr -ms -my -nb -ne -nl -nn -oc -pa -pl -pt-BR -pt-PT -rm -ro -ru -sc -sco -si -sk -sl -son -sq -sr -sv -szl -ta -te -th -tl -tr -trs -uk -ur -uz -vi -xh -zh-CN -zh-TW" LLVM_SLOT="17"
emerge --info (we'll see how pertinent that info is to the issue at hand): Portage 3.0.63 (python 3.10.14-final-0, default/linux/amd64/17.1, gcc-13, glibc-2.39-r3, 6.9.0-rc4-pouet-00113-g2668e3ae2ef3 x86_64) ================================================================= System uname: Linux-6.9.0-rc4-pouet-00113-g2668e3ae2ef3-x86_64-Intel-R-_Xeon-R-_E-2176M_CPU_@_2.70GHz-with-glibc2.39 Head commit of repository gentoo: 9aabbe8f15073f6a2292fb78a018d33cfef46465 [...] sh bash 5.2_p26-r1 ld GNU ld (Gentoo 2.41 p5) 2.41.0 distcc 3.4 x86_64-pc-linux-gnu [disabled] ccache version 4.9.1 [disabled] app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.2_p26-r1::gentoo dev-build/autoconf: 2.13-r8::gentoo, 2.64-r1::gentoo, 2.69-r5::gentoo, 2.71-r7::gentoo, 2.72-r1::gentoo dev-build/automake: 1.11.6-r3::gentoo, 1.13.4-r2::gentoo, 1.16.5-r2::gentoo dev-build/cmake: 3.29.2::gentoo dev-build/libtool: 2.4.7-r4::gentoo dev-build/make: 4.4.1-r1::gentoo dev-build/meson: 1.4.0-r1::gentoo dev-java/java-config: 2.3.3-r1::gentoo dev-lang/perl: 5.38.2-r2::gentoo dev-lang/python: 2.7.18_p16-r2::gentoo, 3.7.12_p1-r1::gentoo, 3.8.16_p4::gentoo, 3.9.18::gentoo, 3.10.14_p1::gentoo, 3.11.9::gentoo, 3.12.3::gentoo dev-lang/rust: 1.77.1::gentoo dev-lang/rust-bin: 1.77.1::gentoo dev-util/ccache: 4.9.1-r1::gentoo sys-apps/baselayout: 2.15::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.38::gentoo sys-apps/systemd: 254.10::gentoo sys-devel/binutils: 2.32-r1::gentoo, 2.34-r2::gentoo, 2.35.2::gentoo, 2.36.1-r1::gentoo, 2.37_p1-r2::gentoo, 2.38-r2::gentoo, 2.39-r4::gentoo, 2.40-r5::gentoo, 2.41-r5::gentoo, 2.42-r1::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/clang: 11.1.0::gentoo, 12.0.0::gentoo, 13.0.1::gentoo, 14.0.6-r4::gentoo, 15.0.7-r3::gentoo, 16.0.6::gentoo, 17.0.6::gentoo, 18.1.4::gentoo sys-devel/gcc: 5.5.0::toolchain, 6.5.0::toolchain, 7.5.0::toolchain, 8.4.0::gentoo, 9.2.0-r4::gentoo, 9.3.0::gentoo, 10.2.0-r5::gentoo, 11.4.1_p20230622::gentoo, 12.3.1_p20240209::gentoo, 13.2.1_p20240210::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/lld: 13.0.1-r1::gentoo, 14.0.6-r1::gentoo, 15.0.7::gentoo, 16.0.6::gentoo, 17.0.6::gentoo, 18.1.4::gentoo sys-devel/llvm: 11.1.0::gentoo, 12.0.1::gentoo, 13.0.1::gentoo, 14.0.6-r4::gentoo, 15.0.7-r3::gentoo, 16.0.6::gentoo, 17.0.6::gentoo, 18.1.4::gentoo sys-kernel/linux-headers: 6.8-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.39-r3::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: git://github.com/gentoo/gentoo priority: -1000 volatile: True Installed sets: @cJ-machine-pouet ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /var/lib/i2pd/certificates" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe" DISTDIR="/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox network-sandbox-proxy news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" MAKEOPTS="-j13" PKGDIR="/var/cache/binpkgs" PORTAGE_COMPRESS="xz" PORTAGE_COMPRESS_FLAGS="--memory=max --extreme" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="7zip X Xaw3d a52 aac aacplus acl alsa amd64 amr amrenc asf bash-completion build_options_dwarf_compress build_options_symbols=compress bzip2 cairo caps cdda cddb cdio cli colordiff crypt cups dirac dri dv dvdread emacs encode faac fat ffmpeg filecaps flac fortran fuse gdbm gif gpm gtk gtk3 gzip h323 harfbuzz heif iconv icu imlib iproute2 ipv6 jbig jfs jpeg jpeg2k jpegxl latex libkms libsamplerate libtirpc libv4l libv4l2 linguas_en linguas_fr linguas_zh_CN lz4 lzma lzo mad matroska mikmod mjpeg mod modplug mp2 mp3 mp4 multilib musepack ncurses nls nptl nptlonly ntfs ogg opengl openmp openssl opus pam pango pcf pcre pdf png policykit qt qt4 qt5 quicktime readline real reiser4 reiserfs ruby_single_target_ruby3_3 screen sdl seccomp sha512 shout sid slang sndfile speex split-usr ssh ssl stream svg systemd test-rust theora threads tiff timidity tls truetype unicode upcall userlocales v4l v4l2 vcd vim-syntax visibility vorbis vpx vulkan wayland webkit webp win32codecs wma wxwidgets wxwindows x264 x265 xattr xcb xfs xinerama xv xvid zip zlib zstd" ABI_X86="64 32" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 3dnow aes avx avx2 f16c fma3 popcnt sse3 sse4 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="evdev synaptics wacom" KERNEL="linux" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LLVM_TARGETS="AArch64 AMDGPU ARM AVR BPF Hexagon NVPTX WebAssembly X86" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11 pypy pypy3 python2_7 python3_10 python3_12 python3_7 python3_8 python3_9" RUBY_TARGETS="ruby31 ruby32 ruby33" VIDEO_CARDS="intel nouveau i965 iris nvidia vmware" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
I asked on #gentoo-chat if someone could try to reproduce, and a gentle soul juippis got a build with -clang and +lto and did not reproduce the problem. So I will investigate further and report. Devs please don't take action, this is a developing story, I'll try to come up with something as narrowed down as possible :) emerge --info did not capture that I'm using -march=native or ccache for this build, and it might be a problem (I hope ccache is not...).
New build with USE="-clang lto" and plain -O2 CXXFLAGS, no ccache, and no problem. Refining...
(In reply to cJ from comment #8) > New build with USE="-clang lto" and plain -O2 CXXFLAGS, no ccache, and no > problem. Refining... Do I understand this "no problem" part correctly as in, the problem is gone now and source-built Firefox is working correctly? And we can just blame it on ccache issues?
Aha! after 9 firefox compilations, I got to a more useful state: -Os -march=native -> NG -O2-> OK -O2 -march=native -> OK -Os -> NG So it would be the combination of -Os and lto that leads to the problem. Can someone confirm that -Os causes the issue? Then I'd dig deeper.
juippis, not a ccache issue. The problem only occurs with certain build flags.
For the curious, I'm archiving binpkgs and build logs in https://zougloub.eu/firefox-lto-bug/
The bug is located in libxul.so (found by replacing files from a NG binpkg to an OK installation).
Mozilla team, I'm not familiar with the Firefox build process, I'm doing a build and aborting at install, to have files ready for playing around. I first thought about going in and rebuilding libxul.so selectively, and was going to ask if this possible and if so, how, but then I realize that we're using LTO and I'm really scratching my head as to how to minimize this. I'll try to capture the execution while running a scenario that reproduces the bug. I notice it also happens with video progress bars: clicking in the middle of a progress bar doesn't work, it's as if it was completely rewinded.
Oh, I noticed that mouse scrolling through tabs not working is also a symptom: 1. Open a clean firefox session 2. Create about 30 new tabs 3. Put mouse pointer on tabs and mouse scroll 4. Expected result: tabs are scrolling ltr/rtl Actual result: tabs are not moving
Talking with upstream on #build:mozilla.org ; performing bisection on CFLAGS to find exactly which GCC option is causing the bug.
Created attachment 891890 [details] build.log for firefox built with gcc and -Os + lto I can reproduce this issue. Here's the successful build.log.
Currently favouring the strategy which doesn't involve too much human involvement: - Played with gcc (13) optimizer flags, only to realize that when optimizer flags are used on top of -O2 to bring `g{cc,++} -Q --help=optimizers ${,CXX}FLAGS` to be the same as -Os, the issue doesn't occur... so it's something less superficial; digging deeper will take some gcc code inspection; - Started to check older gcc versions ; issue also occurs with gcc 12 (gcc version 12.3.1 20240209 (Gentoo 12.3.1_p20240209 p2)); will continue to see if there's a possible way to find a regression.
(In reply to cJ from comment #18) > [...] > > - Played with gcc (13) optimizer flags, only to realize that when optimizer > flags are used on top of -O2 to bring `g{cc,++} -Q --help=optimizers > ${,CXX}FLAGS` to be the same as -Os, the issue doesn't occur... so it's > something less superficial; digging deeper will take some gcc code > inspection; -On cannot be purely decomposed into -O(n-1) + options, unfortunately. A lot of things do have toggles but not everything.
