2024-11-15 08:28:35,479 root INFO running build_ext -- Building with NumPy bindings -- Not using cuDNN -- Not using CUDA -- Not using XPU -- Not using MKLDNN -- Using system provided NCCL library at False, False -- Building without distributed package -- Building Executorch -- Not using ITT 2024-11-15 08:28:35,546 root INFO building 'torch._C' extension 2024-11-15 08:28:35,547 root INFO creating /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.12/build/temp.linux-x86_64-cpython-312/torch/csrc 2024-11-15 08:28:35,547 root INFO building 'functorch._C' extension 2024-11-15 08:28:35,547 root INFO x86_64-pc-linux-gnu-gcc -shared -O2 -pipe -march=native -DNDEBUG -L/usr/lib64 -o /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.12/build/lib.linux-x86_64-cpython-312/functorch/_C.cpython-312-x86_64-linux-gnu.so 2024-11-15 08:28:35,549 root INFO x86_64-pc-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -O2 -pipe -march=native -DNDEBUG -fPIC -I/usr/include/python3.12 -c torch/csrc/stub.c -o /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.12/build/temp.linux-x86_64-cpython-312/torch/csrc/stub.o -Wall -Wextra -Wno-strict-overflow -Wno-unused-parameter -Wno-missing-field-initializers -Wno-unknown-pragmas -fno-strict-aliasing -g x86_64-pc-linux-gnu-gcc: fatal error: no input files compilation terminated. 2024-11-15 08:28:35,622 root INFO x86_64-pc-linux-gnu-gcc -shared -O2 -pipe -march=native -DNDEBUG /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.12/build/temp.linux-x86_64-cpython-312/torch/csrc/stub.o -L/var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1/torch/lib -L/usr/lib64 -ltorch_python -o /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.12/build/lib.linux-x86_64-cpython-312/torch/_C.cpython-312-x86_64-linux-gnu.so -g -Wl,-rpath,$ORIGIN/lib error: command '/usr/bin/x86_64-pc-linux-gnu-gcc' failed with exit code 1 * ERROR: sci-libs/pytorch-2.5.1::gentoo failed (compile phase): * Wheel build failed Portage 3.0.66.1 (python 3.12.7-final-0, default/linux/amd64/23.0/split-usr, gcc-14, glibc-2.40-r5, 6.10.13-gentoo-x86_64 x86_64) ================================================================= System uname: Linux-6.10.13-gentoo-x86_64-x86_64-AMD_Ryzen_9_5900X_12-Core_Processor-with-glibc2.40 KiB Mem: 65778420 total, 25679536 free KiB Swap: 20971516 total, 19687676 free Head commit of repository gentoo: 48a24729cbf8c2ccf12d084d140a13ffd74c5b33 sh bash 5.2_p37 ld GNU ld (Gentoo 2.43 p3) 2.43.1 app-misc/pax-utils: 1.3.8::gentoo app-shells/bash: 5.2_p37::gentoo dev-build/autoconf: 2.13-r8::gentoo, 2.72-r1::gentoo dev-build/automake: 1.16.5-r2::gentoo, 1.17-r1::gentoo dev-build/cmake: 3.31.0::gentoo dev-build/libtool: 2.5.3::gentoo dev-build/make: 4.4.1-r100::gentoo dev-build/meson: 1.6.0::gentoo dev-lang/perl: 5.40.0::gentoo dev-lang/python: 3.11.10_p1::gentoo, 3.12.7_p1::gentoo, 3.13.0::gentoo dev-lang/rust: 1.81.0-r100::gentoo, 1.82.0-r100::gentoo sys-apps/baselayout: 2.17::gentoo sys-apps/openrc: 0.55.1::gentoo sys-apps/sandbox: 2.40::gentoo sys-devel/binutils: 2.43-r2::gentoo sys-devel/binutils-config: 5.5.2::gentoo sys-devel/clang: 18.1.8-r6::gentoo, 19.1.3::gentoo sys-devel/gcc: 13.3.1_p20241025::gentoo, 14.2.1_p20241026::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/lld: 18.1.8::gentoo, 19.1.3::gentoo sys-devel/llvm: 18.1.8-r6::gentoo, 19.1.3::gentoo sys-kernel/linux-headers: 6.11::gentoo (virtual/os-headers) sys-libs/glibc: 2.40-r5::gentoo Repositories: gentoo location: /home/me/code/gentoo sync-type: git sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 volatile: True ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=native -flto=8 -fuse-linker-plugin -ggdb3" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -march=native -flto=8 -fuse-linker-plugin -ggdb3" DISTDIR="/chroot/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe -march=native -flto=8 -fuse-linker-plugin -ggdb3" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=native -flto=8 -fuse-linker-plugin -ggdb3" GENTOO_MIRRORS="http://ftp.uni-erlangen.de/pub/mirrors/gentoo" INSTALL_MASK="/etc/X11/xinit/xinitrc.d/80-dbus /usr/bin/dbus-launch" LANG="C.UTF8" LDFLAGS="-O2 -flto=8" LEX="flex" MAKEOPTS="-j24" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" RUSTFLAGS="-C target-cpu=native" SHELL="/bin/bash" USE="X acl alsa amd64 apng bzip2 cet crypt cups dri ffmpeg gdbm glamor gmp gpo iconv icu ipv6 lapack libtirpc lto minizip multilib ncurses nls opencl opengl openmp pam pcre pgo readline seccomp split-usr sqlite ssl system-llvm test-rust truetype udev unicode vaapi vdpau vulkan xattr xvmc zlib" ABI_X86="64 32" ADA_TARGET="gcc_12" AMDGPU_TARGETS="gfx1102" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_11 python3_12" VIDEO_CARDS="radeon r600 radeonsi amdgpu" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, SIZE, STRINGS, STRIP, YACC, YFLAGS
Could you please add the use flags used to build caffe2 ?
Exactly the same symptoms here.
(In reply to Adrien Dessemond from comment #2) > Exactly the same symptoms here. Could you please add the use flags used to build caffe2 ?
Created attachment 912876 [details] emerge --info '=sci-libs/pytorch-2.5.1'
Created attachment 912877 [details] environment
Created attachment 912878 [details] build.log
Created attachment 912879 [details] eclass-debug.log
(In reply to Adrien Dessemond from comment #7) > Created attachment 912879 [details] > eclass-debug.log (In reply to Tupone Alfredo from comment #3) > (In reply to Adrien Dessemond from comment #2) > > Exactly the same symptoms here. > > Could you please add the use flags used to build caffe2 ? Sure, see attached logs.
(In reply to Adrien Dessemond from comment #8) > (In reply to Adrien Dessemond from comment #7) > > Created attachment 912879 [details] > > eclass-debug.log > > (In reply to Tupone Alfredo from comment #3) > > (In reply to Adrien Dessemond from comment #2) > > > Exactly the same symptoms here. > > > > Could you please add the use flags used to build caffe2 ? > > Sure, see attached logs. If you do not have specific flags for caffe2, should be ok, otherwise the output of: emerge -pv caffe2 pytorch could be help to do try rebuilding here Just a note: caffe2 create a cache file to pytorch and put it in /var/lib/caffe2/ IIRC, so pytorch use it
(In reply to Tupone Alfredo from comment #9) > If you do not have specific flags for caffe2, should be ok, > otherwise the output of: > > emerge -pv caffe2 pytorch > > could be help to do try rebuilding here > > Just a note: caffe2 create a cache file to pytorch and put it in > /var/lib/caffe2/ IIRC, so pytorch use it Same error with the following done: 1. rm -rf /var/lib/caffe2 2. emerge -1 caffe2 3. emerge -1 pytorch emerge -pv caffe2 pytorch gives: These are the packages that would be merged, in order: Calculating dependencies... done! Dependency resolution took 6.36 s (backtrack: 0/20). [ebuild R ] sci-libs/caffe2-2.5.1-r2::gentoo USE="numpy opencl openmp -cuda -distributed -fbgemm -flash -gloo -mkl -mpi -nnpack -onednn -openblas -qnnpack -rocm -xnnpack" AMDGPU_TARGETS="gfx90a gfx906 gfx908 gfx942 gfx1030 gfx1100 -gfx803 -gfx900 -gfx940 -gfx941 -gfx1010 -gfx1011 -gfx1012 -gfx1031 -gfx1101 -gfx1102" PYTHON_SINGLE_TARGET="python3_11 -python3_10 -python3_12" 0 KiB [ebuild U ] sci-libs/pytorch-2.5.1::gentoo [2.4.1::gentoo] USE="-debug" PYTHON_SINGLE_TARGET="python3_11 -python3_10 -python3_12" 0 KiB Total: 2 packages (1 upgrade, 1 reinstall), Size of downloads: 0 KiB
Could you try to rebuild, only pytorch, with MAKEOPTS=-j1 Thanks
(In reply to Tupone Alfredo from comment #11) > Could you try to rebuild, only pytorch, with MAKEOPTS=-j1 > > Thanks Sure.I get the same error however: # MAKEOPTS=-j1 emerge -1O pytorch (...) -- Not using ITT 2024-12-04 12:06:43,323 root INFO building 'torch._C' extension 2024-12-04 12:06:43,323 root INFO creating /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.11/build/temp.linux-x86_64-cpython-311/torch/csrc 2024-12-04 12:06:43,323 root INFO x86_64-pc-linux-gnu-gcc -Wsign-compare -DNDEBUG -O3 -pipe -march=znver5 -fomit-frame-pointer -fcf-protection=return -flto=auto -ffat-lto-objects -DNDEBUG -fPIC -I/usr/include/python3.11 -c torch/csrc/stub.c -o /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.11/build/temp.linux-x86_64-cpython-311/torch/csrc/stub.o -Wall -Wextra -Wno-strict-overflow -Wno-unused-parameter -Wno-missing-field-initializers -Wno-unknown-pragmas -fno-strict-aliasing -g 2024-12-04 12:06:43,364 root INFO x86_64-pc-linux-gnu-gcc -shared -flto=auto -O3 -pipe -march=znver5 -fomit-frame-pointer -fcf-protection=return -flto=auto -ffat-lto-objects -DNDEBUG /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.11/build/temp.linux-x86_64-cpython-311/torch/csrc/stub.o -L/var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1/torch/lib -L/usr/lib64 -ltorch_python -o /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.11/build/lib.linux-x86_64-cpython-311/torch/_C.cpython-311-x86_64-linux-gnu.so -g -Wl,-rpath,$ORIGIN/lib 2024-12-04 12:06:43,403 root INFO building 'functorch._C' extension 2024-12-04 12:06:43,404 root INFO x86_64-pc-linux-gnu-gcc -shared -flto=auto -O3 -pipe -march=znver5 -fomit-frame-pointer -fcf-protection=return -flto=auto -ffat-lto-objects -DNDEBUG -L/usr/lib64 -o /var/tmp/portage/sci-libs/pytorch-2.5.1/work/pytorch-2.5.1_python3.11/build/lib.linux-x86_64-cpython-311/functorch/_C.cpython-311-x86_64-linux-gnu.so x86_64-pc-linux-gnu-gcc: fatal error: no input files I also tried with PYTHON_SINGLE_TARGET="python3_12" and LTO disabled but no improvements. Same issue. Something expected is definitely missing.
See https://github.com/pytorch/pytorch/blob/main/setup.py#L1039.
(In reply to Sam James from comment #13) > See https://github.com/pytorch/pytorch/blob/main/setup.py#L1039. Thanks for pointing in the right direction. Removing this block from setup.py (or making the condition being false) resolves the issue: if cmake_cache_vars["BUILD_FUNCTORCH"]: extensions.append( Extension(name="functorch._C", sources=[]), ) According to https://pytorch.org/functorch/stable/install.html: "As of PyTorch 1.13, functorch is now included in the PyTorch binary and no longer requires the installation of a separate functorch package. That is, after installing PyTorch (instructions), you’ll be able to import functorch in your program." I tried one of the examples from https://pytorch.org/functorch/stable/notebooks/whirlwind_tour.html and it runs just fine by substituting "import functorch" for "from torch.func import ...": >>> from torch.func import jvp >>> x = torch.randn(5) >>> y = torch.randn(5) >>> f = lambda x, y: (x * y) >>> _, output = jvp(f, (x, y), (torch.ones(5), torch.ones(5))) >>> assert torch.allclose(output, x + y) >>> _ tensor([-2.1011, -4.2663, 0.9916, 0.1252, -0.1543]) >>> >>> output tensor([-0.6356, -0.2882, -2.0866, -0.8619, 0.4321])
(In reply to Adrien Dessemond from comment #14) > (In reply to Sam James from comment #13) > > See https://github.com/pytorch/pytorch/blob/main/setup.py#L1039. > > Thanks for pointing in the right direction. > Removing this block from setup.py (or making the condition being false) > resolves the issue: > > if cmake_cache_vars["BUILD_FUNCTORCH"]: > extensions.append( > Extension(name="functorch._C", sources=[]), > ) > > According to https://pytorch.org/functorch/stable/install.html: > > "As of PyTorch 1.13, functorch is now included in the PyTorch binary and no > longer requires the installation of a separate functorch package. That is, > after installing PyTorch (instructions), you’ll be able to import functorch > in your program." > > I tried one of the examples from > https://pytorch.org/functorch/stable/notebooks/whirlwind_tour.html and it > runs just fine by substituting "import functorch" for "from torch.func > import ...": > Yes we could drop the functorch installation at all, but I prefer fixing it. A file that is build on caffe2 is not found by pytorch. I will fix it. Well, some day I will merge the two packages
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3c4ac3c3accc78d1b85b785b32c7adbcdf0aa9c0 commit 3c4ac3c3accc78d1b85b785b32c7adbcdf0aa9c0 Author: Alfredo Tupone <tupone@gentoo.org> AuthorDate: 2024-12-09 08:22:33 +0000 Commit: Alfredo Tupone <tupone@gentoo.org> CommitDate: 2024-12-09 08:25:31 +0000 sci-libs/caffe2: save functorch.so for pytorch ebuild Bug: https://bugs.gentoo.org/943531 Signed-off-by: Alfredo Tupone <tupone@gentoo.org> sci-libs/caffe2/{caffe2-2.5.1-r4.ebuild => caffe2-2.5.1-r5.ebuild} | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=54ba2742b754e3c25add341ace31836d699d2ae1 commit 54ba2742b754e3c25add341ace31836d699d2ae1 Author: Alfredo Tupone <tupone@gentoo.org> AuthorDate: 2024-12-09 08:33:29 +0000 Commit: Alfredo Tupone <tupone@gentoo.org> CommitDate: 2024-12-09 08:33:29 +0000 sci-libs/pytorch: fix functorch install, copying from caffe2 Closes: https://bugs.gentoo.org/943531 Signed-off-by: Alfredo Tupone <tupone@gentoo.org> sci-libs/pytorch/{pytorch-2.5.1.ebuild => pytorch-2.5.1-r1.ebuild} | 4 ++++ 1 file changed, 4 insertions(+)
