Hi, I wonder why on one PBSpro (I think) cluster I cannot get startprefix executed through the batch system. $ qsub -l select=1:ncpus=1:mem=600gb:scratch_local=500gb,walltime=168:00:00 -I -S $my_dir/gentoo/startprefix qsub: waiting for job 3217262.foo to start qsub: job 3217262.foo ready You appear to be in prefix already (SHELL=$my_dir/gentoo/startprefix) qsub: job 3217262.foo completed $ I think it is a false message.
$ startprefix ... $ emerge --info Portage 3.0.28 (python 3.9.7-final-0, default/linux/amd64/17.1/no-multilib/prefix/kernel-3.2+, gcc-11.2.0, glibc-2.33-r7, 4.19.0-17-amd64 x86_64) ================================================================= System uname: Linux-4.19.0-17-amd64-x86_64-Intel_Xeon_Processor_-Skylake,_IBRS-with-glibc2.33 KiB Mem: 16426692 total, 1379036 free KiB Swap: 524284 total, 0 free Timestamp of repository gentoo: Sun, 17 Oct 2021 12:30:01 +0000 Head commit of repository gentoo: 5c721aed64c38a600a86193bc812ef5ceb2c39d5 sh bash 5.1_p8 ld GNU ld (Gentoo 2.37_p1 p0) 2.37 app-shells/bash: 5.1_p8::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.34.0-r3::gentoo dev-lang/python: 3.8.12_p1::gentoo, 3.9.7_p1::gentoo, 3.10.0_p1::gentoo dev-lang/rust: 1.55.0::gentoo dev-lang/rust-bin: 1.55.0::gentoo dev-util/cmake: 3.21.3::gentoo sys-apps/baselayout: 2.8::gentoo sys-apps/sandbox: 2.25::gentoo sys-devel/autoconf: 2.71-r1::gentoo sys-devel/automake: 1.16.5::gentoo sys-devel/binutils: 2.37_p1::gentoo sys-devel/gcc: 9.3.0-r2::gentoo, 9.4.0::gentoo, 10.2.0-r5::gentoo, 11.2.0::gentoo sys-devel/gcc-config: 2.4::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.14::gentoo (virtual/os-headers) sys-libs/glibc: 2.33-r7::gentoo Repositories: gentoo location: /auto/brno3-cerit/nfs4/home/mmokrejs/gentoo/var/db/repos/gentoo sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: no sync-rsync-verify-max-age: 24 sync-rsync-verify-jobs: 1 science location: /auto/brno3-cerit/nfs4/home/mmokrejs/gentoo/var/lib/layman/science masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE Intel-SDP intel-ucode fasta meme mRNAmarkup ApE modeller gmap abyss swiss-prot Aladdin AVASYS pyvcf Non-profit-OSL-3.0 estscan blat staden CeCILL-1.1 phrap STRIDE cctbx-2.0 bh-luxi ARIADNE unRAR trf freedist free-noncomm genscan no-source-code BSD MIT BSD-2 unicode Subversion all-rights-reserved BSD-BroadInstitute" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -mtune=native -ftree-vectorize" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /storage/brno3-cerit/home/mmokrejs/gentoo/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /storage/brno3-cerit/home/mmokrejs/gentoo/etc/env.d /storage/brno3-cerit/home/mmokrejs/gentoo/etc/gentoo-release" CXXFLAGS="-O2 -pipe -mtune=native -ftree-vectorize" DISTDIR="/storage/brno3-cerit/home/mmokrejs/gentoo/var/cache/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe -mtune=native -ftree-vectorize" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles force-prefix ipc-sandbox merge-sync multilib-strict network-sandbox news nostrip parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans unprivileged" FFLAGS="-O2 -pipe -mtune=native -ftree-vectorize" GENTOO_MIRRORS="http://distfiles.gentoo.org" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j1" PKGDIR="/storage/brno3-cerit/home/mmokrejs/gentoo/var/cache/binpkgs" PORTAGE_CONFIGROOT="/storage/brno3-cerit/home/mmokrejs/gentoo/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/storage/brno3-cerit/home/mmokrejs/gentoo/var/tmp" USE="acl amd64 asm avx avx2 berkdb boost bzip2 cli crypt dri emboss fortran gdbm gmp iconv icu java jpeg jpg lapack libglvnd libtirpc mariadb mmx ncurses nfs nls nptl nptlonly objc openmp openssl pbs pcre pcre16 pdf perl png prefix readline sctp seccomp server session split-usr sqlite sse sse2 sse3 sse4_1 sse4_2 ssl sslv3 ssse3 svg tcl tcpd threads tiff tk truetype unicode vim-syntax xml zlib" ABI_X86="64" ADA_TARGET="gnat_2019" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 avx512f avx512dq avx512cd avx512bw avx512vl f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9" RUBY_TARGETS="ruby26 ruby27" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RUSTFLAGS
You get this message when the shell trying to run startprefix is already the prefix shell. In your case, startprefix is probably being run by $EPREFIX/bin/bash. Please check how you are starting the job. This is definitely not a problem in prefix itself. Maybe you should try replacing $my_dir/gentoo/startprefix with $my_dir/bin/bash to start your job. Otherwise, just start your job with /bin/bash and run startprefix after qsub puts you into the interactive session.
Hi, no, I am starting the pbs jobs from a normal /bin/bash. PBS gives me normallly also /bin/bash . Yes, I can start $my_dir/gentoo/bin/bash trhough PBS but the problem is that my environment and $PATH do not point to teh remainder of prefix installation, so it is not working. I can start a PBS job with a normal /bin/bash (printenv does not show any variable related to Gentoo prefix, I even deleted it from PATH) and then run startprefix but that zaps my env settings. I think I used to be able to run $my_dir/gentoo/startprefix directly on other clusters, do not have an account there anymore, sorry. How can I disable this check?
To answer your question, startprefix does a particularly odd check to see if SHELL starts with EPREFIX. So you can simply disable this check by removing it from the script, or by ensuring SHELL doesn't contain EPREFIX, e.g. by exporting it as SHELL=bash. The script modifies SHELL in any case to be a binary in $EPREFIX/bin, so the contents of SHELL don't matter, as long as the last component is an executable present in $EPREFIX/bin
So I am guessing that PBS sets the SHELL variable and other variables, in the end executes the shell to be executed, in my case $my_dir/gentoo/startprefix. Then the startprefix immediately fails as the SHELL variable contains $my_dir/gentoo. Could someone verfity the theory. I know that normally I get SHELL=/bin/bash through PBS. I used to use PBSpro 2 years ago successfully and having $my_dir/gentoo/startprefix as my shell. Is this a relatively new change?
I think startprefix's behaviour hasn't changed for a while, and the particular behaviour for checking if already in Prefix is in there for 10+ years, maybe?
got the impression this can be worked around by hacking startprefix to cater for your environment
