Hi, I know there is boostrap-bash.sh too but I went with: $ module load Bash/4.4-intel-2017b $ bash --help GNU bash, version 4.4.0(1)-release-(x86_64-unknown-linux-gnu) Usage: bash [GNU long option] [option] ... bash [GNU long option] [option] script-file ... GNU long options: --debug --debugger --dump-po-strings --dump-strings --help --init-file --login --noediting --noprofile --norc --posix --rcfile --restricted --verbose --version Shell options: -ilrsD or -c command or -O shopt_option (invocation only) -abefhkmnptuvxBCHP or -o option Type `bash -c "help set"' for more information about shell options. Type `bash -c help' for more information about shell builtin commands. Use the `bashbug' command to report bugs. bash home page: <http://www.gnu.org/software/bash> General help using GNU software: <http://www.gnu.org/gethelp/> $ $ wget https://gitweb.gentoo.org/repo/proj/prefix.git/plain/scripts/bootstrap-prefix.sh $ chmod a+r bootstrap-prefix.sh $ ./bootstrap-rap.sh . .vir. d$b .d$$$$$$b. .cd$$b. .d$$b. d$$$$$$$$$$$b .d$$b. .d$$b. $$$$( )$$$b d$$$()$$$. d$$$$$$$b Q$$$$$$$P$$$P.$$$$$$$b. .$$$$$$$b. Q$$$$$$$$$$B$$$$$$$$P" d$$$PQ$$$$b. $$$$. .$$$P' `$$$ .$$$P' `$$$ "$$$$$$$P Q$$$$$$$b d$$$P Q$$$$b $$$$b $$$$b..d$$$ $$$$b..d$$$ d$$$$$$P" "$$$$$$$$ Q$$$ Q$$$$ $$$$$ `Q$$$$$$$P `Q$$$$$$$P $$$$$$$P `""""" "" "" Q$$$P "Q$$$P" "Q$$$P" `Q$$P" """ Welcome to the Gentoo Prefix interactive installer! I will attempt to install Gentoo Prefix on your system. To do so, I'll ask you some questions first. After that, you'll have to practise patience as your computer and I try to figure out a way to get a lot of software packages compiled. If everything goes according to plan, you'll end up with what we call "a Prefix install", but by that time, I'll tell you more. Do you want me to start off now? [Yn] It seems to me you are 'mmokrejs' (3900), that looks cool to me. I'm going to check for some variables in your environment now: it appears ASFLAGS is not set :) it appears CFLAGS is not set :) it appears CPPFLAGS is not set :) it appears CXXFLAGS is not set :) it appears DYLD_LIBRARY_PATH is not set :) it appears GREP_OPTIONS is not set :) it appears LDFLAGS is not set :) it appears LD_LIBRARY_PATH is not set :) it appears LIBPATH is not set :) it appears PERL_MM_OPT is not set :) it appears PERL5LIB is not set :) it appears PKG_CONFIG_PATH is not set :) it appears PYTHONPATH is not set :) it appears ROOT is not set :) I'm excited! Seems we can finally do something productive now. Ok, I'm going to do a little bit of guesswork here. Thing is, your machine appears to be identified by CHOST=x86_64-pc-linux-gnu. Great! You appear to have a compiler in your PATH I did my utmost best, and found that you have 28 cpu cores. If this looks wrong to you, you can happily ignore me. Based on the number of cores you have, I came up with the idea of parallelising compilation work where possible with 8 parallel make threads. If you have no clue what this means, you should go with my excellent default I've chosen below, really! How many parallel make jobs do you want? [8] 4 Each and every Prefix has a home. That is, a place where everything is supposed to be in. That place must be fully writable by you (duh), but should also be able to hold some fair amount of data and preferably be reasonably fast. In terms of space, I advise something around 2GiB (it's less if you're lucky). I suggest a reasonably fast place because we're going to compile a lot, and that generates a fair bit of IO. If some networked filesystem like NFS is the only option for you, then you're just going to have to wait a fair bit longer. This place which is your Prefix' home, is often referred to by a variable called EPREFIX. What do you want EPREFIX to be? [/home/mmokrejs/gentoo] OK! I'm going to give it a try, this is what I have collected sofar: EPREFIX=/home/mmokrejs/gentoo CHOST=x86_64-pc-linux-gnu PATH=/home/mmokrejs/gentoo/usr/bin:/home/mmokrejs/gentoo/bin:/home/mmokrejs/gentoo/tmp/usr/bin:/home/mmokrejs/gentoo/tmp/bin:/home/mmokrejs/gentoo/tmp/usr/local/bin:/usr/bin:/bin MAKEOPTS=-j4 I'm now going to make an awful lot of noise going through a sequence of stages to make your box as groovy as I am myself, setting up your Prefix. In short, I'm going to run stage1, stage2, stage3, followed by emerge -e system. If any of these stages fail, both you and me are in deep trouble. So let's hope that doesn't happen. Type here what you want to wish me [luck] /apps/all/Bash/4.4-intel-2017b/bin/bash: error while loading shared libraries: libiomp5.so: cannot open shared object file: No such file or directory I tried running /apps/all/Bash/4.4-intel-2017b/bin/bash ./bootstrap-rap.sh "/home/mmokrejs/gentoo" stage1 but that failed :( I have no clue, really. Please find friendly folks in #gentoo-prefix on irc.gentoo.org, gentoo-alt@lists.gentoo.org mailing list, or file a bug at bugs.gentoo.org under Gentoo/Alt, Prefix Support. Sorry that I have failed you master. I shall now return to my humble cave. You can find a log of what happened in /home/mmokrejs/gentoo/stage1.log $ $ ldd /apps/all/Bash/4.4-intel-2017b/bin/bash linux-vdso.so.1 => (0x00007ffff7ffb000) libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007ffff7bbb000) libdl.so.2 => /lib64/libdl.so.2 (0x00007ffff79b7000) libiomp5.so => /apps/all/imkl/2017.4.239-iimpi-2017b/lib/intel64/libiomp5.so (0x00007ffff7613000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ffff73f6000) libm.so.6 => /lib64/libm.so.6 (0x00007ffff7172000) libgcc_s.so.1 => /apps/all/GCCcore/6.3.0/lib64/libgcc_s.so.1 (0x00007ffff7fc6000) libc.so.6 => /lib64/libc.so.6 (0x00007ffff6dde000) /lib64/ld-linux-x86-64.so.2 (0x00007ffff7ddc000) $ OK, seems I cannot use bash from parent's host system through modules.
Created attachment 523558 [details] bash -x bootstrap-prefix.sh On line 139 it shows the script still bother about system-wide bash, installed in /bin/bash. 139 ++ env -i HOME=/home/mmokrejs /bin/bash -l I explicitly compiled bash-4.4 in my $HOME/bash-4.4 and pointed PATH to it, before calling bootstrap-prefix.sh . Therefore, this is a bug IMO. Anyway, the LD_LIBRARY_PATH was cleared in bootstrap-prefix.sh so I am not surprised of the error. $ echo $LD_LIBRARY_PATH /apps/all/Bash/4.4-intel-2017b/lib:/apps/all/libreadline/7.0/lib:/apps/all/ncurses/6.0/lib:/apps/all/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64:/apps/all/imkl/2017.4.239-iimpi-2017b/lib/intel64:/apps/all/impi/2017.4.239-iccifort-2017.5.239-GCC-6.3.0-2.27/lib64:/apps/all/ifort/2017.5.239-GCC-6.3.0-2.27/compilers_and_libraries_2017.5.239/linux/compiler/lib/intel64:/apps/all/icc/2017.5.239-GCC-6.3.0-2.27/compilers_and_libraries_2017.5.239/linux/compiler/lib/intel64:/apps/all/binutils/2.27-GCCcore-6.3.0/lib:/apps/all/GCCcore/6.3.0/lib/gcc/x86_64-pc-linux-gnu/6.3.0:/apps/all/GCCcore/6.3.0/lib64:/apps/all/GCCcore/6.3.0/lib:/apps/all/Lua/5.1.4-8/lib $ I suggest emptying LD_LIBRARY_PATH as long as bash can be executed succesfully. Or, fix https://wiki.gentoo.org/wiki/Project:Prefix and make it clear I am supposed to prepare a static binary. Finally, make it clear if/whether modules are supported or not. Seems not.
I think we should lift the ban for <bash-4.2 in stage1, because we build bash there. Then we don't need these cludgy workarounds.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=bdb73088dff33872b9f18a81fcc353aacab2e4c2 commit bdb73088dff33872b9f18a81fcc353aacab2e4c2 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2018-03-13 08:11:31 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2018-03-13 08:11:31 +0000 scripts/bootstrap-prefix: lift restriction on bash-4.2 Allow older bash (like 2.59 or 3.00) to be used in chost, interactive, stage1, since the latter builds bash in such a way that stage2 and 3 can use more advanced features. Bug: https://bugs.gentoo.org/650278 Bug: https://bugs.gentoo.org/650284 scripts/bootstrap-prefix.sh | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-)}
@heroxbd: if we would launch bash --norc --noprofile for stage1 here, we'd still have this fail, as we wipe out LD_LIBRARY_PATH, which is the reason I decided to keep running in the existing interpreter. (In reply to Martin Mokrejš from comment #1) > Created attachment 523558 [details] > bash -x bootstrap-prefix.sh > > On line 139 it shows the script still bother about system-wide bash, > installed in /bin/bash. > > 139 ++ env -i HOME=/home/mmokrejs /bin/bash -l That's probably an expansion from env -i HOME=$HOME $SHELL -l In other words, we can't tell, can we? Your SHELL var doesn't point to the shell you're using for some reason. > I explicitly compiled bash-4.4 in my $HOME/bash-4.4 and pointed PATH to it, > before calling bootstrap-prefix.sh . Therefore, this is a bug IMO. No. your shell need not to be in PATH, SHELL should point to it though. > Finally, make it clear if/whether modules are supported or not. Seems not. I'm not really familiar with modules. In what way do they conflict with the bootstrap? If we can compile bash from source, with our settings to make it read from the offset location, we should be safe as we no longer rely on any behaviour from host, no?
(In reply to Fabian Groffen from comment #4) > @heroxbd: if we would launch bash --norc --noprofile for stage1 here, we'd > still have this fail, as we wipe out LD_LIBRARY_PATH, which is the reason I > decided to keep running in the existing interpreter. Okay, I see your point. > (In reply to Martin Mokrejš from comment #1) > > Created attachment 523558 [details] > > bash -x bootstrap-prefix.sh > > > > On line 139 it shows the script still bother about system-wide bash, > > installed in /bin/bash. > > > > 139 ++ env -i HOME=/home/mmokrejs /bin/bash -l > > That's probably an expansion from > > env -i HOME=$HOME $SHELL -l > > In other words, we can't tell, can we? Your SHELL var doesn't point to the > shell you're using for some reason. > > > I explicitly compiled bash-4.4 in my $HOME/bash-4.4 and pointed PATH to it, > > before calling bootstrap-prefix.sh . Therefore, this is a bug IMO. > > No. your shell need not to be in PATH, SHELL should point to it though. > > > Finally, make it clear if/whether modules are supported or not. Seems not. > I'm not really familiar with modules. In what way do they conflict with the > bootstrap? If we can compile bash from source, with our settings to make it > read from the offset location, we should be safe as we no longer rely on any > behaviour from host, no? Modules is a fancy way to manage a load of environment variable, in the niche of academia. If we rely on our own bash, we are safe.
(In reply to Fabian Groffen from comment #4) > @heroxbd: if we would launch bash --norc --noprofile for stage1 here, we'd > still have this fail, as we wipe out LD_LIBRARY_PATH, which is the reason I > decided to keep running in the existing interpreter. > > (In reply to Martin Mokrejš from comment #1) > > Created attachment 523558 [details] > > bash -x bootstrap-prefix.sh > > > > On line 139 it shows the script still bother about system-wide bash, > > installed in /bin/bash. > > > > 139 ++ env -i HOME=/home/mmokrejs /bin/bash -l > > That's probably an expansion from > > env -i HOME=$HOME $SHELL -l > > In other words, we can't tell, can we? Your SHELL var doesn't point to the > shell you're using for some reason. The parent OS I used to call bootstrapprefix.sh says: $ echo $SHELL /bin/bash $ grep -H SHELL /etc/bashrc /etc/profile .bashrc .bash_profile $ There is one match but I assume irrelevant: /etc/crontab:SHELL=/bin/bash Isn't my SHELL inherited from /etc/passwd and in my case called by sshd during login? > > I explicitly compiled bash-4.4 in my $HOME/bash-4.4 and pointed PATH to it, > > before calling bootstrap-prefix.sh . Therefore, this is a bug IMO. > > No. your shell need not to be in PATH, SHELL should point to it though. Interesting, I could try override the SHELL with HOME/bash-4.4/bash . I gave it a try with this version: $ ls -latr bootstrap-prefix.sh -rwxr-xr-x 1 mmokrejs mmokrejs 89699 Mar 12 12:00 bootstrap-prefix.sh $ $ bootstrap-prefix.sh --version !!! This script requires GNU bash 4.2 or newer to run. !!! You are using /bin/bash which is 4.1.2(2)-release !!! If you don't have a newer one, please bootstrap bash 4.2 first: !!! https://wiki.gentoo.org/wiki/Project:Prefix/Bootstrap !!! You can prepend a better bash's location to your PATH, or call !!! this script with the alternative bash like so: !!! /path/to/bash-4.2 bootstrap-prefix.sh $ $ sha1sum bootstrap-prefix.sh 787395f75da4a2223fcc0a932ab035ead9193377 bootstrap-prefix.sh $ $ export PREFIX=/var/tmp/mmokrejs $ export SHELL=$HOME/bash-4.4/bash $ bash bootstrap-prefix.sh /var/tmp/mmokrejs !!! This script requires GNU bash 4.2 or newer to run. !!! You are using /bin/bash which is 4.1.2(2)-release !!! If you don't have a newer one, please bootstrap bash 4.2 first: !!! https://wiki.gentoo.org/wiki/Project:Prefix/Bootstrap !!! You can prepend a better bash's location to your PATH, or call !!! this script with the alternative bash like so: !!! /path/to/bash-4.2 bootstrap-prefix.sh $ $ $SHELL bootstrap-prefix.sh . .vir. d$b .d$$$$$$b. .cd$$b. .d$$b. d$$$$$$$$$$$b .d$$b. .d$$b. $$$$( )$$$b d$$$()$$$. d$$$$$$$b Q$$$$$$$P$$$P.$$$$$$$b. .$$$$$$$b. Q$$$$$$$$$$B$$$$$$$$P" d$$$PQ$$$$b. $$$$. .$$$P' `$$$ .$$$P' `$$$ "$$$$$$$P Q$$$$$$$b d$$$P Q$$$$b $$$$b $$$$b..d$$$ $$$$b..d$$$ d$$$$$$P" "$$$$$$$$ Q$$$ Q$$$$ $$$$$ `Q$$$$$$$P `Q$$$$$$$P $$$$$$$P `""""" "" "" Q$$$P "Q$$$P" "Q$$$P" `Q$$P" """ Welcome to the Gentoo Prefix interactive installer! ... During emerging bash I saw the "usual" errors caused by our system-wide calls of LMOD stuff from /etc/profile. Well, this bug was about bootstrap-prefix.sh messing up with LD_LIBRARY_PATH and possibly other variables needed to get the bash running, nevertheless I tried to bootstrap using the a bit outdated script but current portage. I failed, see bug #651036 . > > > Finally, make it clear if/whether modules are supported or not. Seems not. > > I'm not really familiar with modules. In what way do they conflict with the > bootstrap? If we can compile bash from source, with our settings to make it > read from the offset location, we should be safe as we no longer rely on any > behaviour from host, no? They alter PATH and LD_LIBRARY_PATH, maybe more. I bootstrap zaps the contents of those variables the say bash provided in a special PATH by modules will stop working if the LD_LIBRARY_PATH is mangled. There is also lmod software, but that is probably not any different in this regard. It is a package manaement system, allowing you to import different namespaces (for example to switch between various versions of applications and the system automatically imports needed dependencies). In this regard, you do not have issues with non-FHS paths, file collisions, etc. The tools are just installed in their own, separate directories. Using modules/lmod typically a lot of software (with many versions) are deployed in cluster environments.
(In reply to Martin Mokrejš from comment #6) > > $ export PREFIX=/var/tmp/mmokrejs > $ export SHELL=$HOME/bash-4.4/bash > $ bash bootstrap-prefix.sh /var/tmp/mmokrejs > !!! This script requires GNU bash 4.2 or newer to run. Which bash is used in your PATH? Do a which -a bash before `bash bootstrap-prefix.sh /var/tmp/mmokrejs`.
