I have installed * bash-3.0-r12 * app-shells/bash-completion-20050121-r10 * app-shells/gentoo-bashcomp-20050516 When I try to use tab completion with emerge # emerge <TAB> I get # emerge -bash: cd: ": No such file or directory # If I write some letters, I get the same error again. At the end the completion works but the string '-bash: cd: ": No such file or directory' is annoying :-( Portage 2.0.51.22-r2 (default-linux/x86/2005.0, gcc-3.3.5-20050130, glibc-2.3.5-r0, 2.6.12-gentoo-r6 i686) ================================================================= System uname: 2.6.12-gentoo-r6 i686 AMD Athlon(tm) Processor Gentoo Base System version 1.6.13 dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=athlon-tbird -fomit-frame-pointer -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /opt/OpenOffice.org/share/dict/ooo /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=athlon-tbird -fomit-frame-pointer -pipe" DISTDIR="/mnt/lfs/distfiles/" FEATURES="autoconfig distlocks sandbox sfperms strict" GENTOO_MIRRORS=" http://www.die.unipd.it/pub/Linux/distributions/gentoo-sources/ ftp://ftp.unina.it/pub/linux/distributions/gentoo http://ftp.students.cs.unibo.it/gentoo/ " LANG="it_IT" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /usr/local/portage-mydev" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 3dnow 3dnowext X aac acl alsa apache2 arts audiofile avi bash-completion berkdb bitmap-fonts blas boehm-gc bonobo bzip2 cddb cdparanoia cdr chroot cmucl crypt cups curl directfb divx4linux doc dv dvb emboss encode ethereal examples faac faad fam fbcon ffmpeg fftw flac foomaticdb fortran fpx gcj gd gdbm gif gimpprint gmp gnutls gphoto2 gpm graphviz gtk gtk2 gtkhtml guile imagemagick imlib innodb java jbig jpeg jpeg2k kde kdeenablefinal lcms libg++ libwww live lm_sensors logitech-mouse lzo mad mikmod mime mjpeg mmx mmxext motif mozdevelop mozilla mozsvg mozxmlterm mp3 mpeg mysql ncurses network nls nntp nptl odbc ogg oggvorbis opengl oss pam pdflib perl php plotutils png postgres ppds python qt quicktime readline real samba sdk sdl skey smime speex spell sql ssl stats subversion tcltk tcpd tetex theora threads tiff truetype truetype-fonts type1 type1-fonts unicode vorbis wifi win32codecs wmf wxgtk1 wxwindows xanim xine xml xml2 xmms xv xvid yv12 zlib video_cards_radeon userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LC_ALL, LDFLAGS, LINGUAS
Works fine here with the same versions.
I have just recompiled all three packages without success :-( In the forums I have found the post http://forums.gentoo.org/viewtopic-t-328479-highlight-bashcompletion+tab.html reporting the same problem Is there a way to discover the source of the problem?
the only thing I can think of to find out what is causing it is to: $ set -o xtrace $ emerge <TAB> you'll get LOTS of output and it may be quite hard to read it all.
Thanks for the tip Aaron, I have discovered the problem :-) The trace is the following: ... [cut] ++ for pd in '${portdir}' ++ builtin cd /usr/portage ++ compgen -S / -G '*-*' ++ for pd in '${portdir}' ++ builtin cd '"' -bash: cd: ": No such file or directory ... [cut] It appears to be a problem in parsing make.conf. Infact in this file I have set the variable PORTDIR_OVERLAY in this way: PORTDIR_OVERLAY=" /usr/local/portage /usr/local/portage-mydev " Setting PORTDIR_OVERLAY as PORTDIR_OVERLAY="/usr/local/portage /usr/local/portage-mydev" solves the problem but I think that the real problem is the parsing of the file /etc/make.conf in the function _portdir() of /usr/share/bash-completion/gentoo I have produced a patch redefining the function _portdir()
Created attachment 64523 [details, diff] gentoo.patch
Created attachment 64678 [details, diff] gentoo.patch Sorry, I forgot make.globals
uh do you mind? I am not CC'd because I am on the shell-tools@ alias.
(In reply to comment #7) > uh do you mind? I am not CC'd because I am on the shell-tools@ alias. Sorry, I didn't know it.
make.conf is not a bash file, so don't treat it like one (dont use multi-line literals).
Ok, but it's better to write a note in the Handbook and in make.conf.5. However emerge reads correctly multiline literals :-) I think that multiline literals are useful: it's easy to remove an entry in GENTOO_MIRRORS, PORTDIR_OVERLAY, etc and it's easier to read long variables as USE
(In reply to comment #10) > Ok, but it's better to write a note in the Handbook and in make.conf.5. True, although you'd have to bug the portage doc folks. > However emerge reads correctly multiline literals :-) It seems to work on somethings, but not others (try using it with CFLAGS for example). Bottom line, it's "undefined behaviour" afaict. > > I think that multiline literals are useful: it's easy to remove an entry in > GENTOO_MIRRORS, PORTDIR_OVERLAY, etc and it's easier to read long variables as USE Couldn't agree more.
(In reply to comment #11) > True, although you'd have to bug the portage doc folks. Ok, I'll open another bug linking this one. Thanks for your help
(In reply to comment #9) > make.conf is not a bash file, so don't treat it like one (dont use multi-line > literals). FYI, multiline stuff works fine with make.conf...
No it doesn't.
(In reply to comment #14) > No it doesn't. Score: 2 vs 1 :-) So emerge has an unwanted and unsupported feature...
(In reply to comment #14) > No it doesn't. Have you tried? :) I have, and it works.
(In reply to comment #15) > Score: 2 vs 1 :-) So emerge has an unwanted and unsupported feature... Score: Get your basics straight before filing bugs :D
USE="a52 aac alsa acpi apache2 -arts avi bluetooth bash-completion bzlib cups curl cdr crypt cscope ctype -doc ethereal fbcon gd directfb divx4linux examples exif fbcon ftp geoip gphoto2 gif gimpprint gnome gpm gstreamer gtk2 gtkhtml imlib imlib2 innodb -java javascript jpeg -kde libg++ libwww mad mbox md5sum mp3 mikmod mmx motif mozilla mpeg mpeg4 mysql ncurses nvidia oggvorbis odbc offensive opengl pam pdflib perl php png python quicktime readline sdl spell sse ssl svga tcltk tiff truetype usb unicode vanilla win32codecs X xml2 xmms xosd xv xvid zlib" CFLAGS="-O3 -march=pentium3 -mmmx -pipe" GENTOO_MIRRORS="http://gentoo.mirrors.pair.com http://gentoo.osuosl.org http://distro.ibiblio.org/pub/Linux/distributions/gentoo" If the above are not multiline, I don't know what is. Works fine.
(In reply to comment #16) > Have you tried? :) I have, and it works. Yes, I have tried. I know it works and for this reason I have opened this bug. But there are some problems as explained by SpanKY in #101486.
(In reply to comment #17) > Score: Get your basics straight before filing bugs :D I have opened the bug #101486 because Aaron's request...
*** Bug 103627 has been marked as a duplicate of this bug. ***
Why was this resolved WONTFIX? If multiline literals are not supported in make.conf then the docs need to be updated to reflect this, and portage needs to refuse multiline literals. If multiline literals are supported in make.conf then gentoo-bashcomp should support them. If the issue is sourcing make.conf, then an appropriate sed can be written.
Created attachment 66862 [details, diff] gentoo.patch Patch that uses sed to process /etc/make.conf supporting multiline literals
You might be able to get away using multilines and simple bashisms in make.conf with some tools. If you do, lucky you. If you don't, stop using them.
OK.. sorry, but I'm still trying to work this out. Is that the official Gentoo policy, or is it your personal opinion? (Please understand, I'm not trying to be rude, I just want to get this straightened out.)
Created attachment 77868 [details, diff] 100373.bugs.gentoo.org-multiline-overlays.patch Sorry, that patch was slightly broken. Corrected patch here.
Ed Catmur: been having the described problem since last summer. Thanks for the patch! Ciaran: "If you do, lucky you. If you don't, stop using them.". Since when is that the spirit of free software? "If you do, lucky you. If you don't, fix it." seems way more appropriate. As a user I've got the impression nobody having the "power" to add the patch to the tree cares about the quality of this software. It's pretty sad a one line fix to an almost year(!!!) old bug has _not_ been applied, seemingly because ego matters more than technical correctness...
I think this a portage bug because I feel that make.conf should not be anything more that what bash supports.
> I think this a portage bug because I feel that make.conf should not be anything more that what bash supports. I'm not sure if I'm parsing that correctly, but do you mean make.conf should not support multiline literals because bash doesn't?. Looks like it's works just fine? dieterv@PO001 ~ $ cat test.sh #!/bin/bash MULTILINE_LITERAL="/home/dieterv/dev/stuff/overlay/ /home/dieterv/dev/test/overlay/ /home/dieterv/dev/maintenence/overlay/" MULTILINE_LITERAL_WITH_NEWLINES_CONTINUATION=" \ /home/dieterv/dev/stuff/overlay/ \ /home/dieterv/dev/test/overlay/ \ /home/dieterv/dev/maintenence/overlay/" echo $MULTILINE_LITERAL echo $MULTILINE_LITERAL_WITH_NEWLINES_CONTINUATION dieterv@PO001 ~ $ ./test.sh /home/dieterv/dev/stuff/overlay/ /home/dieterv/dev/test/overlay/ /home/dieterv/dev/maintenence/overlay/ /home/dieterv/dev/stuff/overlay/ /home/dieterv/dev/test/overlay/ /home/dieterv/dev/maintenence/overlay/
Oh, I was led to believe that portage supported some syntax that is unsupported by bash. Now I see that gentoo-bashcomp uses sed to parse make.conf. Sorry for the noise.
(In reply to comment #30) > Oh, I was led to believe that portage supported some syntax that is unsupported > by bash. Now I see that gentoo-bashcomp uses sed to parse make.conf. Sorry > for the noise. try setting FEATURES multiple times in make.conf (2.1.1_pre5-r3), comes through incrementally when it shouldn't. or... x=asdf\ b VIDEO_CARDS="asdf"b VIDEO_CARDS='asdf'b'asdf' x=asdf\ b So... portage doesn't actually support bash parsing. via the features example above, it actually differs noticably from the equiv bash parsing. Short version, python's shlex sucks, have to monkey patch it a bit to get it to behave properly for bash rules. Mentioned it to zmedico, mentioning it here for anyone interested to grab the code and glue it into portage- http://gentooexperimental.org/~ferringb/bzr/pkgcore/pkgcore/util/file.py http://gentooexperimental.org/~ferringb/bzr/pkgcore/pkgcore/test/util/test_file.py bash parser, and unittest; uses ProtectedDict to shield mutation of the passed in env dict, src for that is http://gentooexperimental.org/~ferringb/bzr/pkgcore/pkgcore/util/mappings.py http://gentooexperimental.org/~ferringb/bzr/pkgcore/pkgcore/test/util/test_mappings.py Any takers to rip the shlex out of portage_util and glue in file.bash_parser? It's got tests ;) Bash-completion may screw up parsing, but portages bash parsing has historically been buggy, examples stated above.
Let me quote Marius Mauch from bug #100373 (comment #3): > It's bash compatible (it can be parsed by bash), so such a statement would be > incorrect (how do you define a "bash file"?) > Multiline vars are supported in make.conf. ("It" is make.conf) And Alec Warner from bug #101416 (comment #6): > I believe the consensus from gentoo-portage-dev ML was that /etc/make.conf > should be bash sourceable, and anything not bash sourceable was in fact an > invalid make.conf file. I know Brian is working on the parsing for the portage > rewrite to be as close to bash syntax as he can make it so that no functionality > is lost. However in this case the user would need to modify their make.conf to > be sourceable by bash. Applications may depend on that assumption ( which euse > does ). So, make.conf should be bash sourceable. This seems to include support multiline declarations, and variable expansion. Having the following in make.conf also breaks gentoo-bashcomp: source /usr/portage/local/layman/make.conf PORTDIR_OVERLAY="${PORTDIR_OVERLAY} /usr/portage/local " (I have it this way so that my local overlay of custom ebuilds overrides layman overlays.) First, the 'source' is not honored, so I miss several ebuilds. Next, I get similar ENOENT ("No such file...") errors on <TAB> when it tries to cd into '${PORTDIR_OVERLAY}'. Since make.conf seems officially meant to be bash-compatible, why not use bash (which is already running, since this is bash-completion) to solve these issues, and actually RESOLVE FIXED this bug using the solution in bug #103627 (reproduced below)? The overhead of spawning the sed process is worse than invoking the full bash lexer in a subshell, which does not spawn a new process [1], if that might be your reason. There seems to be no downside. The solution from bug #103626 (comment #0): > _portdir() > { > ( source /etc/make.globals; source /etc/make.conf > echo $PORTDIR ) 2>/dev/null > > if [[ $1 == '-o' ]] ; then > ( source /etc/make.conf; echo $PORTDIR_OVERLAY ) 2>/dev/null > fi > } [1] Why, yes, I did test this: whereami@thang ~/src $ time ( for (( i=0 ; i<1000 ; i++ )) ; do ( . /etc/make.conf ; echo $PORTDIR_OVERLAY > /dev/null ) ; done) real 0m1.816s user 0m0.899s sys 0m0.913s whereami@thang ~/src $ time ( for (( i=0 ; i<1000 ; i++ )) ; do ( PORTDIR_OVERLAY=$(sed -n -e '/^PORTDIR_OVERLAY=/ { s/^[^=]\+="\?\([^"]\+\|\S\+\).*/\1/p ; q }' /etc/make.conf 2>/dev/null) ; echo $PORTDIR_OVERLAY > /dev/null ) ; done) real 0m3.974s user 0m1.316s sys 0m2.651s Note that my make.conf isn't exactly short either, and includes a source for layman overlays: whereami@thang ~/src $ wc /etc/make.conf 38 111 1132 /etc/make.conf whereami@thang ~/src $ wc /usr/portage/local/layman/make.conf 6 6 180 /usr/portage/local/layman/make.conf I also tested that subshells and source directives do not spawn processes: whereami@thang ~/src $ echo $$ 19294 whereami@thang ~/src $ (echo $$) # subshell 19294 whereami@thang ~/src $ (. getpid.bash) # sourcing 19294
Forgot to include what getpid.bash is: whereami@thang ~/src $ cat getpid.bash #!/bin/bash echo $$
Reopening for reassignment.
Not a portage bug.
So, if you can guarantee that make.conf should have sane bash syntax, I'll change gentoo-bashcomp to work on that assumption.
Actually, on my system the sed is slightly faster. However using source in a subshell is arguably more elegant and is the easiest way to support source syntax. re comment 36: make.conf syntax is currently a subset of bash syntax. Parsing semantic differ slightly, but I can't see anyone actually using those differences; in any case the sed script doesn't handle them any better.
ahem, can we apply this now?
(In reply to comment #38) > ahem, can we apply this now? Sorry, real life (job, school) has been taking priority for a while. I'll have this pushed out soon-ish. Of course, you're always free to make these changes locally, that's the beauty of open source software. The maintainers have their own priorities and limited time, so being rude isn't the greatest way to get us to do things for you.
(In reply to comment #39) > so being rude isn't the greatest way to get us to do things for you. It was meant as a gentle reminder. Sorry if it seemed rude. Thank you for your efforts.
Funny that I have in make.conf following lines: === # Overlays by layman utility source /usr/portage/local/layman/make.conf # Local handmade overlay (high order) PORTDIR_OVERLAY="/usr/local/portage $PORTDIR_OVERLAY" === Of course, bash completion works wrong enough: === emerge <tab>-su: cd: $PORTDIR_OVERLAY: No such file or directory <tab>-su: cd: $PORTDIR_OVERLAY: No such file or directory Display all 161 possibilities? (y or n) === It thinks that $PORTDIR_OVERLAY is a real path. BUT, emerge --info resolves overlay dirs correctly: === # emerge --info | grep OVERLAY PORTDIR_OVERLAY="/usr/local/portage /usr/portage/local/layman/sunrise" === Then I thought, may be we should use the same algorithm as emerge (or part of emerge) that resolves this variable (instead of using sed)? I think, it's hard to handle all such cases with sed. BTW, /usr/portage/local/layman/make.conf, auto-generated, has multiline PORTDIR_OVERLAY (but easily processed by emerge): === # cat /usr/portage/local/layman/make.conf PORTDIR_OVERLAY=" /usr/portage/local/layman/sunrise $PORTDIR_OVERLAY " ===
Should be fixed with bug 182809
