Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 73266 - bash-completion has /usr/portage hardcoded
Summary: bash-completion has /usr/portage hardcoded
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Unspecified (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo Shell Tools project
URL:
Whiteboard:
Keywords: InVCS
: 75916 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-12-03 14:48 UTC by Patrick Fourniols
Modified: 2004-12-28 08:19 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Patrick Fourniols 2004-12-03 14:48:16 UTC
UnKnOwN ~ # set -v
UnKnOwN ~ # emerge ap (HERE A "TAB" WITHOUT A SPACE BEFORE ;) )
echo "${compwords}" | egrep -o --regex="${actionre}" | tr -d ' '
echo "${compwords}" | egrep -o -regexp=" -(V|-version)\b"
echo "${compwords}" | egrep -o --regexp=" -(S|-searchdesc)\b"
echo "${compwords}" | egrep -o --regexp=" -(h|-help)\b"
echo "${compwords}" | egrep -o --regex="${stopre}"
echo "${cur}" | grep '/'
cd /usr/portage; compgen -X "!*-*" -S '/' -G "${cur}*" | ${sedcmd} <=== !! HERE THE PB :(
echo "${words}" | ${grepcmd} | grep '/' | wc -l
echo "${words}" | ${grepcmd}
cd /usr/portage; for i in [a-z]*-[a-z0-9]*/${cur}*; do [ -d $i ] && echo ${i##*/}; done
compgen -W "${words}" -- ${cur}

Reproducible: Always
Steps to Reproduce:
1.just try to use bash completion with emerge on a computer with no /usr/portage or an old one give no or outdated answer
2.
3.



Expected Results:  
i think you will find what without any help from me ;)

Portage 2.0.51-r8 (default-linux/x86/2004.3, gcc-3.4.3, glibc-2.3.4.20041102-r0,
 2.6.9-gentoo-r9 i686)
=================================================================
System uname: 2.6.9-gentoo-r9 i686 AMD Athlon(tm) XP 1800+
Gentoo Base System version 1.6.6
Python:              dev-lang/python-2.3.4 [2.3.4 (#1, Sep 23 2004, 18:33:11)]
distcc 2.18.2 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disable d]
ccache version 2.3 [enabled]
dev-lang/python:     2.3.4
sys-devel/autoconf:  2.59-r6, 2.13
sys-devel/automake:  1.8.5-r2, 1.5, 1.4_p6, 1.6.3, 1.7.9, 1.9.3
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10
virtual/os-headers:  2.6.8.1-r1
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CFLAGS="-march=athlon-xp -O3 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.3
/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /us
r/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /
usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/te
xmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=athlon-xp -O3 -pipe"
DISTDIR="/mnt/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks fixpackages sandbox sfperms"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/Linux/
distributions/gentoo"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/mnt/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="3dnow 3dnowex X Xaw3d aalib acpi alsa apache2 apm arts avi bash-completion
bdf berkdb bitmap-fonts bonobo cdr clamav crypt cups dga divx4linux dvd dvdr enc
ode esd f77 fam flac foomaticdb fortran gdbm gif gimpprint glep gnome gnomedb gp
hoto2 gpm gstreamer gtk gtk2 gtkhtml guile imagemagick imap imlib innodb ipalias
 jack java jikes joystick jpeg junit kde krb4 ldap libg++ libwww mad maildir mai
lwrapper mdb mikmod mmx motif mozilla mpeg mysql nas ncurses nls odbc offensive
oggvorbis opengl oss pam pdflib perl pic png postgres ppds python qt quicktime r
eadline samba sasl scanner sdl slang slp speex spell sqlite sse sse2 ssl svga tc
ltk tcpd tetex tiff truetype usb v4l wmf x86 xml xml2 xmms xv xvid zlib"
Comment 1 Patrick Fourniols 2004-12-03 14:58:10 UTC
sorry omited to supply the (rapid ;) ) workaround: just replace in /etc/bash_completion/gentoo /usr/portage by the real dir ( here /mnt/portage for me ;) )
Comment 2 Aaron Walker (RETIRED) gentoo-dev 2004-12-03 15:22:11 UTC
Well, using portageq is out of the question, as it is just too slow for a completion function, IMO.

anyone have any other ideas besides doing a s:/usr/portage:$PORTDIR:g on install?
Comment 3 Mr. Bones. (RETIRED) gentoo-dev 2004-12-03 15:24:02 UTC
sourcing /etc/make.conf might work.
Comment 4 Ciaran McCreesh 2004-12-03 15:34:48 UTC
sourcing could be fairly icky, might be simpler to just try to sed the thing...
Comment 5 Aaron Walker (RETIRED) gentoo-dev 2004-12-03 15:49:10 UTC
Are you sure this would work 100% of the time?  I wondered this as well once, and iirc it was ciaranm who told be that sourcing make.conf was not always guaranteed to work (or something along those lines).

Another downside to sourcing make.conf, is all the garbage that is left behind in the user's env.  Imagine a user with something like this in their make.conf FEATURES="${FEATURES} foo bar", however unlikely.  That would get expanded every time the user hit tab and the completion function was run.
Comment 6 Mr. Bones. (RETIRED) gentoo-dev 2004-12-03 17:38:28 UTC
clearly you would want to source make.conf only once in a subshell and get the value.  I'm definitely not saying it should source make.conf in the current shell.
Comment 7 Aaron Walker (RETIRED) gentoo-dev 2004-12-04 02:09:03 UTC
What about something like:

local portdir=$(grep ^PORTDIR= /etc/make.conf 2>/dev/null)
if [ -z "${portdir}" ] ; then
    portdir="/usr/portage"
else
    portdir=$(echo ${portdir} | sed -e 's:.*="\(.*\)":\1:')
fi

The gentoo-bashcomp maintainer has given me CVS access on SourceForge, so I can do this without having to patch it.
Comment 8 Aaron Walker (RETIRED) gentoo-dev 2004-12-04 02:49:52 UTC
In upstream CVS.  CC'ing maintainer.

Zach, let me know if you want to do a new release or not.  If so, I'll probably do a bash-completion revbump, which besides the gentoo-bashcomp verbump would also include the move of gentoo-bashcomp from /etc/bash_completion.d to /usr/share/bash-completion, and replacing it with a symlink.
Comment 9 Mr. Bones. (RETIRED) gentoo-dev 2004-12-04 02:59:57 UTC
Oh please no.

eval $(. /etc/make.conf && echo PORTDIR=${PORTDIR:-/usr/portage})
Comment 10 Aaron Walker (RETIRED) gentoo-dev 2004-12-04 03:08:12 UTC
Well, I just really wanted to avoid sourcing make.conf.
Zach, you are upstream, so it's really up to you.
Comment 11 Zach Forrest 2004-12-04 12:37:56 UTC
I like Mr. Bones solution. I think it covers all the bases. While it does source /etc/make.conf, it leaves no environment clutter and is bash-only. If there is a custom location for the portage tree, it should be specified in make.conf. I'm not sure why this wouldn't work 100% of the time.

The only change I would make is to the name of the variable to something like BASHCOMP_PORTDIR to avoid any possible conflicts with the portage system. (I'm not sure if emerge unsets the PORTDIR variable when it is finished, for example. I suppose I could run some tests, but I'm on my way out shortly and wanted to respond to the discussion.)

I think putting the following code at the top of the script and changing the hard-coded paths is a good solution:

eval $(. /etc/make.conf && echo BASHCOMP_PORTDIR=${PORTDIR:-/usr/portage})

Let me know what you think.
Comment 12 Ciaran McCreesh 2004-12-04 12:42:28 UTC
You need to parse at least make.globals as well... Oh, and they're not bash files, they just happen to have vaguely similar syntax.
Comment 13 Aaron Walker (RETIRED) gentoo-dev 2004-12-04 13:19:32 UTC
> <snip> they're not bash files, they just happen to have vaguely similar syntax

This was my main concern.  make.conf is not a bash file and portage does not treat it as such.  If you decide to go this route though, I definitely would do something like:

eval $(. /etc/make.conf 2>/dev/null ; echo BASHCOMP_PORTDIR=${PORTDIR:-/usr/portage})

2 advantages over what you showed:
 - ignore stderr in the extremely slim chance sourcing it fails
 - use ; instead of && because if sourcing fails BASHCOMP_PORTDIR won't be defined at all...

Whatever you decide on, if you're going to do a version bump, let me know first as I have an ekeyword completion function (thanks Ciaran), that I was planning on adding as soon as the this bug is resolved (it needs portdir for arch.list ;p). 
Comment 14 Zach Forrest 2004-12-04 14:58:55 UTC
Okay, make.conf is not a bash file, so let's not source it. What about using the grep/sed combination? Using portageq is an option, but I think it's just *way* too slow -- about 185 times slower than grep/sed on my machine, as you can see below.

zach@meta4:~ $ time portageq portdir
/usr/portage

real    0m2.594s
user    0m2.307s
sys     0m0.283s

zach@meta4:~ $ time BASHCOMP_PORTDIR=$(grep ^PORTDIR= /etc/make.conf 2>/dev/null | sed -r -e 's:.*="?([^"]*)"?:\1:')

real    0m0.014s
user    0m0.003s
sys     0m0.011s

For completeness, here is the timing for the variable assignment the must be done to ensure we have a directory:

zach@meta4:~ $ time BASHCOMP_PORTDIR=${BASHCOMP_PORTDIR:-/usr/portage}

real    0m0.000s
user    0m0.000s
sys     0m0.000s

Are there any arguments for using portageq that would make it a better choice? And does such an argument outweigh the speed issue?

Anyway, how about this:

BASHCOMP_PORTDIR=$(grep ^PORTDIR= /etc/make.conf 2>/dev/null | sed -r -e 's:.*="?([^"]*)"?:\1:')
BASHCOMP_PORTDIR=${BASHCOMP_PORTDIR:-/usr/portage}

Does this sit well with everyone?
Comment 15 Ciaran McCreesh 2004-12-04 15:02:12 UTC
Why are you using grep + sed? That's two apps, you can do it all in a single sed -n call.
Comment 16 Zach Forrest 2004-12-04 15:06:03 UTC
Aaron, I will do a version bump when the issue is resolved, but I'll make sure to let you know before hand, so you can commit the new function. :)
Comment 17 Zach Forrest 2004-12-04 15:20:14 UTC
> Why are you using grep + sed? That's two apps, you can do it all in a single sed -n call.

The grep command isolates the PORTDIR= line before doing the replacement. I'm by no means a sed guru, so if there is a way to get sed to do the replacement without outputting the rest of the file contents (I just made a few attempts), would you mind showing me how? Thanks.
Comment 18 Ciaran McCreesh 2004-12-04 15:40:19 UTC
How about one of these?

sed -n -e 's/^PORTDIR=\("[^\"]\+"\|\S\+\)/\1/1p'  /etc/make.globals /etc/make.conf | tail -n 1

sed -n -e '/^PORTDIR=/ { s/^[^=]\+=\("[^"]\+\|\S\+\).*/\1/p ; q }'  /etc/make.conf /etc/make.globals

Disclaimer: regexes produced when I'm moderately intoxicated may induce severe headaches.
Comment 19 Aaron Walker (RETIRED) gentoo-dev 2004-12-04 16:14:23 UTC
Yeah that's much better than what us mere mortals came up with ;p

/me hides from ciaranm's regexps.
Comment 20 Aron Griffis (RETIRED) gentoo-dev 2004-12-04 18:41:08 UTC
FWIW, this is what I use in my own stuff.  You can argue that the files aren't bash, but frankly, they source just fine.  And if they don't, this code uses portageq as a backup.

    eval $(
    { ( . /etc/make.globals 2>/dev/null
        . /etc/make.conf    2>/dev/null
        export PORTDIR PORTAGE_TMPDIR PORTDIR_OVERLAY
        export; )
      export # defs in environment override configuration
    } | sed -n '
      s/^declare -x PORTDIR=/portdir=/p
      s/^declare -x PORTAGE_TMPDIR=\(.*\)/tmpdir=\1\/portage/p
      s/^declare -x PORTDIR_OVERLAY=\(.*\)/overlay=\1/p')
    if [[ -z $portdir ]]; then
        echo "Warning: Fast portageq failed, resorting to calling python" >&2
        portdir=$(portageq envvar PORTDIR)
        tmpdir=$(portageq envvar PORTAGE_TMPDIR)/portage
        overlay=$(portageq envvar PORTDIR_OVERLAY)
    fi
Comment 21 Zach Forrest 2004-12-04 20:33:54 UTC
I think the point about make.conf not being a bash file is that it is possible it may not be compatible at some future date. The above first tries sourcing the files and then, if that fails, uses sed, anyway. I'd rather just use sed from the beginning and not treat make.conf in a way it was not intended.

Thanks for the input, Ciaran. I think the second expression is the best choice as it only uses one command. Here is the timing for those interested:

zach@meta4:~ $ time sed -n -e 's/^PORTDIR=\("[^\"]\+"\|\S\+\)/\1/1p'  /etc/make.globals /etc/make.conf | tail -n 1
/usr/portage

real    0m0.025s
user    0m0.001s
sys     0m0.010s

zach@meta4:~ $ time sed -n -e '/^PORTDIR=/ { s/^[^=]\+=\("[^"]\+\|\S\+\).*/\1/p; q }'  /etc/make.conf /etc/make.globals
/usr/portage

real    0m0.006s
user    0m0.002s
sys     0m0.004s

I'll make the changes and commit them to CVS. Thanks, mortals and Superior Beings alike. :P

Aaron, let me know when you've committed the new code and I'll put together a release.
Comment 22 Zach Forrest 2004-12-04 21:11:35 UTC
Aaron, I changed your _portdir function to use Ciaran's sed command and committed it. Might as well forget the environment variable -- I like your solution. :) Add the new function when you have time and we'll get this thing out. Thanks, again.
Comment 23 Aaron Walker (RETIRED) gentoo-dev 2004-12-04 21:42:58 UTC
Ok, ekeyword completion added.
Comment 24 Zach Forrest 2004-12-06 00:40:32 UTC
I just added the latest release on SourceForge: gentoo-bashcomp-1.0_beta4.
Comment 25 Aaron Walker (RETIRED) gentoo-dev 2004-12-06 08:29:09 UTC
bash-completion-20041017-r3 is in cvs.  Thanks everybody.
Comment 26 Ciaran McCreesh 2004-12-28 08:19:15 UTC
*** Bug 75916 has been marked as a duplicate of this bug. ***