euse shows a USE flag as disabled even though it is enabled for a given package in /etc/portage/package.use. Reproducible: Always Steps to Reproduce: 1. echo "media-video/ffmpeg ssse3" >> /etc/portage/package.use 2. euse -i ssse3 Actual Results: local use flags (searching: ssse3) ************************************************************ [- ] ssse3 (media-video/ffmpeg): Expected Results: local use flags (searching: ssse3) ************************************************************ [+ ] ssse3 (media-video/ffmpeg): Portage 2.1.6.7 (default/linux/amd64/2008.0/desktop, gcc-4.3.3, glibc-2.9_p20081201-r1, 2.6.28-gentoo-r1 x86_64) ================================================================= System uname: Linux-2.6.28-gentoo-r1-x86_64-Intel-R-_Core-TM-2_CPU_6300_@_1.86GHz-with-glibc2.2.5 Timestamp of tree: Mon, 16 Feb 2009 04:45:01 +0000 distcc 3.1 x86_64-pc-linux-gnu [disabled] app-shells/bash: 3.2_p48-r1 dev-java/java-config: 1.3.7-r1, 2.1.7 dev-lang/python: 2.5.4-r2 dev-python/pycrypto: 2.0.1-r6 dev-util/cmake: 2.6.2 sys-apps/baselayout: 2.0.0 sys-apps/openrc: 0.4.3-r1 sys-apps/sandbox: 1.3.7 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.19.1 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.28-r1 ACCEPT_KEYWORDS="amd64 ~amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=core2" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -pipe -march=core2" DISTDIR="/home/portage/distfiles" FEATURES="collision-protect distlocks fixpackages parallel-fetch prelink protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://ftp.iinet.net.au/pub/Gentoo " LANG="en_AU.UTF-8" LC_ALL="en_AU.UTF-8" LDFLAGS="-Wl,-O1" LINGUAS="en_AU.UTF-8 en_AU en_GB.UTF-8 en_GB" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/home" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage/layman/sunrise /usr/local/portage" SYNC="rsync://rsync.au.gentoo.org/gentoo-portage" USE="X a52 aac aalib acpi alsa amd64 ao bash-completion berkdb blas bluetooth branding bzip2 cairo cddb cdr cli cracklib crypt curl dbus dia djvu doc dvd dvdr dvdread emboss encode exif expat fam ffmpeg fftw firefox flac fontconfig fortran gd gdbm gif gnome gnutls gpm graphviz gsl gstreamer gtk hal iconv icq imagemagick imlib ipod ipv6 isdnlog java javascript jpeg jpeg2k kde lapack ldap libnotify lm_sensors lzo mad matroska midi mikmod mmap mmx mng mp3 mpeg mplayer msn mudflap multilib mysql mysqli ncurses nls nptl nptlonly nsplugin ntfs offensive ogg openal opengl openmp pam pch pcre pdf perl png ppds pppd python qt3 qt3support qt4 quicktime readline reflection samba sdl session sharedmem speex spell spl sqlite sqlite3 sse sse2 ssl startup-notification subversion svg sysfs syslog tcpd theora threads tiff timidity truetype unicode usb vcd vorbis wavpack wxwindows xcomposite xml xorg xpm xscreensaver xulrunner xv xvid xvmc zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en_AU.UTF-8 en_AU en_GB.UTF-8 en_GB" USERLAND="GNU" VIDEO_CARDS="nvidia" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Thanks for reporting this issue, assigning to maintainers.
$ euse --help [snip] Notes: euse currently only works for global flags defined in make.globals, make.defaults or make.conf, it doesn't handle use.defaults, use.mask or package.use yet (see portage(5) for details on these files). It also might have issues with cascaded profiles. If multiple options are specified only the last one will be used. So not really a bug, just an unimplemented feature. I may work on this if I have some time later...
Ah yeah, I think I had read that before, but evidently forgot at the time when I noticed the 'bug'. It would be very nice to have it implemented, as the feature does feel incomplete without it. It sounds relatively easy to do, maybe I could have a crack at it for you?
(In reply to comment #3) > It sounds relatively easy to do, maybe I could have a crack at it for you? Definitely :) Not sure how familiar you are with these things, but do be aware that package.use can be either a text file or a directory containing arbitrarily named files. Read man portage section /etc/portage for more info.
Created attachment 182988 [details, diff] euse patch that searches /etc/portage/package.use for euse -i Here is a possible solution. I'm not the best bash programmer, but this seems to work well on my system and doesn't use any hacks per se. I'm using package.use directory. Can someone verify this works well for package.use file.
Created attachment 182989 [details, diff] Better implementation of package.use inclusion in euse -i Whoops. Previous patch gives false positives for global scope flag search hits
Created attachment 183014 [details, diff] Better-still implementation of package.use inclusion in euse -i Use flags are allowed to include dashes. My bad.
Created attachment 183015 [details, diff] Better-still implementation of package.use inclusion in euse -i I'm so sorry for the noise. I really hope this is of some use. This time I've actually included a patch, not a diff.
Created attachment 183081 [details, diff] Fixed a few bugs in original request and support other euse features Supports euse -a (view active flags) -- shows active flags by exception as found in package.use Supports euse -E (enable flag) -- use as euse -E {flag} {package} -- creates/edits appropriate file in package.use directory or creates/edits line in package.use file Supports euse -D (disable flag) -- use as euse -D {flag} {package} -- edits appropriate file in package.use directory or edits line in package.use file (euse -P not supported yet -- it should be hooked into current prune method to remove said use flag from all packages in package.use[/] I would think) Supports euse -I (I think)
I think the output of the script should probably add that the use flag is enabled/disabled in package.use: for instance, Expected Results: local use flags (searching: ssse3) ************************************************************ [+ P] ssse3 (media-video/ffmpeg): perhaps use 'P' for _p_ackage.use
Hey Jared, thanks so much for working on this! Looks good so far. I agree that a 'P' would be suitably informative. A few issues that I've noticed: (1) -I: no such file prison bin # euse.new -I ssse3 global use flags (searching: ssse3) ************************************************************ no matching entries found local use flags (searching: ssse3) ************************************************************ /usr/bin/euse.new: line 66: cd: /usr/portage/profiles/default/linux/amd64/2008.0/desktop/.. ../../../../../targets/desktop: No such file or directory /usr/bin/euse.new: line 68: cd: : No such file or directory [+ ] ssse3 (media-video/ffmpeg): faster floating point optimization for SSSE3 capable chips (Intel Core 2 and later chips) (2) -a: Not ignoring lines that are comments. :) pango [+ ] (#) pch [+ C ] pch [+ C ] (#) (3) and this might not be a bug, I'm not sure, but the -I output differs to the -i output in this respect: local use flags (searching: qt3) ************************************************************ no matching entries found ---versus--- local use flags (searching: qt3) ************************************************************ [+ D ] qt3 (net-print/hplip): Enable graphical interface using Qt 3 (recommended); when both qt3 and qt4 USE flags are enabled then qt3 is prioritary over qt4
Created attachment 183439 [details, diff] Fixed two bugs, added more stuff Dude! Most excellent testing skills hast thou, Jeremy! Thanks so much! (1) Apparently exists in the stock euse. I was able to fix it. (2) People actually comment in these files? hehe. My bad. Fixed it. (3) Do you have hplip installed? I get the same output, but don't have hplip installed Extras: -1- Supports -E and -D with packages with -p|--package switch so multiple use flags is possible: euse -p media-libs/ffmpeg -E mmx ssse3 -2- Beginings of support for package wildcards, so: euse -p =media-libs/ffmpeg-0.4.9* -E ssse3 -or- euse -p >=x11-libs/qt-4* -D qt3support is possible. Wildcards are supported in searching too (-i, -a, -I). Comments are preserved in affected files (thanks, Jeremy) -3- Pruning (-P) is now supported, so euse -P ssse3 will remove the ssse3 use flag from make.conf and all files in package.use[/] -4- [+ P] -- P/p added to output of searching ------ I think euse needs some color (like equery), too
Created attachment 183504 [details, diff] Couple typo and bug fixes I'm a perfectionist. (And I'm having a great time)
Great, I can confirm that those bugs are fixed, and I'll give you a slightly more detailed example of (3), in which I do have vlc but do not have hplip installed. The difference is that -i shows the local USE flag for packages that are not installed. The question is, should it? The man page does not clearly define the intended purpose here. Douglas? prison bin # euse -i qt4 global use flags (searching: qt4) ************************************************************ [+ D ] qt4 - Adds support for the Qt GUI/Application Toolkit version 4.x local use flags (searching: qt4) ************************************************************ [+ D ] qt4 (media-video/vlc): Builds a x11-libs/qt based frontend. It is now the most up-to-date graphical interface available. [+ D ] qt4 (net-print/hplip): Enable graphical interface using Qt 4 (experimental); when both qt3 and qt4 USE flags are enabled then qt3 is prioritary over qt4 prison bin # euse -I qt4 global use flags (searching: qt4) ************************************************************ [+ D ] qt4 - Adds support for the Qt GUI/Application Toolkit version 4.x Installed packages matching this USE flag: dev-util/cmake-2.6.2-r1 app-text/poppler-bindings-0.10.4 app-doc/doxygen-1.5.8 local use flags (searching: qt4) ************************************************************ [+ D ] qt4 (media-video/vlc): Builds a x11-libs/qt based frontend. It is now the most up-to-date graphical interface available. (4) -a: Not strictly a bug... but the first "p" just doesn't look right. This might lead into the bigger question of contrasting -I, -i and -a (with no arguments). crypt [+ D p] crypt [- D p] (sys-apps/hal) (5) -I: Only shows the first search term in the "searching: " header. Actual results are correct. This is a bug in the stock euse. prison bin # euse -I blas fftw qt3 global use flags (searching: blas) (6) -I: When used with no arguments gives mangled output. Bug in stock euse. prison bin # euse.old -I global use flags (searching: *) ************************************************************ [+ CD ] # $Header: /var/cvsroot/gentoo-x86/profiles/use.desc,v 1.409 2009/02/28 10:14:17 maekke Exp $ 3dfx - Enable support for Voodoo chipsets, also called as 3DFX and TDFX 3dnow - Adds support for 3dnow multimedia processor instructions Great work, Jared, looking forward to the next one. :)
Created attachment 184367 [details, diff] Fixed described bugs (4), (5), and (6) (3) I don't think I understand the output/intention of -I completely. I think that -i should definately show USE flag information for packages not installed on you system, though. (4) I agree it looks fishy and incorrect. Fixed. (5) Simple fix. Good find! (6) The output of -I [*] is almost useless it's so lengthy, but the stock euse definately does not output correctly for global scope. Fixed.
If you're still enjoying this, I just realised that there is some more work required to make the local USE flag feature complete, because USE flags can be turned on/off in the ebuild. See the IUSE line in the media-video/mplayer-20090226.28734 ebuild for a good example.
Interesting. Do you think we should add another check/report to the software, so it might show something like?: -------------------------------------- | Unspecified in package.use and make.conf -------------------------------------- $ euse -i faac Actual Results: local use flags (searching: faac) ************************************************************ [- ] faac (media-video/ffmpeg): Use external faac library for AAC encoding Expected Results: local use flags (searching: faac) ************************************************************ [+ B] faac (media-video/ffmpeg): Use external faac library for AAC encoding -------------------------------------- | Enabled in package.use -------------------------------------- $ euse -E faac --package media-video/mplayer; euse -i faac Expected Results: local use flags (searching: faac) ************************************************************ [+ PB] faac (media-video/ffmpeg): Use external faac library for AAC encoding Where {B|b| } is show for 'B' to indicate faac is enabled by default in the ebuild. (B=default enable, b=default disabled, ' '=considered but not defaulted) How should this be reflected between package version ebuilds? -- Perhaps only the one which would currently be installed (given (un)masks and keywords) should be queried? $ ACCEPT_KEYWORDS="~x86" euse -i faac Expected Results: local use flags (searching: faac) ************************************************************ [+ B] faac (=media-video/ffmpeg-20090226.28734): Use external faac library for AAC encoding --- Or maybe a semi-exhaustive list --- Expected Results: local use flags (searching: faac) ************************************************************ [- ] faac (media-video/ffmpeg): [+ B] faac (=media-video/ffmpeg-20090226.28734): Use external faac library for AAC encoding
Copy/pasting hazard... Please replace 'media-video/ffmpeg' with 'media-video/mplayer' in the previous post.
Another thought/question: How should the software deal with global use flags enabled in ebuild IUSE variables, such as "+X" in media-video/ffmpeg-20090226.28734 ?
I think you have the right idea, but I think the semi-exhaustive list example is too verbose. Whether it should specify the version or not is a good question. It probably doesn't have to, because if the -i output says that flag X is enabled in the ebuild for package Y, then of course it's talking about the currently installed version. I think. I'm at work using a Mac <cringe> right now, so I can't test it. (In reply to comment #19) > Another thought/question: How should the software deal with global use flags > enabled in ebuild IUSE variables, such as "+X" in > media-video/ffmpeg-20090226.28734 ? Hmmm. If it changes the default value, then I think we need to list it under local flags. So for example the recent mplayer would get a bunch of new entries because it enables flags that are disabled by default. But aren't there some flags that are global and local? I'm suddenly confused... the Mac is hurting my brain, I'll get back to you later.
Created attachment 190800 [details, diff] euse patch to enable package.use file/folder and ebuild flags Took the first stab at supporting flags enabled or disabled by default in ebuilds. Also added support to the -E and -D flags for individual packages in package.use when flags are enabled or disabled in corresponding ebuilds. Sorry for the long delay -- I've had a slew of computer issues for the past month. According to euse documentation, we still have use.defaults, use.mask, and package.use.mask to go.
Cool, thanks. Do you use ~arch? I know we started back on 0.2.4.2-r1, but 0.3.0_rc5 is in portage now and euse has changed slightly. It's probably worth keeping up to date with it.
Created attachment 192995 [details, diff] euse patch to enable package.use file/folder and ebuild flags for euse 0.3.0 Patched for euse 0.3.0 I don't know of any regression tests for euse, so I don't know if I may have broken something fixed between 0.2.4.2 and 0.3.0
Created attachment 193030 [details, diff] Re-ported to 0.3.0_rc6 and squashed some bugs Apparently I didn't test the previous patch at all. When I sat down to look at implementing use.force and use.mask support, I realized it was pretty buggy. I fixed the bugs I introduced and re-fixed bug (6) from this post. Disabling a flag enabled in an ebuild is possible: greezy@baconator ~/proj/euse $ euse -i ass global use flags (searching: ass) ************************************************************ no matching entries found local use flags (searching: ass) ************************************************************ [- ] ass (media-video/mplayer): Internal SRT/SSA/ASS (SubRip / SubStation Alpha) subtitle support greezy@baconator ~/proj/euse $ ./euse -i ass global use flags (searching: ass) ************************************************************ no matching entries found local use flags (searching: ass) ************************************************************ [+ B] ass (media-video/mplayer): Internal SRT/SSA/ASS (SubRip / SubStation Alpha) subtitle support greezy@baconator ~/proj/euse $ sudo ./euse -D ass --package media-video/mplayer Adding media-video/mplayer:-ass use flag in /etc/portage/package.use greezy@baconator ~/proj/euse $ ./euse -i ass global use flags (searching: ass) ************************************************************ no matching entries found local use flags (searching: ass) ************************************************************ [- pB] ass (media-video/mplayer): Internal SRT/SSA/ASS (SubRip / SubStation Alpha) subtitle support
Cool, thanks for keeping up the work. I'm not getting the same output as you, though: if I have "ass" in package.use then I get the correct output: [+ PB] ass (media-video/mplayer): if I have "-ass" in package.use then I get this: [- pb] ass (media-video/mplayer): and if I remove any reference to it then I get this: [+ ] ass (media-video/mplayer): So it seems like my package.use entry is somehow determining the reported ebuild status. I noticed that you're calling python for a few things, and you're probably right that a lot of this string processing would be easier to write in it. I'm just concerned that calls to python are slowing the script down noticeably, and may also introduce new complexity given different versions of it. Also, at what point shall we try to get these changes into the actual package? We already have some very useful improvements, so it might be worth submitting them now rather than waiting until we finish everything. I've kind of lost track of our milestones, if we had any clearly defined to begin with. :)
Created attachment 194335 [details, diff] Whoops. Think it works now. I would agree that using pure bash, if possible, would perhaps be faster; however, I think that the call to emerge is taking far longer than the python calls. On top of that, to implement ebuild USE flag searches, I added a call to equery, which also takes a while. So I revamped the logic that determines the +/- of whether the flag is enabled. Before it would check the USE= output of 'emerge --info', but that doesn't account for ebuilds and package.use, so I had some other strange logic to (patheticly and incorrectly) handle it. Instead, I'm now using a scientific approach using the USE flag precedence (environment, package.use, make.conf, ...). This way I can avoid the call to 'emerge --info' except for one place that needs it. So I just call it in that one instance rather than every time. This speeds euse up by about 25% per query on my machine (5.0s from 6.7s). My efforts on this are definitely in hopes in starting a footprint in the Gentoo community. I'm cool with putting off merging this into the mainline euse for any stretch of time as long as there is promise that this work is relevant and will be used eventually. I certainly appreciate Jeremy's efforts toward the QA of my code.
Yep, that works now and it is noticeably quicker. There are still some weird things going on with the ebuild flag in showdesc, though. Firstly it's because different version ebuilds of the same package can have different local flags. Take the example of the local use flags "gtk" and "gmplayer" for media-video/mplayer: [+ D b] gtk (media-video/mplayer): Build gmplayer, a GTK+ MPlayer gui (UNSUPPORTED) [- ] gmplayer (media-video/mplayer): Build gmplayer, a GTK+ MPlayer gui (UNSUPPORTED) The version of mplayer that I have installed only has "gmplayer" in the ebuild, so this output is confusing. My feeling is that local use flags will have to be listed per package version, with the installed version highlighted. (Although the description of the local flag is still just per ebuild.) This also makes me wonder if we should take a step back and ask ourselves what the purpose of 'euse -i' vs 'equery h' is, not to mention the other flag related functions. Equery focuses on installed packages whereas euse is about the flags in general; the extra information that 'euse -I' provides beyond '-i' just duplicates 'equery h', so I wonder if '-I' could be removed entirely? Then consider the output of 'equery u' for a package like mplayer: would that feature be better served if it left out the flag descriptions and origin of the flag and just focused on the true/false condition of whether the flag is enabled (much like eix)? Perhaps the descriptions are useful there, but the flag status is primitive compared with what euse gives. Regardless (or perhaps in light) of the above contemplations, I think making 'euse -i' list local flags per package version is the way to go. What do you think?
(In reply to comment #17) > How should this be reflected between package version ebuilds? -- Perhaps only > the one which would currently be installed (given (un)masks and keywords) > should be queried? > > $ ACCEPT_KEYWORDS="~x86" euse -i faac > > Expected Results: > local use flags (searching: faac) > ************************************************************ > [+ B] faac (=media-video/ffmpeg-20090226.28734): > Use external faac library for AAC encoding > > > --- Or maybe a semi-exhaustive list --- > > Expected Results: > local use flags (searching: faac) > ************************************************************ > [- ] faac (media-video/ffmpeg): > [+ B] faac (=media-video/ffmpeg-20090226.28734): > Use external faac library for AAC encoding > Is something like this what you mean by listing by package version? Should we provide a switch to enable per-version output? I agree that euse -I does not offer anything not already provided by equery
Yes, sorry, I did recall that you had suggested something similar before, I wasn't meaning to take credit for it. I've only just now realised the necessity of it. The purpose of 'euse -i' given by the program is to "show descriptions for the given useflags", which is per package and does not depend on whether the package is installed. The complication is that the flag status is based on several things and are per ebuild. Now, if someone just wanted the description of a local flag, it's not that hard to do 'grep :FLAG /usr/portage/profiles/use.local.desc', so it's the added value of the flag status that makes euse so useful. So, even though displaying local flag status per ebuild will generate considerably more information than the existing functionality, it is justified because that is what makes euse useful. Now that I've rationalised it out loud to myself, and you, I'm going to have lunch and come back with a draft output format. :)
OK, here's an idea of what information I think needs to be in the output. I've put in some formatting just to make the structure clear. The most important thing is that the [PB] flags are now per ebuild. I may have the size of the [box] in column 1 wrong, because I can't remember all the flags off the top of my head. What do you think? global use flags (searching: gmplayer gtk mudflap) ************************************************************ [+ D ] gtk - Adds support for x11-libs/gtk+ (The GIMP Toolkit) local use flags (searching: gmplayer gtk mudflap) ************************************************************ [+ D ] mudflap # tells you what global settings affect this local flag sys-devel/gcc: Add support for mudflap, a pointer use checking library [-pB] (4.0) 4.0.4 # Flags can be disabled for specific versions in [-pB] (4.1) 4.1.2 # package.use - tricky! [+ B] (4.2) 4.2.4-r1 [+ B] (4.3) 4.3.2-r3 [+ B] (4.3) 4.3.2-r4 [+ B] (4.3) 4.3.3-r2 # highlighted to show that it is installed [+ B] (4.4) 4.4.0 [- ] gmplayer media-video/mplayer: Build gmplayer, a GTK+ MPlayer gui (UNSUPPORTED) [- b] 1.0_rc2_p20090530 # highlighted to show that it is installed [+ D ] gtk # This just repeats the flags from the global section. app-misc/beagle: Enables the GTK+ Beagle Search UI for showing search results. This is the default GUI for Beagle. [+ ] 0.3.9 [+ ] 0.3.9-r1 media-video/mplayer: Build gmplayer, a GTK+ MPlayer gui (UNSUPPORTED) [+ b] 1.0_rc2_p28450 [+ b] 1.0_rc2_p20090322 # many more packages with local "gtk" flag...
Jared, what are your thoughts on the major change to the output I've suggested? Also of note is that 0.3.0_rc7 hit the portage tree just recently.
I think you're suggestion is the correct method. I've almost finished a prototype. June 3 I became a new father :). Time's been a little constrained this month, hehe.
Congratulations! I think you will find that time will be constrained for the next twenty years, but it's worth it. :) I am travelling overseas for a month now, so I won't be able to look at a Gentoo machine until the end of July, but I can still discuss ideas, etc. Cheers.
Created attachment 200549 [details, diff] Show per-version output of packages per USE flag query. Updated for gentoolkit 0.3.0_r7 This thing may be full of bugs, but I wanted to throw it out for comments. I think the installed version should perhaps be listed with a star (*), and the masked version shown with some other indication, just for consistency and readability.
I like what I have seen so far, though I haven't had time to test it thoroughly. I would be reluctant to add more information than necessary to this output since it's already long. More info will just make it less readable. Utilities like eix already provide that kind of dense package information, so I say we just focus on the USE flag stuff for now. Although... some colour like eix would make it seem more familiar, etc. :) One trivial change to make is with the version; it seems that the /etc/gentoolkit-version file is obsolete (and not found on my system). In the local use flags section, I don't think we need to repeat the flag name for each package, just once is sufficient. Hmmm, and a possible bug or something that I just don't understand: have a look in the output for 'euse -i webkit' at the packages gimp and liferea. Their latest version appears to have the flag enabled but it's not from the ebuild or package.use. Mysterious. Good work, it's really great to see that kind of detailed output! Here's a thought, it might be worth showing for a given ebuild that the local USE flag doesn't exist at all. So it's not simply disabled, but invalid. For example in gimp, the webkit flag appeared in version 2.6.*, so it would be informative to indicate that versions 2.3 and 2.4 don't have that flag. Cheers.
Created attachment 200725 [details, diff] Fixed a couple bugs, 100% speed increase, support package versions that do not support a specified USE flag euse on my system uses /usr/share/gentoolkit/VERSION, so I'm not sure why yours still uses /etc/... I would agree to show the use flag only once, except, consider querying two different use flags: 'euse -i ldap ssse3'. In this output, it might lost what goes with what use flag. I think this might also be the case for very long output, like 'euse -i webkit' I added version support for the ebuild; however the package.use will need some more work considering you can place a line in package.use like '<=net-misc/openssh-5.1 -ldap'. This will require some serious text comparing to output correctly. I ditched using equery and really sped up the program; however, not it does not report the SLOT correctly (check out 'euse -i mudflap' and look at gcc). I'm not sure how to indicate that a package verion is not using a specific flag, so the output looks like '[- ]' for one that is globally disabled, '[- b]' for one that is disabled in the ebuild, and ' - ' for one unused in the ebuild (assumed if the USE flag is not listed in IUSE). Perhaps relying on IUSE alone isn't seriously wise, considering it could be inherited or appended to. Cheers.
Created attachment 200742 [details, diff] Fixed a couple (more) bugs, removed unnecessary diffs. Sorry for the noise
Created attachment 200775 [details, diff] Significant speed optimization, fixed Gentoo bugs #181309, #231394, #274472, #275362 Even with detailed output, new euse is faster than the stock
Wow, it's certainly a lot faster! But it's also quite broken. :) I think it would be better to stick with the slower equery method for now; perhaps we or someone else can improve equery itself in the future. I've reverted to the version you posted on 2009-08-08, so that's what my comments are based on. You might also want to consider putting your own postfix version after the official one so that we can keep track of things more easily. Something like 0.3.0_rc7-JH2. Thinking about packages were some ebuilds don't have the local USE flag at all, such as mudflap, showing minus without square brackets could be misleading. Sometimes local flags are added in so that a feature can be disabled. If we show ebuilds without the flag as "-", then it's suggesting that it's the equivalent of disabling the flag, which it's not. So I think it should be left blank or use a different symbol altogether; I'm in favour of leaving it blank right now. Also, if we wanted to save some space, we could put all the ebuilds for a package that don't have the flag on one line, since they're all in the same basket. So, like this: [+ D] mudflap sys-devel/gcc: Add support for mudflap, a pointer use checking library (2.95) 2.95.3-r9, (2.95) 2.95.3-r10, (3.2) 3.2.2, (3.3) 3.3.6, (3.4) 3.4.6 [+ ] (4.0) 4.0.4 [+ ] (4.1) 4.1.2 [+ ] (4.2) 4.2.4-r1 [+ ] (4.3) 4.3.2-r3 [+ ] (4.3) 4.3.2-r4 [+ ] (4.3) 4.3.3-r2 [+ ] (4.3) 4.3.4 [+ ] (4.4) 4.4.1 Just an idea. Oh and yeah, the version file thing was just some leftover weirdness.
Well, the only extra things that equery brings to the table are -Masked status - Installation status - Slot However, the same reason the slot is messed up is the reason is the reason that the use flags (mudflap) for instant is not shown correctly: I'm scanning ebuild files which are themselves bash scripts that "inherit" other bash scripts. For instance, the mudflap use flag is defined for gcc in the inherited toolchain eclass. So, that said, since we aren't currently interested in the installation or masked status, I don't think it's worth bringing equery back for the slot information if I need to do more parsing to get the use flags correct anyway. I'm also not sure what you're considering broken, since it appears to display correct information (albeit incomplete). I'll post the actual euse that I'm currently working from. Perhaps there's some issue with the patches I'm posting. I agree that showing a '-' for undefined use flags is misleading; however, showing the packages in a bare list is too non-specific. Perhaps we could use like an HTTP style header: [+ D] mudflap sys-devel/gcc: Add support for mudflap, a pointer use checking library Undefined-For: (2.95) 2.95.3-r9, (2.95) 2.95.3-r10, (3.2) 3.2.2, (3.3) 3.3.6, (3.4) 3.4.6 [+ ] (4.0) 4.0.4 [+ ] (4.1) 4.1.2 [+ ] (4.2) 4.2.4-r1 [+ ] (4.3) 4.3.2-r3 [+ ] (4.3) 4.3.2-r4 [+ ] (4.3) 4.3.3-r2 [+ ] (4.3) 4.3.4 [+ ] (4.4) 4.4.1
Created attachment 201375 [details] euse-0.3.0_rc7-JH2 gentoolkit euse 0.3.0_rc7 with patches from this bug applied.
Sure, regarding equery, I trust your judgement, and what you say makes sense. I think because I happened to be using mudflap as my testing flag, it made the versions after 2009-08-08 look like a major regression. Just so I know we're seeing the same thing, this is the local mudflap output: [+ D ] mudflap sys-devel/gcc: Add support for mudflap, a pointer use checking library - 2.95.3-r10 - 2.95.3-r9 - (${MY_PV}) 3.1.1-r2 - 3.2.2 - (${GCC_BRANCH_VER}) 3.2.3-r4 - 3.3.6-r1 - 3.4.6-r2 - 4.0.4 - 4.1.2 - 4.2.4-r1 - 4.3.2-r3 - 4.3.2-r4 - 4.3.3-r2 - 4.3.4 - 4.4.1 So yes, otherwise it is functioning quite well. And yes, I agree that the list of ebuilds without the flag needs something. Oh, I did just discover a trivial bug. :) It's the old sorting of numbers vs characters thing: [- ] 64bit www-plugins/adobe-flash: For amd64-multilib, installs the native 64-bit plugin [+ B] 10.0.32.18 - 9.0.246.0 Thanks for finding the time to keep up the work on this!
Have any ideas on how to resolve the SLOT and IUSE variables if they are inherited and/or modifed from eclass modules? I (miserably) attempted to source the ebuild.sh script to load in the functions necessary to properly source the ebuild file, but it seems to wanna execute the ebuild.
Created attachment 203986 [details, diff] Fixed the version sorting issue Additionally, some euse bugs indicate euse should query local use flags from overlays. But I do not know of an overlay with local use flags defined. Does anybody know of one?
I know hardly anything about how the SLOT and IUSE variables work in ebuilds, sorry. I'll take a quick look and see if I can make sense of it. In regard to overlay local USE flags, I presume it's referring to "profiles/use.local.desc". The sunrise overlay has one, but x11 doesn't. They're the only overlays I have installed at the moment. Would you mind uploading the latest version again, but not as a patch, just as is? I don't think we need to trouble ourselves with patches, it's just extra confusion.
Created attachment 204266 [details] Fixed the version sorting issue Yes, I believe I mean the use.local.desc. I'll try the sunrise overlay. Thanks again
With respect to the SLOT/IUSE issue, I would be inclined to ask the eix or equery developers how they did it or look directly at their code. Which equery function were you previously using to get that data?
I was getting slot from equery -N list --portage-tree --exclude-installed "${2}" where ${2} is the qualified package name (e.g. www-client/elinks) I agree their code may be a good jumping off point for how to do this. -Jared
Created attachment 205351 [details] Fixed IUSE and SLOT issues Turns out that portage maintains a list of such items inside PORTDIR/metadata/cache/<cat>/<package>-<version>. Line 3 is SLOT and line 11 is IUSE. Perhaps I should use portage python to retrieve this; however, using it from python is significantly slower
Just wanted to say thanks, Jeremy, for your continued support with this bug. Reading back through my own posts, I think they could definitely be taken harshly, although I consider it a privilege and a joy to work on this issue. So, thanks again for helping me along the way. Some questions on more completeness: The euse help mentions that it does not support use.defaults (see man portage(5)). I cannot find use.defaults in /etc/portage or /usr/portage/profiles anywhere. It is also not listed anywhere in the portage man page. Am I missing something? Is this is misnomer? Also, since use.mask and use.force are globals, how might euse represent them? Should we just add them inside the brackets on the global line? I'm looking at supporting overlays. Should the software display distinctions between USE flags defined in the main tree and in overlays? If so, is the name/path of the overlay sufficient. Maybe display the name/path in parentheses after the global status and flag name: [+ D ] crazy-flag (/usr/local/portage/layman/sunrise) I've not implemented anything for the version of a package that do not support a given use flag. Do you think the HTTP style header is right? Or should I use something else? Is there a document for coding standards to make sure the code, once submitted, will be consistent with the rest of the Gentoo and Gentoolkit code? Again, thanks! Jared
(In reply to comment #49) > Created an attachment (id=205351) [edit] > Fixed IUSE and SLOT issues > > Turns out that portage maintains a list of such items inside > PORTDIR/metadata/cache/<cat>/<package>-<version>. Line 3 is SLOT and line 11 is > IUSE. Perhaps I should use portage python to retrieve this; however, using it > from python is significantly slower > Great discovery about the metadata! The output for mudflap looks perfect. However, some other output doesn't seem to be working: I just tried java, nocxx and vim-syntax (arbitrarily), and there is no output for them at all. :(
(In reply to comment #50) > Just wanted to say thanks, Jeremy, for your continued support with this bug. > Reading back through my own posts, I think they could definitely be taken > harshly, although I consider it a privilege and a joy to work on this issue. > So, thanks again for helping me along the way. No worries, it's a pleasure to be working on something and seeing results! > Some questions on more completeness: > > The euse help mentions that it does not support use.defaults (see man > portage(5)). I cannot find use.defaults in /etc/portage or > /usr/portage/profiles anywhere. It is also not listed anywhere in the portage > man page. Am I missing something? Is this is misnomer? Yup, that reference to "use.defaults" is weird. Maybe there was such a file in the past, or they meant something else. Ignore it. :) > Also, since use.mask and use.force are globals, how might euse represent them? > Should we just add them inside the brackets on the global line? Hmm, I'd never looked at use.mask before; it probably is worth showing that a flag is off because it's masked, but I'll need to look into it more. Let's make it a low priority. > I'm looking at supporting overlays. Should the software display distinctions > between USE flags defined in the main tree and in overlays? If so, is the > name/path of the overlay sufficient. Maybe display the name/path in parentheses > after the global status and flag name: > > [+ D ] crazy-flag (/usr/local/portage/layman/sunrise) Just the name in square brackets is the usual format for overlays, like [x11] or [sunrise]. I think that's sufficient, and yes, I think it's worthwhile to label flags that are from overlays. Could get messy with local flags... > I've not implemented anything for the version of a package that do not support > a given use flag. Do you think the HTTP style header is right? Or should I use > something else? So, at the moment it looks like this, to use an extract from mudflap: - (3.4) 3.4.6-r2 [+ ] (4.0) 4.0.4 What if we made it look like this instead: [ ] (3.4) 3.4.6-r2 [+ ] (4.0) 4.0.4 I'd prefer not to get all wordy with "not defined", etc. I think just an empty status area is appropriate. It probably won't be immediately clear what it means... but so long as we explain it in the documentation (which I volunteer to write), then I think it's fine. > Is there a document for coding standards to make sure the code, once submitted, > will be consistent with the rest of the Gentoo and Gentoolkit code? Not that I know of. > Again, thanks! And thank you!
Created attachment 206046 [details] Fix global flag issue (java, nocxx, ...). Add support for use.force and use.mask. Add support for overlays Perhaps this is all for feature implementations. I'll browse back through the bugs for euse and see if there are any other easily-implementable features, but I think this is pretty complete. All that I know that it does not implement is package.use.mask and package.use.force. For the package versions a use flag is not defined for, I left the whole line blank, lemme know what you think. I think it's more clear than the [ ]; however, it may be less clear where a flag is only defined for one version of a long list, so lemme know what you think. Overlays are only indicated on the package version line, and for use flags that may be defined in both the overlay desc file and the main portage desc file, only one description will show up (the first overlay its found in I believe) Modifying package.use will detect if a use flag is masked and issue an error and exit Any color ideas? Should we kinda follow the equery colors? I don't think this is quite a release candidate. I wanna spend a little more time refactoring and cleaning up the code and adding more function documentation.
I knew I'd forget something: the portage documentation seems to indicate that metadata cache information can be stored in an sqlite database. So I don't know if relying on $PORTDIR/metadata/cache is a good idea project wide. Also, I added a error line if the $PORTDIR/metadata/cache directory is not found to run egencache --repo=xxx --update xxx is filled in for the repo in question. This may add to the confusion, too.
Created attachment 206051 [details] Implemented comment 1 from bug #104396, flags not defined in use.desc (overlays included) cannot be added to make.conf Also alert if attempting to add a masked flag to make.conf.
Created attachment 206094 [details] Fixed issue where not all version were displayed for local flag viewing Sorry for the noise
Created attachment 208863 [details] First possible release candidate implementing all new features and bug fixes As a recap, this version implements: - package.use as a file or folder both for USE flag status and editing - USE flag status defined in ebuild files is considered - use.mask and use.force flag status is considered - Overlays are searched for USE flags and packages - USE flag info is displayed based on package version It also implements bug fixes from: - bug #44219, bug #104396, bug #181309, bug #231394, bug #274472, bug #275362 It should probably be translated over to python, but I'd like to see these features get pushed out before starting over.
Hey Jared, sorry for the silence recently. I just gave the latest version a quick test and it looks good, however there is some kind of bug related to sed happening when checking for local use flags. prison ~ # euse -i gtk global use flags (searching: gtk) ************************************************************ [+ D ] gtk - Adds support for x11-libs/gtk+ (The GIMP Toolkit) local use flags (searching: gtk) ************************************************************ sed: -e expression #2, char 8: unterminated `s' command [+ D ] gtk I won't do any more comprehensive testing now since this sed problem might be interfering. Do you think it would be useful to add a --debug option so that it prints out everything it is doing? Or can I get that effect already via python somehow? Unfortunately, both my systems had simultaneous heart attacks a few weeks back, so I have reverted to stable. This probably won't cause any compatibility issues between our results, but it's just something to keep in mind. Cheers.
(In reply to comment #54) > I knew I'd forget something: the portage documentation seems to indicate that > metadata cache information can be stored in an sqlite database. So I don't know > if relying on $PORTDIR/metadata/cache is a good idea project wide. > > Also, I added a error line if the $PORTDIR/metadata/cache directory is not > found to run > > egencache --repo=xxx --update > > xxx is filled in for the repo in question. This may add to the confusion, too. > Just a curiosity -- even if the cache exists, how do we know that it is up to date? I.e., that the cache accurately reflects the currently installed portage tree? I just sense that there could be a danger in using cached information that is not reliably updated. But maybe we shouldn't care, and it's up to the user to maintain their caches.
With the sed issue, I haven't upgraded to bash 4.0 yet. I'll try that. I can certainly add some debugging information with the --debug switch if it would aid in debugging.
I just downgraded bash to 3.2 from 4.0 and it didn't help. Maybe it's the version of sed that you're using?
Created attachment 212955 [details] Fixed bug where long-running list of packages (-i gtk, for instance) would result in incorrect information The sed issue still remains ...
Created attachment 212957 [details] (Simple) Profiling version I've used for debugging and speed-ups Perhaps you can use this to help me track down where the sed issue is coming from. I've downloaded older attachments and they don't contain the error for me either. I'll try with some other Gentoo rigs tomorrow.
(In reply to comment #59) > (In reply to comment #54) > > I knew I'd forget something: the portage documentation seems to indicate that > > metadata cache information can be stored in an sqlite database. So I don't know > > if relying on $PORTDIR/metadata/cache is a good idea project wide. > > > > Also, I added a error line if the $PORTDIR/metadata/cache directory is not > > found to run > > > > egencache --repo=xxx --update > > > > xxx is filled in for the repo in question. This may add to the confusion, too. > > > > Just a curiosity -- even if the cache exists, how do we know that it is up to > date? I.e., that the cache accurately reflects the currently installed portage > tree? I just sense that there could be a danger in using cached information > that is not reliably updated. But maybe we shouldn't care, and it's up to the > user to maintain their caches. > So the cache for the main portage tree is managed with emerge --sync, so it should always be up-to-date with the tree. The overlays; however, do not manage their cache. (Some may, I don't know). It's only created/updated with the egencache command. There may be another way we could use if we detect the cache is unavailable or out-of-date. Perhaps I could scan through the egencache program to see what it does or see if it has an options to do just one ebuild -- exactly the one euse need at the time. I checked my sed version, it's 4.2
Yay, I fixed the sed problem! It was occuring in the elif section at the beginning of get_useflags(), presumably because our /etc/portage/package.use setups are different. Mine is a regular file, and I'm guessing that yours is a directory? There was an 's' and a couple of '/'s missing, I'm sure you'll see where. I'll try to get around to making some validation tests one of these days.
Created attachment 213875 [details] Fixed the sed bug that was occuring for me. So I called it version "JJ0" since they're our first name initials and everything good starts counting at zero. :) The versioning doesn't really matter, it's just for helping us keep track. Like I said the other day, I think it's working really well, so maybe it's time to ask some other developers to beta test it and give us some feedback? (Other devs reading this, please go ahead.)
Have you tried modifying you package.use with the -E and -D flag and --package switch? It's the only feature that is unique to euse.
Watch out, though. As a warning, euse doesn't back up your package.use before editing. It probably should; however, it's difficult to do so if its a directory.
No, I haven't used those -E and -D commands before because I prefer to edit by hand. I just had a quick look then and there seems to be a problem with -D because it is searching for a USE flag "-foo" when of course it should be searching for "foo". Remind me, how do you debug a bash script? All I know to do is to add "set -x" at the beginning to see the commands echoed, but it's so cumbersome to wade through. Backing up a directory, hmm... simplest solution would be to just recursively copy the whole directory to the backup name, i.e. "cp -r dir dir.backup", rather than worrying about which individual file in the directory was changed. Individual file backups would be nice in the long run.
any progress on new versions of euse? euse --version euse (0.3.0_rc10-r1)
Hi, sorry, looks like Jared and I are both busy with other stuff these days. I don't want our work to be wasted, though, so I would like to see it go into the mainline euse code. I still use our final version every day, and I think as it stands it works perfectly well for the features that we tested, but is probably broken for -E and -D as mentioned previously. I have time to design and test, but I am too unfamiliar with shell scripts to do the code in any reasonable amount of time.
Could you give a snippet of your configuration, the commands you gave with -E and -D, the output given and the expected output so I can test it?
Created attachment 258232 [details] Fixed a couple of bugs related to the --package -E and -D flags I changed my package.use over to a file and I think was able to reproduce your problem with not detecting the USE flags correctly. Try it again. I'll try real soon to see the diffs between euse-0.3.0_rc7 and rc10 and make sure I've integrated those fixes/features as . Sorry for the long silence
Created attachment 258291 [details] euse_0.3.0_rc11 Latest copy of the script with the changes from gentoolkit-0.3.0_rc11 merged into place. The only request I have before I commit all of your hard work into the repository is that the call to perl be converted to either a sed or python call. This is the call in question (line 383): perl -pne "s:$portdir/${1}/${pkg}-(([^.]|\.(?!e))+)\.ebuild:\1:g"
Jared, great to hear from you again! I'm really pleased that you're still interested in finishing off our euse revamp -- I really hope that everyone else will find it as useful as I/we do. I'm testing the useflag modification features as we speak, and an idea has come to mind: * An -R option that Removes the useflag, effectively restoring to default for that flag. If it's the last flag for a package in package.use, then it removes the package entry altogether. If adding this feature would prevent us from meeting the deadline for 0.3.0 then I suggest we postpone it to 0.3.1. One issue has also come to mind (from Jared's 0.3.0-rc7_jh-r2 version): * When modifying a local useflag (using -p), if there is no such useflag then a warning is issued, but the useflag is still modified. Is that the correct behaviour? Should it instead be an error and no action taken? That would prevent accidental mistyping of useflags, and the user can always modify non-existent useflags at their own discretion. On a related note, when attempting to modify a global useflag, even though an error is issued, it still modifies make.conf with no changes and makes a backup. It would be better if it did not modify and backup at all unless actual changes are made. Great to be back on the horse! :) Cheers.
Created attachment 258556 [details] Made the adjustments as requested I'm really excited that the work might actually be released! As Paul requested, I reworked the perl like, so it uses sed now The remove feature already exists, it's called "prune" with the -P option. It wasn't plumbed up right for specifying a package, so I fixed it. You can do euse -P flag --package app-categort/package To reset the useflag for the given package, wherever it may be found in package.use[/], or even euse -P flag to reset the flag for all packages in package.use[/] and make.conf Should we rename it? I like "remove", "reset", and "erase" better than prune, also, the "-x" option is commonly used to perform such a task. I actually exit now if an error is displayed when adding a global use flag that is masked or doesn't exist. Not backup is made, and make.conf is not edited. I originally added the error messages for informational purposes, leaving the user responsible, rather than a buggy program preventing the user from doing what is desired. If we're confident that software is correct, I don't see a problem with it refusing to enable/disable use flags that don't exist or cannot be used. So it will refuse to add a use flag to a package that doesn't exist. However, I'm not 100% sure that it can detect the use flags used by the package exactly correct (since they may vary between versions and the software doesn't necessarily care which version you are configuring), so I kept the warning for use flags not used by the requested package. It will warn you and continue to enable or disable it for the given package. I would like to have my name added to the help / man page, and we should change the copyright to 2011. Cheers, Jared Hancock
(In reply to comment #76) > > The remove feature already exists, it's called "prune" with the -P option. It Uhh... whoops. I don't like to change things without good reason, but I admit that I don't really like "prune" whereas "remove" (or "unset") sounds better. Not sure if it's worth breaking backward compatibility for. I've found a couple of bugs with -E/-D: 1) Disabling an enabled or unset flag searches for "-flag" as the actual flag name and results in, for example: ERROR: Use flag "-oss" is not defined in use.desc and should not be added to make.conf. 2) Disabling a local flag is removing it. I've also noticed that -P is pruning all local flags too... I don't really agree with this behaviour and it's different to how -E and -D work. Being able to remove all flag references with something like -P * would be nice, but I think otherwise -P should work as expected, like -E and -D. Cheers.
The changes are in my local repository. I have added Jared as an author to the header of the script and the man page. Here is the wording that I used: euse: # Author: Marius Mauch <genone@gentoo.org> # Jared Hancock (Signigicant rewrite for package.use support) euse.1: Sigificant rewrite for package.use support by Jared Hancock Let's decide what to do with prune / remove. Personally, I like having the option named remove and allowing prune as an option for backwards compatibility. And let's fix the -flag issue at a miniumum
This has now been committed to the git repository: http://git.overlays.gentoo.org/gitweb/?p=proj/gentoolkit.git;a=commit;h=b62586ede6c2716be976a538d41fac836875ed05
Created attachment 258893 [details] Fixed error output from disabling global flags with -D. Added --remove option and left --prune as an alias Also, as requested, pruning a flag no longer removes it from local package.use unless an explicit package is given. Unfortunately, package wildcards are not yet supported, so the package must be given exactly as it is found in package.use. In my mind, I thought the "prune" feature was to obliterate to existance of use flag no matter where is was enabled. I see your perspective, too. Also emit a warning for invalid package atoms: euse -E ldap -p =net-misc/openssh euse -E ldap -p net-misc/openssh-5.6* will both fail now since they would be ineffective and give further warnings from portage later. euse also would add an extra empty line to make.conf every time it was invoked with -E or -D
Latest update pushed to git: http://git.overlays.gentoo.org/gitweb/?p=proj/gentoolkit.git;a=commit;h=93669a76e1674d123c9a3b816f8fc6a9b5a705c0 Also I fixed a bug where a flag would partially match a masked flag and error out. This is the diff of my fix: @@ -1092,7 +1103,7 @@ if [[ "${ACTIVE}" == "-" ]]; then # If flag is masked, it should be added to package.mask, instead # of package.use. For now, yield a warning and quit - if [[ -n $(echo " ${ACTIVE_FLAGS[6]}" | grep "$flag") ]]; then + if [[ -n $(echo " ${ACTIVE_FLAGS[6]}" | grep " -${flag}") ]]; then error "USE flag \"$flag\" is masked. Enabling in package.use will" \ "\nbe ineffective. You may have an incorrect profile selected." continue My test case is 'euse -E tk -p dev-vcs/git'
Wow. Good catch! We should look for a trailing space too so that use flags that start with another use flag won't be matched either: if [[ -n $(echo " ${ACTIVE_FLAGS[6]} " | grep " -${flag} ") ]]; then Should I suggest changes to the man page to include the new features?
I should be able to update the man page without too many problems. However, any suggestions are always welcome.
Hi! Apologies for the delay and terseness, but I'm travelling overseas at the moment. 1) Disable "flag" causes "-flag": fixed. 2) Disabling a local flag is removing/pruning it: NOT fixed. 3) Global -P also prunes local flags: fixed. 4) Non-existent global use flags modify make.conf: fixed for -E and -D but not for -P.
(In reply to comment #84) > 2) Disabling a local flag is removing/pruning it: NOT fixed. So I don't think I know what you mean exactly. Could you give me an example command line and what you would expect to happen and not happen?
(In reply to comment #85) > (In reply to comment #84) > > 2) Disabling a local flag is removing/pruning it: NOT fixed. > So I don't think I know what you mean exactly. Could you give me an example > command line and what you would expect to happen and not happen? > If, in package.use you have a package: foo/bar flag Then "euse -D flag -p foo/bar" should give: foo/bar -flag likewise, "euse -D hobbit -p foo/bar" should add "-hobbit" to the package: foo/bar flag -hobbit But in both cases, it is removing the flag instead. So either -D is accidentally calling the code for -R, or it is the same code as -R? There is another bug too, sorry. :) 5) When checking for package existence, it is not checking overlays. Also, I'm just looking at line 1,069, and I don't agree with changing the action from Remove to Disable because the flag is globally enabled. Let the user do exactly what they want, ours is not to try and second guess them. Cheers.
Created attachment 261774 [details] Fixes described bugs (In reply to comment #86) > (In reply to comment #85) > > (In reply to comment #84) > > > 2) Disabling a local flag is removing/pruning it: NOT fixed. > > So I don't think I know what you mean exactly. Could you give me an example > > command line and what you would expect to happen and not happen? > > > > If, in package.use you have a package: > > foo/bar flag > > Then "euse -D flag -p foo/bar" should give: > > foo/bar -flag > > likewise, "euse -D hobbit -p foo/bar" should add "-hobbit" to the package: > > foo/bar flag -hobbit > > But in both cases, it is removing the flag instead. So either -D is > accidentally calling the code for -R, or it is the same code as -R? Yeah, whoops. So this is a misconception on my part. I originally wrote the code to help the user, so that if a use flag was to be disabled that is already disabled in the package IUSE, make.conf, etc., it would simply remove the flag. The end result is equivalent; however, I agree that it is confusing. The code is simpler and far less error prone to just honor what the user is asking for. I'm minimalist in my thinking. This is the way I would manage my own files. Now that the software has support for -R, I can still get my way ;-) So it should perform as you expect now. However, if you disable an already-disabled flag or enable and already-enabled one, a warning will be emitted, although the software will still honor the request. > > > There is another bug too, sorry. :) > > 5) When checking for package existence, it is not checking overlays. Good catch. It should work now
Cool, that has fixed the issues I mentioned, thanks. Still a few little things that are not quite right: * Modifying a global flag always reports that make.conf was modified, even if it wasn't. On a related note, it's not warning about enabling an already-enabled flag or disabling an already-disabled flag, etc. I think you fixed this in a previous version, but evidently it's broken again. :) Like with local flags, it's probably worth listing each flag and what action was taken. For example, if the user tries to enable flags A, B and C in one command, but flag A is already enabled, then we should inform the user that A was already enabled (no action taken), B was enabled, and C was enabled. * Removing or disabling multiple local flags (in one command) is not working correctly, as per this output (enabling seems fine): prison ~ # euse-0.3.0-rc11_jh3 -E debug doc examples -p sys-auth/polkit Adding "sys-auth/polkit[debug]" use flag in "/etc/portage/package.use" Adding "sys-auth/polkit[doc]" use flag in "/etc/portage/package.use" Adding "sys-auth/polkit[examples]" use flag in "/etc/portage/package.use" prison ~ # euse-0.3.0-rc11_jh3 -R debug doc examples -p sys-auth/polkit Removing "sys-auth/polkit[debug]" use flag in "/etc/portage/package.use" Adding "sys-auth/polkit[-doc]" use flag in "/etc/portage/package.use" WARNING: USE flag "examples" is already enabled for sys-auth/polkit Adding "sys-auth/polkit[examples]" use flag in "/etc/portage/package.use" prison ~ # Granted, this is the first time that I have tried to modify multiple local flags in a single command. Removing the three flags individually worked fine. It looks like the first modification works OK but subsequent ones get confused. Cheers!
Created attachment 262303 [details] Fixes described issues It's now more vocal when handling global flags, and does not indicate make.conf was edited if it was not. Actually, more specifically, it does not modify make.conf unnecessarily. (As before, it modified make.conf every time regardless of whether or not it changed anything) Also fixed the issue with removing and disabling multiple flags for a package. Cheers
Latest changes have been pushed into the git repository.
Alas, something weird has just started happening! :( The same command with regular "euse" works fine. karaganda portage # euse-0.3.0-rc11_jh4 -i test global use flags (searching: test) ************************************************************ [- ] test - Workaround to pull in packages needed to run with FEATURES=test. Portage-2.1.2 handles this internally, so don't set it in make.conf/package.use anymore local use flags (searching: test) ************************************************************ [+E G M] : egrep: invalid option -- '[' Usage: egrep [OPTION]... PATTERN [FILE]... Try `egrep --help' for more information. ls: cannot access /usr/portage/metadata/cache/-*: No such file or directory [+E G M] : egrep: invalid option -- '[' Usage: egrep [OPTION]... PATTERN [FILE]... Try `egrep --help' for more information. ls: cannot access /usr/portage/metadata/cache/-*: No such file or directory
Do you have any overlays installed? Which ones if so?
(In reply to comment #92) > Do you have any overlays installed? Which ones if so? I did have, but even after I have removed them the errors continue.
(In reply to comment #91) > local use flags (searching: test) > ************************************************************ > [+E G M] > : > egrep: invalid option -- '[' > Usage: egrep [OPTION]... PATTERN [FILE]... > Try `egrep --help' for more information. > ls: cannot access /usr/portage/metadata/cache/-*: No such file or directory I see where and why this is happening. I just can't imagine how the situation would arise to trigger it. And really, you have USE="test" in you environment?! (In reply to comment #93) > (In reply to comment #92) > > Do you have any overlays installed? Which ones if so? > > I did have, but even after I have removed them the errors continue. Have you tried re-downloading the new euse script, just to make sure yours hasn't gotten corrupted?
I'm puzzled too. The good news is that it is only happening on my recently installed laptop, not on my desktop. So it may just be that I have done something weird to my laptop. Ahh... I just figured it out. :) On my laptop, in addition to the usual /etc/make.conf, I also use /etc/portage/make.conf. Our new euse script uses one or the other, but it needs to use both, letting the settings in /etc/portage/make.conf override the other. Solved!
Also, you might want to put some more basic precondition tests in the helper functions so that errors don't propagate. Cheers.
Could you send me you old /etc/portage/make.conf so I could trigger the bug on my end?
In theory, you should only have to create an empty file to trigger it, but here is mine: FEATURES="distcc ${FEATURES}" GENTOO_MIRRORS="http://ftp.iinet.net.au/pub/Gentoo " MAKEOPTS="-j6" But, here's the catch: my /etc/portage/make.conf is actually a symlink. I don't think it should matter, but I'm mentioning it just in case. Cheers.
Created attachment 265751 [details] Works if no overlays are defined It was indeed an issue reading make.conf. It turns out that if /etc/portage/make.conf existed, it would not read /etc/make.conf to look for PORTDIR_OVERLAY. The real bug what that the software would break like this if there were no overlays defined in PORTDIR_OVERLAY. So now it will prefer /etc/portage/make.conf over /etc/make.conf, although, it will actually read both when searching for PORTDIR_OVERLAY. In other words, it will read /etc/make.conf first, then /etc/portage/make.conf (if it exists). Then it will consider the value of $PORTDIR_OVERLAY. It will also work now if no overlay is defined. Cheers
Created attachment 265753 [details] Works if no overlays are defined I'm so sorry. I made one slight change and forgot to transfer it to my staging area.
OK, that fixes the issue, which is great. But... here's a complication, which is the scenario on my system: In /etc/make.conf I have my one and only USE= variable along with all the other typical variables. In /etc/portage/make.conf, there are just a few variables that I want to override or modify relating to networking. Because euse gives preference to /etc/portage/make.conf, it copies the USE= statement from /etc/make.conf into it upon modifying a global variable. Now to put it simply, I personally don't want that. :) It would be better, I think, to retain the location of the USE= statement. That is, if you find USE= in /etc/make.conf and not in /etc/portage/make.conf, then I would say keep the modifications there, because evidently that's where the user wants them. Cheers. Paul, when can we expect this to hit the main portage tree? Just curious.
(In reply to comment #101) > Paul, when can we expect this to hit the main portage tree? Just curious. I would like to get it out within the next week. That will depend on my real life workload though.
Created attachment 266759 [details] Prefers /etc/portage/make.conf when modifing global flags only if it define the USE variable Also fixes a bug where euse would refuse to add hyphenated use flags. So you can euse -E bash-completion Cheers
Created attachment 267621 [details] euse Latest version comitted in git. Same as previous with whitespace fixes to make it consistent.
Golly, now that this is released in gentoolkit-0.3.0, does that mean that we can actually mark it closed? It is no reason not to continue using this bug for discussion and improvement, but technically, it looks like we have achieved our objective. Jared, I offer you a virtual toast to our success! :)
