Fixed + enhanced version based on v0.2 from http://www.stacken.kth.se/~foo/gentoo/ I would suggest to integrate this functionality into emerge. e.g. emerge --distclean Reproducible: Always
Created attachment 276856 [details] v0.3 of distclean based on v0.2 from http://www.stacken.kth.se/~foo/gentoo/
Created attachment 276858 [details] v0.3 of distclean based on v0.2 from http://www.stacken.kth.se/~foo/gentoo/
Does this have advantages over eclean-dist from gentoolkit?
Hi, Thanks for your answer! - To my shame I have to admit, I didn't know about this tool :/ But you asked about the advantages. --> I just used it and here is my conclusion about the advantages [++] and disadvantages [--] of the distclean.py over eclean-dist: ++ it is much faster ++ the output is more focused on the task at hand... Maybe I missed something, but: a) distclean.py tells you that it could not determine the fetchlist for 2 of my installed packages, eclean-dist did NOT mention this :/ b) even with "eclean-dist -q" [== be as quiet as possible] it was telling me about all the broken ebuilds [most of them I did not even have installed]... IMHO this does not make sense: If one wants to just cleanup his DISTDIR -- the colored output is nicer with eclean-dist I am not sure who defines what goes in gentoolkit, what in portage. And I guess this becomes highly debatable at this point :) My original line of reasoning that led to the suggestion to integrate distclean into "emerge" was: emerge is reason why DISTFILES fills up --> emerge should have an option to clean it up. Like "depclean". Emerge is the reason why you need depclean. But at the same time I can understand [and agree to], if someone would argue: emerge is all related to EMERGING/UNEMERGING --> cleaning up DISTFILES is not related to that --> "distclean" is not part of "emerge" but gentoolkit. --> I guess at this point the purpose of this ticket would change to a list of enhancement suggestions for eclean-dist: (a) become faster [because at leas without any more parameters disclean.py and eclean-dist do the same things, except that distclean.py is a lot faster] (b) tell about problems to get the fetchlist for existing packets [or how does eclean-dist get this information for installed packages where there is no ebuild anymore available?] (c) by default be silent about all the ebuild problems ... and possibly have an option to show them if wanted
I've been busy so had not commented on this till now. distclean.py was created long ago when there wasn't really a definitive way to clean it. There were several that came into being about that time. Eclean being the one that stuck around. Eclean is slower mainly for doing a lot more to preserve files you wish to protect, along with more features. I just completed a major re-write/modularization so that it now has a good api for apps to use and embed into other apps. It will also make it easier to maintain and add enhancements, new types of checks. I plan to add multiple client support to it for use when you share your distfiles among a group of machines. > I guess at this point the purpose of this ticket would change to a list of > enhancement suggestions for eclean-dist: > (a) become faster [because at leas without any more parameters disclean.py > and eclean-dist do the same things, except that distclean.py is a lot faster] There is an open bug for improving the speed of eclean which is something we are going to work on for -0.3.1. Bug 346467 > (b) tell about problems to get the fetchlist for existing packets [or how > does eclean-dist get this information for installed packages where there is no > ebuild anymore available?] This came up after the rewite, before the final release. It was decided that if the ebuild was no longer in the tree, re-installing it would be difficult (unless you know what your doing) and therefore should be a candidate for cleaning. You should run eclean with the -p, --pretend option first to check that it isn't going to clean something you want to preserve, otherwise it will be cleaned. > (c) by default be silent about all the ebuild problems ... and possibly > have an option to show them if wanted Those ebuild problems you mention are most likely portage complaining about them, not eclean. Look at the info it spits out to see where it came from. Those type of errors are internal to portage. Eclean uses portage for a lot of it's information gathering. It could be possible to silence them and or record them as an option.
Thanks a lot for the [long] answer! For the speed and rewrite: Sounds all very good :) > (b) tell about problems to get the fetchlist for existing packets I understand your line of reasoning, but would personally like to have an option to keep these files - if this is possible. > (c) by default be silent about all the ebuild problems ... and possibly > have an option to show them if wanted Here I would like to stick too then is the suggestion of really having an option to silence the portage messages. To make this more clear: Here is some of the output I am getting and I would not like to see [by default or at least with -q]: d d d d d d d d d d d d d d d d d d d * * "/mnt/vola/sd/layman/pentoo/net-analyzer/wapiti/wapiti-9999.ebuild": * Deprecation Warning: Usage of distutils.eclass in packages not supporting installation * for multiple Python ABIs in EAPI <=2 is deprecated and will be banned on 2011-06-01. * The ebuild needs to be fixed. Please report a bug, if it has not been already reported. * d * ERROR: net-dialup/umtsmon-0.10_pre9999 failed (depend phase): * qt3.eclass could not be found by inherit() * * Call stack: * ebuild.sh, line 2047: Called source '/mnt/vola/sd/layman/jokey/net-dialup/umtsmon/umtsmon-0.10_pre9999.ebuild' * umtsmon-0.10_pre9999.ebuild, line 10: Called inherit 'eutils' 'qt3' 'cvs'1 * ebuild.sh, line 1386: Called die * The specific snippet of code: * [ ! -e "$location" ] && die "${1}.eclass could not be found by inherit()" * * If you need support, post the output of 'emerge --info =net-dialup/umtsmon-0.10_pre9999', * the complete build log and the output of 'emerge -pqv =net-dialup/umtsmon-0.10_pre9999'. * This ebuild is from an overlay: '/mnt/vola/sd/layman/jokey/' * S: '/dev/shm/portage/net-dialup/umtsmon-0.10_pre9999/work/umtsmon-0.10_pre9999' d /usr/lib64/portage/bin/ebuild.sh: line 2047: /mnt/vola/sd/layman/sunrise/net-wireless/remuco/remuco-0.9.5.ebuild: Permission denied * ERROR: net-wireless/remuco-0.9.5 failed (depend phase): * error sourcing ebuild * * Call stack: * ebuild.sh, line 2047: Called die * The specific snippet of code: * source "$EBUILD" || die "error sourcing ebuild" * * If you need support, post the output of 'emerge --info =net-wireless/remuco-0.9.5', * the complete build log and the output of 'emerge -pqv =net-wireless/remuco-0.9.5'. * This ebuild is from an overlay: '/mnt/vola/sd/layman/sunrise/' * S: '/dev/shm/portage/net-wireless/remuco-0.9.5/work/remuco-0.9.5' So there are the "d " that could go away, the WARNINGS [Dependency] and the ERRORS [on source ebuild or in the "depend phase"...].
I have never seen those "d"'s before. Neither has anything like that been reported before. To me those look like a keyboard malfunction, possibly a sticky "d" key. Also those warnings and errors are from portage's code. zmedico, can those types of warnings and errors be suppressed by setting a portage settings variable? Or perhaps re-directing stderr?
(In reply to comment #7) > Also those warnings and errors are from portage's code. Yeah, that's what happens if you try to query SRC_URI from a broken ebuild. Those messages can be quite useful if you want to actually correct the problem. > zmedico, can those types of warnings and errors be suppressed by setting a > portage settings variable? Or perhaps re-directing stderr? The messages should already be going to stderr, at least the die messages. If there's any other output that doesn't go to stderr then it likely comes from the ebuild or an eclass that it inherits, and in that case it would be best to fix the ebuild or eclass to direct errors/warnings to stderr. Portage doesn't provide any way to suppress this stuff. Generally, this stuff shouldn't be ignored anyway. As said, the messages are useful for correcting the problem. If you just want ignore them then sending stderr to /dev/null would be a reasonable approach.
... if I may: I think "d" stands for dependency. I usually know these from emerge. I AM NOT SUE, but this is how I think it [might] work: If there is new package/ebuilds that emerge did never "source" before it prints the "d" for each new ebuild... if this would be true and if my assumption that your code sources ALL ebuilds that are available in portage and layman [at least in my version 0.4.1] then this would explain why I see all the problems I don't need to see ;) emerge -q suppresses the "d"s beside other noise. "Those messages can be quite useful if you want to actually correct the problem." For the maintainer of these packages I can see that, but for me [and any regular user]: (a) these are problems in overlays I don't own (b) not even emerge showed them to me (c) most if not all are related to packages I don't even have installed (!) That's why I believe that these messges should be hidden by default.
SUE = SURE :) otherwise: Who is SUE? :)
(In reply to comment #9) > ... if I may: > I think "d" stands for dependency. I usually know these from emerge. > I AM NOT SUE, but this is how I think it [might] work: If there is new > package/ebuilds that emerge did never "source" before it prints the "d" for > each new ebuild... if this would be true and if my assumption that your code > sources ALL ebuilds that are available in portage and layman [at least in my > version 0.4.1] then this would explain why I see all the problems I don't need > to see ;) > > emerge -q suppresses the "d"s beside other noise. I'm pretty sure that the "d" is not coming from directly from portage, since I've never seen it before. It may be that you have something in /etc/portage/bashrc that prints the "d" when the bashrc is sourced. > "Those messages can be quite useful if you want to actually correct the > problem." > > For the maintainer of these packages I can see that, but for me [and any > regular user]: > (a) these are problems in overlays I don't own > (b) not even emerge showed them to me > (c) most if not all are related to packages I don't even have installed (!) > > That's why I believe that these messges should be hidden by default. Still, those message are an symptom of unhealthy overlays. What's happening is that eclean checks the SRC_URI of all ebuilds when the --destructive option is not used. Since the SRC_URI of those broken ebuilds can't be determined, the corresponding distfiles can't be protected from removal, so eclean-dist can't do it's job as well as it's designed to. That's an error condition, which deserves some kind of error message. Not being the maintainer of those overlays, you have a few possible course of action besides fixing the overlays yourself: a) Use the eclean --destructive option, so that it won't try to check SRC_URI of ebuilds that aren't installed, thus avoiding the error. b) Sync the overlay, in case the breakage has already been fixed upstream. c) Report a bug to the overlay maintainer if it hasn't been fixed yet. d) Remove the overlay from your system if you don't use it (this will make eclean-dist run faster when you don't have the --destructive option enabled). e) Run eclean-dist 2>/dev/null in order to ignore error messages. That said, I'd be happy to provide a way for portage API consumers to retrieve the error message via an exception instead of having it printed directly to stderr. Capturing of the ebuild output could easily be implemented in the EbuildMetadataPhase class.
Thanks for your answer! I think your list of possibilities is quite complete :) ----- You wrote: "eclean checks the SRC_URI" What does this mean? Like it checks if the URL is valid or it get just the filesize via HTTP (I think that is possible)? because eclean-dist does not download all the SRC_URI, right? ;) ----- The "d"s...: that is funny! I am using gentoo since dunno ... about 7 years now ... lets just say: A WHILE :) ... and VERY early I adopted the /etc/portage/bashrc from a friend... and in there is indeed: if [ "$EBUILD_PHASE" == "depend" ]; then echo -n "d " >&2 hehe :) So sorry about that and thanks for clarifying! ----- I can live with your options even though I still don't feel like taking care of several overlays - I already *DO* commit a lot to bugs.gentoo.org :) [and I synced yesterday the last time ;]
> You wrote: "eclean checks the SRC_URI" > What does this mean? it gets the SRC_URI from the ebuild and determines the distfile name from it. sometimes it has to follow the renaming sequence in the ebuild which prevents filename conflicts and filenames that do not follow decent naming/versioning schemes. That is something that distclean.py does not do. It just does a quick and dirty method of getting the filenames which can miss filename changes in the ebuilds. That is something that came into being after distclean.py was created. Zac. as long as errors like that are sent to stderr, I will get that all set up in the public api similar to what I did for laymans new api. It would be great if portage could keep things separated like that so stdout, stderr, stdin can be redifined in the public api for the api consumers. It is quite easy to setup a capture method for stderr so the output can be logged, etc.
