Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 352909 - app-portage/portpeek-2.0.9 crash with package from both tree and overlay
Summary: app-portage/portpeek-2.0.9 crash with package from both tree and overlay
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Mike Pagano
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-01-27 09:02 UTC by Sven Eden
Modified: 2011-06-17 19:01 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
patch to initialize current_package (portpeek.patch,1.29 KB, patch)
2011-02-06 02:09 UTC, Mike Pagano
Details | Diff
portpeek 2.0.10 (portpeek,38.55 KB, text/plain)
2011-05-24 18:15 UTC, Mike Pagano
Details
Portpeek 2.0.10 new version (portpeek,38.53 KB, text/plain)
2011-06-15 16:47 UTC, Mike Pagano
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sven Eden 2011-01-27 09:02:15 UTC
portpeek fails when trying to analyse packag.keywords with the following traceback:

(call: portpeek -arfd)

Analyzing line: =dev-util/boost-build-1.45.0::gentoo
Traceback (most recent call last):
  File "/usr/bin/portpeek", line 939, in <module>
    get_recursive_info(USER_CONFIG_PATH + "/package.keywords")
  File "/usr/bin/portpeek", line 360, in get_recursive_info
    get_info(filename)
  File "/usr/bin/portpeek", line 381, in get_info
    diffs_found = parse_line(line, filename)
  File "/usr/bin/portpeek", line 329, in parse_line
    print_output(info,portage.output.green("\n" + fields[0] + ": ") + portage.output.yellow("Not Installed"),current_package)
UnboundLocalError: local variable 'current_package' referenced before assignment

---

The reason for this trace seems to be the following:

I had tested Boost-1.45 via ebuilds placed in my local overlay. Now that Boost-1.45 is in the tree, the said line above unkeywords the version from the tree, which is not installed, but the version from my local overlay is still installed.

The output of eix is:

[I] dev-util/boost-build
     Available versions:  
        (0)     1.34.1 1.35.0-r1 1.35.0-r2
        (1.37)  ~1.37.0
        (1.39)  ~1.39.0
        (1.40)  ~1.40.0
        (1.41)  1.41.0
        (1.42)  1.42.0
        (1.45)  {M}(~)1.45.0
        {examples python}
     Installed versions:  1.45.0(1.45)[1](14:46:41 14.12.2010)(examples python)
     Homepage:            http://www.boost.org/doc/tools/build/index.html
     Description:         A system for large project software construction, which is simple to use and powerful.

[1] "SED-Local" /usr/local/portage

---

Expected behavior: State that the unkeyworded package is not installed and add it to the list to removable entries.
Comment 1 Sven Eden 2011-01-27 09:47:45 UTC
After fixing boost-build, the same behavior occured with media-libs/mesa-7.10, that was installed from x11 overlay and is now available from the main tree:


---
Analyzing line: =media-libs/mesa-7.10::gentoo
Traceback (most recent call last):
  File "/usr/bin/portpeek", line 939, in <module>
    get_recursive_info(USER_CONFIG_PATH + "/package.keywords")
  File "/usr/bin/portpeek", line 360, in get_recursive_info
    get_info(filename)
  File "/usr/bin/portpeek", line 381, in get_info
    diffs_found = parse_line(line, filename)
  File "/usr/bin/portpeek", line 329, in parse_line
    print_output(info,portage.output.green("\n" + fields[0] + ": ") + portage.output.yellow("Not Installed"),current_package)
UnboundLocalError: local variable 'current_package' referenced before assignment
---
sed-notebook /home/sed # eix -e mesa
[I] media-libs/mesa
     Available versions:  7.7.1 7.8.2 7.9 ~7.9.1 (~)7.10 [M]~9999[1] {+classic d3d debug +gallium gles kernel_FreeBSD llvm motif +nptl pic selinux video_cards_i810 video_cards_i915 video_cards_i965 video_cards_intel video_cards_mach64 video_cards_mga video_cards_nouveau video_cards_r100 video_cards_r128 video_cards_r200 video_cards_r300 video_cards_r600 video_cards_radeon video_cards_savage video_cards_sis video_cards_sunffb video_cards_svga video_cards_tdfx video_cards_via video_cards_vmware +xcb}                                                                             
     Installed versions:  7.10[1](10:08:21 10.01.2011)(classic gallium gles llvm motif nptl video_cards_intel -d3d -debug -kernel_FreeBSD -pic -selinux -video_cards_mach64 -video_cards_mga -video_cards_nouveau -video_cards_r128 -video_cards_radeon -video_cards_savage -video_cards_sis -video_cards_tdfx -video_cards_via -video_cards_vmware)                                                                                               
     Homepage:            http://mesa3d.sourceforge.net/
     Description:         OpenGL-like graphic library for Linux

[1] "x11" /var/lib/layman/x11
---
Comment 2 Mike Pagano gentoo-dev 2011-01-31 15:50:55 UTC
what's in package.keywords?
Comment 3 Sven Eden 2011-02-02 21:33:34 UTC
(In reply to comment #2)
> what's in package.keywords?
> 

I hope you mean the two issues reported, as my package.keywords is rather large. (Naturally on a dev laptop)

Before this happened, I changed

=media-libs/mesa-7.10::SED-Local to
=media-libs/mesa-7.10::gentoo

(SED-Local is the reponame of my usr/local/portage repository)

One detail might be, that I *deleted* the ebuilds from my local overlay. So if portpeek looks into what is installed, and finds an installation from an overlay where the installed ebuild is absent, might this be the cause?
Comment 4 Mike Pagano gentoo-dev 2011-02-06 02:09:04 UTC
Created attachment 261622 [details, diff]
patch to initialize current_package

Hi Sven, I cant reproduce, but I hope this might help.

Can you patch your portpeek with this patch?
Comment 5 Sven Eden 2011-02-15 13:57:44 UTC
(In reply to comment #4)
> Created an attachment (id=261622) [details]
> patch to initialize current_package
> 
> Hi Sven, I cant reproduce, but I hope this might help.
> 
> Can you patch your portpeek with this patch?
> 

Sorry for the late reply!

I have no ebuilds in my local Overlay to install and then to move to gentoo repo to. I' have to forge a situation like the one that led to the said crash, and will report back once I tested your patch.

Unfortunately it won't be until next week, and I found another issue:

portpeek does not like wildcards in package.use:

---------------------------
package.use:
Traceback (most recent call last):
  File "/usr/bin/portpeek", line 951, in <module>
    get_recursive_info(USER_CONFIG_PATH + "/package.use")
  File "/usr/bin/portpeek", line 360, in get_recursive_info
    get_info(filename)
  File "/usr/bin/portpeek", line 384, in get_info
    diffs_found = parse_package_use(line,filename)
  File "/usr/bin/portpeek", line 420, in parse_package_use
    package_exists = portage.portdb.xmatch("match-all", check_pkg)
  File "/usr/lib64/portage/pym/portage/dbapi/porttree.py", line 788, in xmatch
    mydep = dep_expand(origdep, mydb=self, settings=self.settings)
  File "/usr/lib64/portage/pym/portage/dbapi/dep_expand.py", line 33, in dep_expand
    mydep = Atom(mydep, allow_repo=True)
  File "/usr/lib64/portage/pym/portage/dep/__init__.py", line 1089, in __init__
    raise InvalidAtom(self)
portage.exception.InvalidAtom: app-text/texlive-*
---------------------------

I have these lines in my package.use, which work well with portage:

app-text/texlive-* doc examples
dev-tex/latex-* doc examples
dev-texlive/texlive-* doc examples

I need doc and examples on everything [la]tex related, but didn't want to have a line per package in my package.use. (I added them so portpeek works just fine again. But maybe support for wildcarding might be added?)
Comment 6 Sven Eden 2011-05-11 09:10:29 UTC
(In reply to comment #4)
> Created attachment 261622 [details, diff]
> patch to initialize current_package
> 
> Hi Sven, I cant reproduce, but I hope this might help.
> 
> Can you patch your portpeek with this patch?

Hi again, I finally have a situation where I can test it. First, the current error looks like this:

----------------------------------------
Analyzing line: =kde-base/kdelibs-4.6.3*::gentoo
Traceback (most recent call last):
  File "/usr/bin/portpeek", line 939, in <module>
    get_recursive_info(USER_CONFIG_PATH + "/package.keywords")
  File "/usr/bin/portpeek", line 360, in get_recursive_info
    get_info(filename)
  File "/usr/bin/portpeek", line 381, in get_info
    diffs_found = parse_line(line, filename)
  File "/usr/bin/portpeek", line 243, in parse_line
    stable = check_for_stable_release(current_package)
  File "/usr/bin/portpeek", line 551, in check_for_stable_release
    if not is_pkg_package_masked(str(pkg.cpv)):
  File "/usr/bin/portpeek", line 512, in is_pkg_package_masked
    raise KeyError("CPV %s does not exist" % cpv)
KeyError: 'CPV kde-base/kdelibs-4.6.3 does not exist'
----------------------------------------
kdelibs-4.6.3 does not exist but is installed. The tree now knows only kdelibs-4.6.3-r1 which is unkeyworded by the analyzed line.

=====================

I have applied your patch and the result is, that it fails earlier on a line that does not cause a problem with the unpatched portpeek:

----------------------------------------
Analyzing line: =kde-base/akonadi-4.4.10::gentoo
Traceback (most recent call last):
  File "/usr/bin/portpeek", line 942, in <module>
    get_recursive_info(USER_CONFIG_PATH + "/package.keywords")
  File "/usr/bin/portpeek", line 363, in get_recursive_info
    get_info(filename)
  File "/usr/bin/portpeek", line 384, in get_info
    diffs_found = parse_line(line, filename)
  File "/usr/bin/portpeek", line 332, in parse_line
    print_output(info,portage.output.green("\n" + fields[0] + ": ") + portage.output.yellow("Not Installed"),current_package)
  File "/usr/bin/portpeek", line 748, in print_output
    if (package.is_overlay()):
AttributeError: 'str' object has no attribute 'is_overlay'
----------------------------------------
Comment 7 Sven Eden 2011-05-11 10:14:57 UTC
Note:

After I upgraded kdelibs from the missing kde-base/kdelibs-4.6.3 to the now available kde-base/kdelibs-4.6.3-r1, portpeek finished without problems.
Comment 8 Sven Eden 2011-05-20 09:17:34 UTC
Okay, it has problems if the version installed is no longer in the tree:

Analyzing line: =kde-base/kdm-4.6.3*::gentoo
Traceback (most recent call last):
  File "/usr/bin/portpeek", line 939, in <module>
    get_recursive_info(USER_CONFIG_PATH + "/package.keywords")
  File "/usr/bin/portpeek", line 360, in get_recursive_info
    get_info(filename)
  File "/usr/bin/portpeek", line 381, in get_info
    diffs_found = parse_line(line, filename)
  File "/usr/bin/portpeek", line 243, in parse_line
    stable = check_for_stable_release(current_package)
  File "/usr/bin/portpeek", line 551, in check_for_stable_release
    if not is_pkg_package_masked(str(pkg.cpv)):
  File "/usr/bin/portpeek", line 512, in is_pkg_package_masked
    raise KeyError("CPV %s does not exist" % cpv)
KeyError: 'CPV kde-base/kdm-4.6.3 does not exist'
Comment 9 Mike Pagano gentoo-dev 2011-05-24 18:15:10 UTC
Created attachment 274523 [details]
portpeek 2.0.10

Can you try this version of portpeek and tell me how far it gets.
Comment 10 Sven Eden 2011-05-26 08:58:04 UTC
Unfortunately it fails with the first line:

===========================================
 # portpeek -ard

package.keywords:
Analyzing line: =app-cdr/isomaster-1.3.7-r1::gentoo
Traceback (most recent call last):
  File "/usr/bin/portpeek", line 940, in <module>
    get_recursive_info(USER_CONFIG_PATH + "/package.keywords")
  File "/usr/bin/portpeek", line 360, in get_recursive_info
    get_info(filename)
  File "/usr/bin/portpeek", line 381, in get_info
    diffs_found = parse_line(line, filename)
  File "/usr/bin/portpeek", line 235, in parse_line
    keywords = "%s" % (get_keywords(current_package,"KEYWORDS").split())
  File "/usr/bin/portpeek", line 147, in get_keywords
    filtered_keywords = filter_keywords(keywords)
  File "/usr/bin/portpeek", line 530, in filter_keywords
    key = str.replace(key, "[", "")
TypeError: descriptor 'replace' requires a 'str' object but received a 'unicode'
===========================================

But a note: The original error only occurs with *slotted* packages. As an example: I have kdelibs-4.6.3-r1 installed. I renamed the ebuild to read 4.6.3-r2 and portpeek-2.0.9 fails on the line unkeywording the installed -r1 version, which is then no longer available in the tree.
Comment 11 Sven Eden 2011-05-26 08:59:56 UTC
Note: I am using Python-2.7 if that is relevant. And I forgot to mention, that portpeek-2.0.9 hasn't got any problem with said first line.
Comment 12 Mike Pagano gentoo-dev 2011-06-15 16:47:43 UTC
Created attachment 277135 [details]
Portpeek 2.0.10 new version

Can you try this version of portpeek and tell me the results?

Mike
Comment 13 Sven Eden 2011-06-16 20:21:51 UTC
I have moved the installed kde-base/kdelibs/kdelibs-4.6.3-r2.ebuild to kde-base/kdelibs/kdelibs-4.6.3-r3.ebuild and re-digested.

The new portpeek does the following:

-------------------------
Analyzing line: =kde-base/kdelibs-4.6.3*::gentoo

Installed: kde-base/kdelibs-4.6.3-r2  Keywords [u'~amd64'] : /etc/portage/package.keywords
Checking package: kde-base/kdelibs-4.6.2-r3.pkgmask is 0
Checking package: kde-base/kdelibs-4.6.4.pkgmask is 4
Checking package: kde-base/kdelibs-4.6.3-r3.pkgmask is 0
-------------------------

*YAI* it works!

Great! 8^D
Comment 14 Mike Pagano gentoo-dev 2011-06-16 21:26:32 UTC
Thanks for your patience and testing. (Especially, your patience.)

I'll put a portpeek 2.0.10 out within a few days.
Comment 15 Mike Pagano gentoo-dev 2011-06-17 19:01:32 UTC
portpeek 2.0.10 released