Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 569942 - sys-apps/portage-9999 portage/elog/mod_save.py", line 36, in process causes KeyError: 'CATEGORY'
Summary: sys-apps/portage-9999 portage/elog/mod_save.py", line 36, in process causes K...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS, REGRESSION
Depends on:
Blocks:
 
Reported: 2015-12-27 22:05 UTC by Brian Dolbec
Modified: 2016-06-08 15:06 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
ebuild and files causing the traceback (arduino.tar.bz2,2.72 KB, application/x-bzip)
2015-12-27 22:14 UTC, Brian Dolbec (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Dolbec (RETIRED) gentoo-dev 2015-12-27 22:05:09 UTC
professor-x portage # emerge --version
!!! Repository 'gentoo-guis' has sync-type attribute, but is missing sync-uri attribute
Portage 2.2.26_p47 (python 3.4.3-final-0, default/linux/amd64/13.0/desktop, gcc-5.3.0, glibc-2.22-r1, 4.1.8-vanilla-workstation-3 x86_64)
professor-x portage # emerge -va arduino
!!! Repository 'gentoo-guis' has sync-type attribute, but is missing sync-uri attribute

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N F   ] dev-java/oracle-jdk-bin-1.8.0.66:1.8::gentoo  USE="alsa awt cups fontconfig -derby -doc -examples -javafx -jce -nsplugin -pax_kernel (-selinux) -source" 177,039 KiB
[ebuild  N     ] virtual/jdk-1.8.0:1.8::gentoo  0 KiB
[ebuild  N     ] dev-java/ant-core-1.9.2::gentoo  USE="-doc -source" 0 KiB
[ebuild  N     ] virtual/jre-1.8.0:1.8::gentoo  0 KiB
[ebuild  N     ] dev-java/rxtx-2.2_pre2:2::gentoo  USE="-doc -lfd -source" 0 KiB
[ebuild  N     ] dev-java/ant-nodeps-1.9.2::gentoo  0 KiB
[ebuild  N     ] dev-java/jna-3.4.1::gentoo  USE="awt nio-buffers -doc -source {-test}" 0 KiB
[ebuild  N     ] dev-embedded/arduino-1.6.7::gentoo-guis  USE="-doc -examples" 0 KiB

Total: 8 packages (8 new), Size of downloads: 177,039 KiB
Fetch Restriction: 1 package (1 unsatisfied)

Fetch instructions for dev-java/oracle-jdk-bin-1.8.0.66:
 * 
 * Oracle requires you to download the needed files manually after
 * accepting their license through a javascript capable web browser.
 * 
 * Download the following files:
 *   jdk-8u66-linux-x64.tar.gz
 * at 'http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html'
 * and move them to '/var/db/distfiles'
 * 
 * If the above mentioned urls do not point to the correct version anymore,
 * please download the files from Oracle's java download archive:
 * 
 *    http://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html#jdk-8u66-oth-JPR
 * 

Would you like to merge these packages? [Yes/No] y
>>> Verifying ebuild manifests
>>> Emerging (1 of 8) dev-java/oracle-jdk-bin-1.8.0.66::gentoo
>>> Failed to emerge dev-java/oracle-jdk-bin-1.8.0.66, Log file:
>>>  '/var/log/portage/dev-java:oracle-jdk-bin-1.8.0.66:20151227-215641.log'
>>> Jobs: 0 of 8 complete, 1 failed                 Load avg: 0.60, 0.26, 0.21
*** Resuming merge...

These are the packages that would be merged, in order:

Calculating dependencies... done!

Total: 0 packages, Size of downloads: 0 KiB
================================================================================================
======== Traceback for invalid call to portage.package.ebuild.config.config.__getitem__ ========
  File "./bin/emerge", line 50, in <module>
    retval = emerge_main()
  File "/home/brian/Dev/git/portage/pym/_emerge/main.py", line 1174, in emerge_main
    return run_action(emerge_config)
  File "/home/brian/Dev/git/portage/pym/_emerge/actions.py", line 3232, in run_action
    emerge_config.args, spinner)
  File "/home/brian/Dev/git/portage/pym/_emerge/actions.py", line 501, in action_build
    retval = mergetask.merge()
  File "/home/brian/Dev/git/portage/pym/_emerge/Scheduler.py", line 1071, in merge
    if not self._calc_resume_list():
  File "/home/brian/Dev/git/portage/pym/_emerge/Scheduler.py", line 1927, in _calc_resume_list
    portage.elog.elog_process(pkg.cpv, settings)
  File "/home/brian/Dev/git/portage/pym/portage/elog/__init__.py", line 176, in elog_process
    m.process(mysettings, str(key), mod_logentries, mod_fulllog)
  File "/home/brian/Dev/git/portage/pym/portage/elog/mod_save.py", line 36, in process
    cat = mysettings['CATEGORY']
  File "/home/brian/Dev/git/portage/pym/portage/package/ebuild/config.py", line 2570, in __getitem__
    return self._getitem(key)
  File "/home/brian/Dev/git/portage/pym/portage/package/ebuild/config.py", line 2623, in _getitem
    raise KeyError(mykey)
KeyError: 'CATEGORY'
================================================================================================
 * emerge --keep-going: virtual/jre-1.8.0 dropped because it requires
 * =virtual/jdk-1.8.0*
Traceback (most recent call last):
  File "./bin/emerge", line 50, in <module>
    retval = emerge_main()
  File "/home/brian/Dev/git/portage/pym/_emerge/main.py", line 1174, in emerge_main
    return run_action(emerge_config)
  File "/home/brian/Dev/git/portage/pym/_emerge/actions.py", line 3232, in run_action
    emerge_config.args, spinner)
  File "/home/brian/Dev/git/portage/pym/_emerge/actions.py", line 501, in action_build
    retval = mergetask.merge()
  File "/home/brian/Dev/git/portage/pym/_emerge/Scheduler.py", line 1071, in merge
    if not self._calc_resume_list():
  File "/home/brian/Dev/git/portage/pym/_emerge/Scheduler.py", line 1927, in _calc_resume_list
    portage.elog.elog_process(pkg.cpv, settings)
  File "/home/brian/Dev/git/portage/pym/portage/elog/__init__.py", line 176, in elog_process
    m.process(mysettings, str(key), mod_logentries, mod_fulllog)
  File "/home/brian/Dev/git/portage/pym/portage/elog/mod_save.py", line 36, in process
    cat = mysettings['CATEGORY']
  File "/home/brian/Dev/git/portage/pym/portage/package/ebuild/config.py", line 2570, in __getitem__
    return self._getitem(key)
  File "/home/brian/Dev/git/portage/pym/portage/package/ebuild/config.py", line 2623, in _getitem
    raise KeyError(mykey)
KeyError: 'CATEGORY'
professor-x portage # 


Reproducible: Always
Comment 1 Brian Dolbec (RETIRED) gentoo-dev 2015-12-27 22:14:51 UTC
Created attachment 420964 [details]
ebuild and files causing the traceback

This is a straight bump of a sabayon ebuild for arduino-1.6.6-r1.

Added for easy reproduce-ability.  Although I guessing so far the reason is due to the first pkg immediately failing before the 'CATEGORY' is actually set.
Comment 2 Brian Dolbec (RETIRED) gentoo-dev 2015-12-27 22:21:00 UTC
The primary trigger would be the fetch restricted files not having being downloaded and placed in the distfiles dir.

Still it's an area of code that should be fixed to not traceback.
Comment 3 Brian Dolbec (RETIRED) gentoo-dev 2015-12-28 00:00:55 UTC
hmm, and another one...  Is this recent breakage?  Or am I just getting lucky...


brian@professor-x ~/Dev/git/gentoo-guis/dev-embedded/arduino $ repoman manifest
!!! Repository 'gentoo-guis' has sync-type attribute, but is missing sync-uri attribute
>>> Downloading 'http://distfiles.gentoo.org/distfiles/arduino-1.6.7-src.tar.gz'
--2015-12-27 15:57:20--  http://distfiles.gentoo.org/distfiles/arduino-1.6.7-src.tar.gz
Resolving distfiles.gentoo.org... 216.165.129.135, 137.226.34.42, 140.211.166.134, ...
Connecting to distfiles.gentoo.org|216.165.129.135|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2015-12-27 15:57:20 ERROR 404: Not Found.

>>> Downloading 'https://arduino.googlecode.com/files/arduino-1.6.7-src.tar.gz'
--2015-12-27 15:57:20--  https://arduino.googlecode.com/files/arduino-1.6.7-src.tar.gz
Resolving arduino.googlecode.com... 173.194.203.82, 2607:f8b0:400e:c05::52
Connecting to arduino.googlecode.com|173.194.203.82|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2015-12-27 15:57:20 ERROR 404: Not Found.

!!! Couldn't download 'arduino-1.6.7-src.tar.gz'. Aborting.
================================================================================================
======== Traceback for invalid call to portage.package.ebuild.config.config.__getitem__ ========
  File "/usr/lib/python-exec/python3.4/repoman", line 37, in <module>
    repoman_main(sys.argv[1:])
  File "/usr/lib64/python3.4/site-packages/repoman/main.py", line 104, in repoman_main
    qatracker, can_force = scanner.scan_pkgs(can_force)
  File "/usr/lib64/python3.4/site-packages/repoman/scanner.py", line 240, in scan_pkgs
    if self.manifester.run(checkdir, self.portdb):
  File "/usr/lib64/python3.4/site-packages/repoman/checks/ebuilds/manifests.py", line 39, in run
    mysettings=self.repoman_settings, myportdb=portdb)
  File "/usr/lib64/python3.4/site-packages/portage/package/ebuild/digestgen.py", line 148, in digestgen
    spawn_nofetch(myportdb, myebuild)
  File "/usr/lib64/python3.4/site-packages/portage/proxy/objectproxy.py", line 31, in __call__
    return result(*args, **kwargs)
  File "/usr/lib64/python3.4/site-packages/portage/package/ebuild/_spawn_nofetch.py", line 71, in spawn_nofetch
    defined_phases = settings['DEFINED_PHASES'].split()
  File "/usr/lib64/python3.4/site-packages/portage/package/ebuild/config.py", line 2570, in __getitem__
    return self._getitem(key)
  File "/usr/lib64/python3.4/site-packages/portage/package/ebuild/config.py", line 2623, in _getitem
    raise KeyError(mykey)
KeyError: 'DEFINED_PHASES'
================================================================================================
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.4/repoman", line 37, in <module>
    repoman_main(sys.argv[1:])
  File "/usr/lib64/python3.4/site-packages/repoman/main.py", line 104, in repoman_main
    qatracker, can_force = scanner.scan_pkgs(can_force)
  File "/usr/lib64/python3.4/site-packages/repoman/scanner.py", line 240, in scan_pkgs
    if self.manifester.run(checkdir, self.portdb):
  File "/usr/lib64/python3.4/site-packages/repoman/checks/ebuilds/manifests.py", line 39, in run
    mysettings=self.repoman_settings, myportdb=portdb)
  File "/usr/lib64/python3.4/site-packages/portage/package/ebuild/digestgen.py", line 148, in digestgen
    spawn_nofetch(myportdb, myebuild)
  File "/usr/lib64/python3.4/site-packages/portage/proxy/objectproxy.py", line 31, in __call__
    return result(*args, **kwargs)
  File "/usr/lib64/python3.4/site-packages/portage/package/ebuild/_spawn_nofetch.py", line 71, in spawn_nofetch
    defined_phases = settings['DEFINED_PHASES'].split()
  File "/usr/lib64/python3.4/site-packages/portage/package/ebuild/config.py", line 2570, in __getitem__
    return self._getitem(key)
  File "/usr/lib64/python3.4/site-packages/portage/package/ebuild/config.py", line 2623, in _getitem
    raise KeyError(mykey)
KeyError: 'DEFINED_PHASES'
brian@professor-x ~/Dev/git/gentoo-guis/dev-embedded/arduino $
Comment 4 Zac Medico gentoo-dev 2015-12-28 00:16:15 UTC
(In reply to Brian Dolbec from comment #3)
> hmm, and another one...  Is this recent breakage?  Or am I just getting
> lucky...

It's all fallout from this commit:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=39d81c514c33733209963110d90ae88020ffcf7d
Comment 5 Zac Medico gentoo-dev 2015-12-28 02:16:51 UTC
(In reply to Brian Dolbec from comment #0)
>     m.process(mysettings, str(key), mod_logentries, mod_fulllog)
>   File "/home/brian/Dev/git/portage/pym/portage/elog/mod_save.py", line 36,
> in process
>     cat = mysettings['CATEGORY']
>   File "/home/brian/Dev/git/portage/pym/portage/package/ebuild/config.py",
> line 2570, in __getitem__
>     return self._getitem(key)
>   File "/home/brian/Dev/git/portage/pym/portage/package/ebuild/config.py",
> line 2623, in _getitem
>     raise KeyError(mykey)
> KeyError: 'CATEGORY'

This one is fixed in git:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=022f0ba6fb75aba80d801772b4e622d44db2d668

(In reply to Brian Dolbec from comment #3)
>   File
> "/usr/lib64/python3.4/site-packages/portage/package/ebuild/_spawn_nofetch.
> py", line 71, in spawn_nofetch
>     defined_phases = settings['DEFINED_PHASES'].split()
>   File
> "/usr/lib64/python3.4/site-packages/portage/package/ebuild/config.py", line
> 2570, in __getitem__
>     return self._getitem(key)
>   File
> "/usr/lib64/python3.4/site-packages/portage/package/ebuild/config.py", line
> 2623, in _getitem
>     raise KeyError(mykey)
> KeyError: 'DEFINED_PHASES'

There's a patch for this in the following branch:

https://github.com/zmedico/portage/tree/bug_569942

I've posted it for review here:

https://archives.gentoo.org/gentoo-portage-dev/message/81a49a144c53bbbfdfad99474640641d
Comment 6 Zac Medico gentoo-dev 2015-12-29 20:40:12 UTC
(In reply to Zac Medico from comment #5)
> (In reply to Brian Dolbec from comment #3)
> >   File
> > "/usr/lib64/python3.4/site-packages/portage/package/ebuild/_spawn_nofetch.
> > py", line 71, in spawn_nofetch
> >     defined_phases = settings['DEFINED_PHASES'].split()
> >   File
> > "/usr/lib64/python3.4/site-packages/portage/package/ebuild/config.py", line
> > 2570, in __getitem__
> >     return self._getitem(key)
> >   File
> > "/usr/lib64/python3.4/site-packages/portage/package/ebuild/config.py", line
> > 2623, in _getitem
> >     raise KeyError(mykey)
> > KeyError: 'DEFINED_PHASES'
> 
> There's a patch for this in the following branch:
> 
> https://github.com/zmedico/portage/tree/bug_569942
> 
> I've posted it for review here:
> 
> https://archives.gentoo.org/gentoo-portage-dev/message/
> 81a49a144c53bbbfdfad99474640641d

In the master branch now:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=87e32d4863f3f72ace3fc123b04a0e6f5ff96799