Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 449512 - app-portage/porthole with python3 - File "/usr/lib/portage/pym/portage/__init__.py", line 224, in __call__ rval = self._func(*wrapped_args, **wrapped_kwargs) OSError: [Errno 9] Bad file descriptor
Summary: app-portage/porthole with python3 - File "/usr/lib/portage/pym/portage/__init...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Third-Party Tools (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: Brian Dolbec
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-12-31 21:20 UTC by Kete Tefid
Modified: 2014-01-04 00:39 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kete Tefid 2012-12-31 21:20:05 UTC
Unfortunately porthole crashes during operation. 
There are several issues:
1) The main python version in the system is 3.2. It is set to 3.2 but porthole still tries to use 2.7. I am wondering if this is normal or not.
2) It crashes with the following information. It does not depend on what python version is selected. With both versions (3.2 and 2.7) both applications (porthole and porthole-2.7) produce the same output and result.

Porthole: python version =  2.7.3 (default, Nov 11 2012, 14:17:54) 
[GCC 4.6.3]
importing installed modules
starting main()
CONFIG: id initialized to  887255
CONFIGURATION: id initialized to  887452
PREFERENCES: id initialized to  890492
initializing utils.__init__.py
DEBUG: id initialized to  891117
UTILS: mydebug.id =  891117
PREFERENCES: imported debug.id =  891117
BACKENDS: id initialized to  913071
PORTAGELIB: id initialized to  913558
STERMINAL: id initialized to  917619
VERSION_SORT: id initialized to  919268
PROPERTIES: id initialized to  922716
METADATA: id initialized to  923201
PORTAGELIB: imported portage-2.2 modules
PORTAGELIB: portage version = 2.1.11.31
BACKENDS: portage_lib import complete : <module 'porthole.backends.portagelib' from '/usr/lib64/python2.7/site-packages/porthole/backends/portagelib.pyc'>
MAINWINDOW: id initialized to  953217
DATABASE: id initialized to  959512
PACKAGE: id initialized to  959848
DBREADER: import id initialized to  960061
UTILITIES: id initialized to  960786
USERCONFIGS: id initialized to  961106
DBREADER: DatabaseReader.id initialized to  961922
PLUGIN: id initialized to  971698
ADVEMERGE: id initialized to  975287
PORTHOLE: i18n_DIR = /usr/share/locale/
threading.enumerate() =  [<_MainThread(MainThread, started 140424585910016)>]
this thread is : 140424585910016  current thread  <_MainThread(MainThread, started 140424585910016)>
threading.enumerate() =  [<_MainThread(MainThread, started 140424585910016)>, <DescriptionReader(Thread-3, started daemon 140424371799808)>]
this thread is : 140424585910016  current thread  <_MainThread(MainThread, started 140424585910016)>
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.7/site-packages/porthole/readers/descriptions.py", line 42, in run
    self.descriptions[name] = package.get_description()
  File "/usr/lib64/python2.7/site-packages/porthole/db/package.py", line 202, in get_description
    return self.get_properties().description
  File "/usr/lib64/python2.7/site-packages/porthole/db/package.py", line 217, in get_properties
    ebuild = self.get_default_ebuild()
  File "/usr/lib64/python2.7/site-packages/porthole/db/package.py", line 157, in get_default_ebuild
    return (self.get_best_ebuild() or
  File "/usr/lib64/python2.7/site-packages/porthole/db/package.py", line 138, in get_best_ebuild
    self.best_ebuild = portage_lib.get_best_ebuild(self.full_name)
  File "/usr/lib64/python2.7/site-packages/porthole/backends/portagelib.py", line 422, in get_best_ebuild
    return xmatch("bestmatch-visible",str(full_name)) # no unicode
  File "/usr/lib64/python2.7/site-packages/porthole/backends/portagelib.py", line 328, in xmatch
    results  =  settings.portdb.xmatch(*args, **kwargs)[:] # make a copy.  needed for <portage-svn-r5382
  File "/usr/lib/portage/pym/portage/dbapi/porttree.py", line 825, in xmatch
    self.aux_get(cpv, aux_keys, myrepo=repo)))
  File "/usr/lib/portage/pym/portage/dbapi/porttree.py", line 455, in aux_get
    proc.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 54, in wait
    self._wait()
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 90, in _wait
    self._waitpid_loop()
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 96, in _waitpid_loop
    self.pid, self._waitpid_cb)
  File "/usr/lib/portage/pym/portage/util/_eventloop/EventLoop.py", line 244, in child_watch_add
    self._poll_child_processes()
  File "/usr/lib/portage/pym/portage/util/_eventloop/EventLoop.py", line 288, in _poll_child_processes
    x.callback(x.pid, wait_retval[1], x.data)
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 106, in _waitpid_cb
    self._set_returncode((pid, condition))
  File "/usr/lib/portage/pym/_emerge/EbuildMetadataPhase.py", line 143, in _set_returncode
    SubProcess._set_returncode(self, wait_retval)
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 136, in _set_returncode
    self._unregister()
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 125, in _unregister
    os.close(f)
  File "/usr/lib/portage/pym/portage/__init__.py", line 224, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
OSError: [Errno 9] Bad file descriptor

^CTraceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/porthole/utils/dispatcher.py", line 55, in on_data
    self.callback(*self.queue.get(), **self.callback_kwargs)
  File "/usr/lib64/python2.7/site-packages/porthole/mainwindow.py", line 1042, in search_done
    self.package_view.populate(package_list)
  File "/usr/lib64/python2.7/site-packages/porthole/views/package.py", line 439, in populate
    upgradable = packages[name].is_dep_upgradable()
  File "/usr/lib64/python2.7/site-packages/porthole/db/package.py", line 283, in is_dep_upgradable
    best, keyworded, hardmasked = portage_lib.get_dep_ebuild(dep)
  File "/usr/lib64/python2.7/site-packages/porthole/backends/portagelib.py", line 429, in get_dep_ebuild
    best_ebuild = xmatch("bestmatch-visible", dep)
  File "/usr/lib64/python2.7/site-packages/porthole/backends/portagelib.py", line 328, in xmatch
    results  =  settings.portdb.xmatch(*args, **kwargs)[:] # make a copy.  needed for <portage-svn-r5382
  File "/usr/lib/portage/pym/portage/dbapi/porttree.py", line 825, in xmatch
    self.aux_get(cpv, aux_keys, myrepo=repo)))
  File "/usr/lib/portage/pym/portage/dbapi/porttree.py", line 455, in aux_get
    proc.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 54, in wait
    self._wait()
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 79, in _wait
    self._wait_loop()
  File "/usr/lib/portage/pym/_emerge/AbstractPollTask.py", line 141, in _wait_loop
    self.scheduler.iteration()
  File "/usr/lib/portage/pym/portage/util/_eventloop/EventLoop.py", line 180, in iteration
    if not x.callback(f, event, *x.args):
  File "/usr/lib/portage/pym/_emerge/EbuildMetadataPhase.py", line 138, in _output_handler
    self._unregister_if_appropriate(event)
  File "/usr/lib/portage/pym/_emerge/AbstractPollTask.py", line 129, in _unregister_if_appropriate
    self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 54, in wait
    self._wait()
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 90, in _wait
    self._waitpid_loop()
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 99, in _waitpid_loop
    self.scheduler.iteration()
KeyboardInterrupt
^CTraceback (most recent call last):
  File "/usr/bin/porthole-2.7", line 114, in <module>
    main()
  File "/usr/lib64/python2.7/site-packages/porthole/startup.py", line 191, in main
    gtk.main()
KeyboardInterrupt

It hangs and I have to press Ctrl+c to kill the process.
emerge --info:
Portage 2.1.11.31 (default/linux/amd64/10.0/desktop/kde, gcc-4.6.3, glibc-2.15-r3, 3.7.1-pf x86_64)
=================================================================
System uname: Linux-3.7.1-pf-x86_64-Intel-R-_Core-TM-_i3_CPU_M_380_@_2.53GHz-with-gentoo-2.1
Timestamp of tree: Sun, 17 Jun 2012 17:15:01 +0000
ld GNU ld (GNU Binutils) 2.22
app-shells/bash:          4.2_p37
dev-lang/python:          2.7.3-r2, 3.2.3
dev-util/cmake:           2.8.9
dev-util/pkgconfig:       0.27.1
sys-apps/baselayout:      2.1-r1
sys-apps/openrc:          0.11.8
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.9.6-r3, 1.11.6
sys-devel/binutils:       2.22-r1
sys-devel/gcc:            4.6.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.6 (virtual/os-headers)
sys-libs/glibc:           2.15-r3
Repositories: gentoo tlp linuxsystems wavilen x-lymnprtg my_local
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA PUEL sun-bcla-java-vm skype-eula dlj-1.1 RTCW-ETEULA AdobeFlash-10.3 AdobeFlash-10 Nero-EULA-US skype-4.0.0.7-copyright"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=corei7 -mcx16 -msahf -mno-movbe -mno-aes -mno-pclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-tbm -mno-avx -msse4.2 -msse4.1 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=corei7 -I/usr/include/opencore-amrnb"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0"
CONFIG_PROTECT_MASK="${EPREFIX}/etc/gconf /etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=corei7 -mcx16 -msahf -mno-movbe -mno-aes -mno-pclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-tbm -mno-avx -msse4.2 -msse4.1 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=corei7 -I/usr/include/opencore-amrnb"
DISTDIR="/usr/portage/distfiles/"
EMERGE_DEFAULT_OPTS="--keep-going --jobs=5 --load-average=10.0 --with-bdeps y"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs buildpkg ccache config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch protect-owned sandbox sfperms unknown-features-warn unmerge-logs unmerge-orphans userfetch xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp.uni-erlangen.de/pub/mirrors/gentoo http://trumpetti.atm.tut.fi/gentoo/ http://portage.org.ua/"
LANG="en_US"
LDFLAGS="-Wl,-z,now,--as-needed,-O1,--hash-style=gnu,--sort-common"
MAKEOPTS="-j5 -l8"
PKGDIR="/usr/portage/GentooPackages/"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/usr/portage/prtgtmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/lymnprtg/tlp /usr/portage/lymnprtg/linuxsystems /usr/portage/lymnprtg/wavilen /usr/portage/lymnprtg /usr/portage/lymnprtg/localoverlays"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext 3gp X a52 aac aalib acl acpi alsa amd64 amr archive asf automount avahi berkdb bittorrent bluetooth bmp branding bzip2 cairo cdda cddb cdio cdparanoia cdr cli consolekit cracklib crypt cups cxx dbus declarative directfb djvu double-precision dri dts dv dvb dvd dvdnav dvdr emboss encode exif faac faad fam fbcon fbcondecor fbsplash ffmpeg firefox flac flash fortran fortran95 fuse gdbm gif gmp gpg gpm gstreamer gtk gzip h323 html iconv id3tag inotify ipv6 jingle jpeg jpeg2k kde kipi lame lcms ldap libnotify live lm_sensors lzma mad matroska mjpeg mms mmx mmxext mng modules mozilla mp3 mp4 mpeg mudflap multilib ncurses network nfs nls nptl nptlonly nsplugin ogg ogm openal opengl openmp oss pam pango pcre pdf phonon plasma png pnm policykit postscript ppds pppd python3 qt3support qt4 quicktime rar readline samba scanner sdl search semantic-desktop session sftp sip slp smp socks socks5 spell sse sse2 sse3 sse4 sse4_1 sse4a ssh ssl ssse3 startup-notification stream svg symlink taglib tcpd theora tiff tk truetype type3 udev udisks unicode upower usb v4l vaapi vcd vcdx vlm vorbis vpx wav wavpack webgl wifi wma wmf wxwidgets x264 xattr xcb xcomposite xinerama xml xpm xscreensaver xv xvid yahoo youtube zeroconf zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en ar fa" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" SANE_BACKENDS="gt68xx" USERLAND="GNU" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, USE_PYTHON


emerge -pv porthole


Calculating dependencies... done!
[ebuild   R   ~] app-portage/porthole-0.6.1-r3  USE="nls" LINGUAS="-de -fr -it -pl -ru -tr -vi" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB

I have set this bug to major. If necessary, please change it. Porthole is completely broken and totally unusable. The version that I am using has fixed a bad bug (after nearly two years) and the so-called stable version (0.6.1-r1) has the bug.
I have been waiting to use a functional porthole since 2010 but everytime there is a problem with that. That is frustrating.
I am sorry for this offtopic questions but are there any functional frondends for emerge/portage? Everything that I searched for was not in the portage.



Reproducible: Always
Comment 1 Zac Medico gentoo-dev 2013-01-02 00:15:11 UTC
It looks like the traceback shows 2 threads entering portdbapi.aux_get() at the same time, which is unsafe. You'll need to wrap that call with a threading lock.
Comment 2 Brian Dolbec gentoo-dev 2013-01-02 00:29:19 UTC
yeah, the description reader was running in a thread, updating the description db and there was a pkg display call in the main thread.  After all these years, this is the first collision of that sort.  At least reported.  I've never run into that myself, and I've been doing pkg display changes while the description reader has been running.

As for the python3 issue, porthole is not yet capable of running in python3, that is why it runs in python2 only.  It will need a lot of updating to be python3 capable.  I also need to work on the gtk migration to the introspection python interface.

The easiest workaround is to start porthole and leave it be for awhile, so it can update it's description db and save it.
Comment 3 Zac Medico gentoo-dev 2013-01-02 00:54:52 UTC
(In reply to comment #2)
> The easiest workaround is to start porthole and leave it be for awhile, so
> it can update it's description db and save it.

Alternately, run `emerge --regen` before running porthole. That way, portdbapi.aux_get() won't enter the thread-sensitive areas of portdbapi.aux_get() while porthole is running.
Comment 4 Brian Dolbec gentoo-dev 2013-01-02 02:34:43 UTC
That won't work reliably since porthole can be running, and call emerge --sync, then it will automatically reload the description db.  It has been my intention to move all portage/pkgcore activity to a single thread/process (for the data gathering instance, user privileges).  There will be another instance running with root privileges for merging and other operations.

In that way, I'd be able to enable the sqlite backend, since it does not work threaded.

Just too many other projects/distractions/real life this past couple years.
Comment 5 Zac Medico gentoo-dev 2013-01-03 22:47:30 UTC
This portage patch should suppress the problem by using a separate EventLoop instance for each thread:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=400a2a983b374ec54e7b574789d1f94752581b81
Comment 6 Kete Tefid 2013-01-05 19:30:09 UTC
Thanks to everybody.
Is there any hope to get it working soon?
Will the patch go to main portage?
Comment 7 Brian Dolbec gentoo-dev 2013-01-05 20:58:33 UTC
yes, it is available now for testing in portage-9999.  It will also be in the next 2.2 alpha release. and will eventually make it's way into portage-2.1 once it's proven to be working correctly and not causing other problems.
Comment 8 Zac Medico gentoo-dev 2013-02-10 22:16:25 UTC
(In reply to comment #5)
> This portage patch should suppress the problem by using a separate EventLoop
> instance for each thread:
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> h=400a2a983b374ec54e7b574789d1f94752581b81

This is in sys-apps/portage-2.1.11.50 which is stable now:

  https://bugs.gentoo.org/show_bug.cgi?id=455930
Comment 9 Sergey Popov gentoo-dev 2014-01-03 22:13:30 UTC
So, is it fixed for now? Can we close this?
Comment 10 Brian Dolbec gentoo-dev 2014-01-04 00:39:26 UTC
closing as it has been in stable releases for some time now.