Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 410981

Summary: app-portage/etc-proposals with dev-lang/python-2.7[-berkdb] - raise error, "db type could not be determined"
Product: Portage Development Reporter: Mark <chaseguard>
Component: ToolsAssignee: Portage Tools Team <tools-portage>
Status: RESOLVED OBSOLETE    
Severity: normal CC: dolsen, tools-portage
Priority: Normal Keywords: PMASKED
Version: unspecified   
Hardware: AMD64   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Deadline: 2020-09-21   

Description Mark 2012-04-06 01:51:37 UTC
I recently emerged dev-python/bsddb3 and then re-emerged my system python with an unset berkdb flag (to avoid the post install deprecated warnings and prepare for the future). After this, etc-proposals fails to launch and throws:  


# etc-proposals
Traceback (most recent call last):
  File "/usr/sbin/etc-proposals", line 11, in <module>
    from etcproposals.etcproposals_lib import Config, EtcProposals
  File "/usr/lib64/python2.7/site-packages/etcproposals/etcproposals_lib.py", line 604, in <module>
    State = EtcProposalsState()
  File "/usr/lib64/python2.7/site-packages/etcproposals/etcproposals_lib.py", line 566, in __init__
    shelve.Shelf.__init__(self, anydbm.open(STATEFILE, 'c'))
  File "/usr/lib64/python2.7/anydbm.py", line 82, in open
    raise error, "db type could not be determined"
anydbm.error: db type could not be determined


I did revdep-rebuild and nothing is missing.  I also ran python-updater (even though there was no version change) to no avail. There are plenty of Google and forum hits about python/berkdb, but there seems to be no overall answer to anydbm.py and "db type could not be determined." Next, I deleted the /var/state/etcproposals.state file and then etc-proposals will launch -- just one time. After recreating etcproposals.state the application will not relaunch (unless the file is again deleted).

The particulars:

emerge --info etc-proposals python 
Portage 2.1.10.55 (default/linux/amd64/10.0/desktop, gcc-4.5.3, glibc-2.14.1-r2, 3.2.1-gentoo-r2 x86_64)
=================================================================
                        System Settings
=================================================================
System uname: Linux-3.2.1-gentoo-r2-x86_64-Dual-Core_AMD_Opteron-tm-_Processor_1218-with-gentoo-2.1
Timestamp of tree: Thu, 05 Apr 2012 17:05:01 +0000
app-shells/bash:          4.2_p24
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.7.2-r3, 3.2.2
dev-util/cmake:           2.8.7-r5
dev-util/pkgconfig:       0.26
sys-apps/baselayout:      2.1
sys-apps/openrc:          0.9.9.3
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.9.6-r3, 1.11.4
sys-devel/binutils:       2.22-r1
sys-devel/gcc:            4.5.3-r1
sys-devel/gcc-config:     1.6
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r3
sys-kernel/linux-headers: 3.1 (virtual/os-headers)
sys-libs/glibc:           2.14.1-r2
Repositories: gentoo x-portage
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=opteron -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/openvpn/easy-rsa /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=opteron -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS=" --oneshot --autounmask=n --quiet-build=n "
FEATURES="assume-digests binpkg-logs collision-protect distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG=""
LC_ALL="POSIX"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/dev/shm"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X a52 aac acl acpi alsa amd64 avi berkdb branding bzip2 cairo cdda cdparanoia cdr chroot cli consolekit cracklib crypt cups cxx dbus dri dts dv dvd dvdr emboss encode exif fam ffmpeg firefox flac fortran freetype gdbm gdu gif gpm gtk iconv ipv6 java javascript jpeg lcms libnotify lm_sensors mad mime mmx mng modules mp3 mp4 mpeg mpi mudflap multilib mysql nas ncurses net network nis nls nptl nptlonly nptonly nsplugin nvidia ogg opengl openmp pam pango pcre pdf png policykit posix ppds pppd python qt3support qt4 readline samba sdl session snmp socks5 spell sse sse2 ssl startup-notification svg swat sysfs tcpd thunar tiff truetype udev unicode usb userlocales vorbis x264 xcb xml xorg xulrunner xv xvid zlib" ALSA_CARDS="ice1712" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON


=================================================================
                        Package Settings
=================================================================

app-portage/etc-proposals-1.4.3-r2 was built with the following:
USE="gtk (multilib) -qt4"


dev-lang/python-2.7.2-r3 was built with the following:
USE="gdbm ipv6 (multilib) ncurses readline sqlite ssl threads tk (wide-unicode) xml -berkdb -build -doc -examples -wininst"
CFLAGS="-march=opteron -O2 -pipe -fwrapv"
CXXFLAGS="-march=opteron -O2 -pipe -fwrapv"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -L."


dev-lang/python-3.2.2 was built with the following:
USE="gdbm ipv6 (multilib) ncurses readline sqlite ssl threads tk (wide-unicode) xml -build -doc -examples -wininst"
CFLAGS="-march=opteron -O2 -pipe -fwrapv"
CXXFLAGS="-march=opteron -O2 -pipe -fwrapv"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -L."




Reproducible: Always

Steps to Reproduce:
1. launch etc-proposals
2.
3.
Actual Results:  
Receive error message

Expected Results:  
etc-proposals launches
Comment 1 Mark 2012-04-06 02:11:07 UTC
 # eselect python list
Available Python interpreters:
  [1]   python2.7 *
  [2]   python3.2
Comment 2 Brian Dolbec (RETIRED) gentoo-dev 2012-04-06 15:09:50 UTC
I've run into this or a similar error myself in the past (don't remember the exact error).  I have installed the bsddb3 pkg as well.

The solution for me was to delete etc-proposals user's config file so that anydbm would then not be given the previous db type to use.  It seems that the db type is saved in the config.
Comment 3 Mark 2012-04-07 04:49:22 UTC
I looked in the conf file (/etc/etc-proposals) and see nothing that indicates the db type. I deleted the conf and got:

etc-proposals
Every configured frontend failed to start.

I then unmerged etc-proposals, deleted the /var/state/etcproposals.state, and re-emerged. That gave me a successful launch. I closed that instance and relaunched only to get the same:

# etc-proposals
Traceback (most recent call last):
  File "/usr/sbin/etc-proposals", line 11, in <module>
    from etcproposals.etcproposals_lib import Config, EtcProposals
  File "/usr/lib64/python2.7/site-packages/etcproposals/etcproposals_lib.py", line 604, in <module>
    State = EtcProposalsState()
  File "/usr/lib64/python2.7/site-packages/etcproposals/etcproposals_lib.py", line 566, in __init__
    shelve.Shelf.__init__(self, anydbm.open(STATEFILE, 'c'))
  File "/usr/lib64/python2.7/anydbm.py", line 82, in open
    raise error, "db type could not be determined"
anydbm.error: db type could not be determined

Seems to me, as the traceback says, the statefile is not telling the right stuff.
Comment 4 Brian Dolbec (RETIRED) gentoo-dev 2012-04-09 19:38:42 UTC
Ok, it's taken a bit, but i've finally remembered all the hassles I had with bsddb3 and python 2.6.  The problem is not etc-proposals as it does not specify a db type, but calls python's builtin anydbm().  In python 2.6 when i first encountered the problem it was still trying to import bsddb, but the new external pkg was now bsddb3.  At first I just edited anydbm to import bsddb3, which worked.  But later updates changed the way anydbm worked.  It now uses whichdb().  It is that function which is failing to determine that it is beddb3.  

For that reason, This particular issue is an upstream issue with python and bsddb3.

So far I have not been able to restore normal operation after unmerging bsddb3.
 although, I was able to long ago when still using python 2.6

So far the only way is for me to restore my working state file from before this recent testing to duplicate your problem.

More checking it looks like gdbm's magic number has changed and python-2.7 isn't looking for the correct one.  

doing more testing...
Comment 5 Brian Dolbec (RETIRED) gentoo-dev 2012-04-09 20:36:29 UTC
Yes, it is definitely a python issue.  It seems that your python2.7 most is likely using gdbm and not bsddb3.  For me it was, even with bsddb3 installed.

The problem is python-2.7.2 is not checking for the correct magic numbers, which gdbm is now using a newer one, so, when python is opening the file, it does not know which db module it belongs to.

This is fixed in the upcoming python-2.7.3.

But I will look into changing etc-proposals into using the bsddb3 module directly.
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-09-21 06:15:02 UTC
removed.