| Summary: | emirrordist fails on dev-lang/python[-berkdb], needs to try bsddb3 | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Robin Johnson <robbat2> |
| Component: | Core - External Interaction | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | Keywords: | InVCS |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: | https://bugs.gentoo.org/show_bug.cgi?id=721680 | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 549914 | ||
| Attachments: |
support bsddb3 (untested)
support bsddb3 |
||
Created attachment 421836 [details, diff]
support bsddb3 (untested)
Your patch is buggy in a few ways, but here's a cleaned up version that works: if e.message == 'No module named bsddb': from bsddb3 import dbshelve db = dbshelve.open(db_file, flags=open_flag) 1. e.name doesn't exist (AttributeError: 'exceptions.ImportError' object has no attribute 'name') 2. cannot use shelve before assignment (UnboundLocalError: local variable 'shelve' referenced before assignment) 3. dbshelve calls it 'flags' plural. Created attachment 421924 [details, diff] support bsddb3 Updated patch with fixes from comment #2. Given the AGPL-3 license of sys-libs/db-6, we might consider moving away from bsddb{,3} usage.
For cases when the database file doesn't exist yet, we can use something like this to force it to gdbm format:
shelve.Shelf(gdbm.open(filename, flag))
If the file already exists, we could have it automatically migrate it from bsddb to gdbm.
(In reply to Zac Medico from comment #3) > Created attachment 421924 [details, diff] [details, diff] > support bsddb3 > > Updated patch with fixes from comment #2. Posted for review here: https://archives.gentoo.org/gentoo-portage-dev/message/322fee5001362fa8975a0f9d0f361644 I'd be fine with any solution that has automatic migration, but might want to check how heavily we're using it, the major different between berkdb, gdbm etc is index performance. The bsddb3 patch is in the master branch: https://gitweb.gentoo.org/proj/portage.git/commit/?id=7c94014a32d173ae61919b762140ac1c32d3b522 (In reply to Robin Johnson from comment #6) > I'd be fine with any solution that has automatic migration, but might want > to check how heavily we're using it, the major different between berkdb, > gdbm etc is index performance. Yeah, I'll have to do some benchmarks. Fixed in 2.2.27. |
Traceback (most recent call last): File "/usr/lib/python-exec/python2.7/emirrordist", line 13, in <module> sys.exit(emirrordist_main(sys.argv[1:])) File "/usr/lib64/python2.7/site-packages/portage/_emirrordist/main.py", line 451, in emirrordist_main SchedulerInterface(global_event_loop())) as config: File "/usr/lib64/python2.7/site-packages/portage/_emirrordist/Config.py", line 68, in __init__ options.distfiles_db, 'distfiles') File "/usr/lib64/python2.7/site-packages/portage/_emirrordist/Config.py", line 112, in _open_shelve db = shelve.open(db_file, flag=open_flag) File "/usr/lib64/python2.7/shelve.py", line 243, in open return DbfilenameShelf(filename, flag, protocol, writeback) File "/usr/lib64/python2.7/shelve.py", line 227, in __init__ Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback) File "/usr/lib64/python2.7/anydbm.py", line 84, in open mod = __import__(result) File "/usr/lib64/python2.7/dbhash.py", line 7, in <module> ImportError: No module named bsddb