Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 736473 - sys-apps/portage: anydbm cache module does not support multiprocessing
Summary: sys-apps/portage: anydbm cache module does not support multiprocessing
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 448462
  Show dependency tree
 
Reported: 2020-08-09 03:16 UTC by Zac Medico
Modified: 2020-08-09 05:18 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
Test case showing that modifications in child process are not visible to parent (dbm_fork_test.py,941 bytes, text/x-python)
2020-08-09 05:18 UTC, Zac Medico
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Zac Medico gentoo-dev 2020-08-09 03:16:13 UTC
In the unit test changes for bug 736334, I added a multiprocessing test which checks to see that cache modifications in a child process are accessible from the parent process:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d4f212def0dcba58e8a146daf8da7b481491de39

The anydbm module fails the multiprocessing test, which can be enabled like this:

> diff --git a/lib/portage/tests/dbapi/test_auxdb.py b/lib/portage/tests/dbapi/test_auxdb.py
> index 907c289fb..4dc18c0af 100644
> --- a/lib/portage/tests/dbapi/test_auxdb.py
> +++ b/lib/portage/tests/dbapi/test_auxdb.py
> @@ -15,7 +15,7 @@ class AuxdbTestCase(TestCase):
>                         from portage.cache.anydbm import database
>                 except ImportError:
>                         self.skipTest('dbm import failed')
> -               self._test_mod('portage.cache.anydbm.database', multiproc=False)
> +               self._test_mod('portage.cache.anydbm.database', multiproc=True)
>  
>         def test_flat_hash_md5(self):
>                 self._test_mod('portage.cache.flat_hash.md5_database')
Comment 1 Zac Medico gentoo-dev 2020-08-09 05:18:43 UTC
Created attachment 653855 [details]
Test case showing that modifications in child process are not visible to parent

The attached test cast shows that modifications in a child process are not visible to the parent process, resulting in this error:

> $ python ./dbm_fork_test.py 
> Traceback (most recent call last):
>   File "./dbm_fork_test.py", line 34, in <module>
>     raise AssertionError('db has missing key {}'.format(k))
> AssertionError: db has missing key b'child_key'

If the database is re-opened, then data written by the child process becomes visible.