Bug 736473 – sys-apps/portage: anydbm cache module does not support multiprocessing

• Product: Portage Development
• Status: CONFIRMED ---

[Description Zac Medico 2020-08-09 03:16:13 UTC]

In the unit test changes for bug 736334, I added a multiprocessing test which checks to see that cache modifications in a child process are accessible from the parent process:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d4f212def0dcba58e8a146daf8da7b481491de39

The anydbm module fails the multiprocessing test, which can be enabled like this:

> diff --git a/lib/portage/tests/dbapi/test_auxdb.py b/lib/portage/tests/dbapi/test_auxdb.py
> index 907c289fb..4dc18c0af 100644
> --- a/lib/portage/tests/dbapi/test_auxdb.py
> +++ b/lib/portage/tests/dbapi/test_auxdb.py
> @@ -15,7 +15,7 @@ class AuxdbTestCase(TestCase):
>                         from portage.cache.anydbm import database
>                 except ImportError:
>                         self.skipTest('dbm import failed')
> -               self._test_mod('portage.cache.anydbm.database', multiproc=False)
> +               self._test_mod('portage.cache.anydbm.database', multiproc=True)
>  
>         def test_flat_hash_md5(self):
>                 self._test_mod('portage.cache.flat_hash.md5_database')

[Comment 1 Zac Medico 2020-08-09 05:18:43 UTC]

Created attachment 653855 [details]
Test case showing that modifications in child process are not visible to parent

The attached test cast shows that modifications in a child process are not visible to the parent process, resulting in this error:

> $ python ./dbm_fork_test.py 
> Traceback (most recent call last):
>   File "./dbm_fork_test.py", line 34, in <module>
>     raise AssertionError('db has missing key {}'.format(k))
> AssertionError: db has missing key b'child_key'

If the database is re-opened, then data written by the child process becomes visible.