Bug 913948 – python: randomness of map/set causes unreproducible .pyc.

• Product: Gentoo Linux
• Status: UNCONFIRMED ---

[Description thssld 2023-09-10 15:23:43 UTC]

We ship .pyc files in python related packages. Some (should be most?) python projects create set/map literals in their source and we compile them into .pyc files.

Python randomly order the set/map literals and makes the .pyc files not reproducible.

Archlinux and Fedora suggest the use of PYTHONHASHSEED=0.
Debian is not affected since they simply don't ship .pyc with them.

See:
https://archlinux.org/todo/unreproducible-python-bytecode/
https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/73

[Comment 1 Sam James 2023-09-10 15:30:02 UTC]

Doesn't this have some security implications?

https://lwn.net/Articles/474912/
https://lwn.net/Articles/916204/
https://docs.python.org/3/using/cmdline.html#cmdoption-R
http://ocert.org/advisories/ocert-2011-003.html

[Comment 2 Sam James 2023-09-10 15:34:22 UTC]

Unclear why https://github.com/python/cpython/pull/25411 was closed.

[Comment 3 Sam James 2023-09-10 15:38:14 UTC]

(In reply to Sam James from comment #1)
> Doesn't this have some security implications?

I suppose it's fine if we only set it in the Python eclasses, just not as a default in Python itself (i.e. we want the runtime randomisation, but not when producing .pyc).

[Comment 4 thssld 2023-09-10 16:37:18 UTC]

(In reply to Sam James from comment #3)
> (In reply to Sam James from comment #1)
> > Doesn't this have some security implications?
> 
> I suppose it's fine if we only set it in the Python eclasses, just not as a
> default in Python itself (i.e. we want the runtime randomisation, but not
> when producing .pyc).

I tried with PYTHONHASHSEED=0 before run catalyst and result seems reproducible. But I don't know if it makes runtime hashing randomization disabled. A small sample seems not working even on a non-hacked build.

[Comment 5 Mike Gilbert 2023-09-11 00:41:34 UTC]

I would suggest that people who want reproducible builds just add PYTHONHASHSEED to make.conf. I don't see any reason to do it for everybody in the python eclasses.