We better start tracking this on on a bug tracker. -pyc files store the EPREFIX of the host where it was built on. -chpathtool does some "simple" heuristic to switch the build EPREFIX to the new target EPREFIX -python (compileall.py) doesn't rebuild .pyc files if the mtime(?) doesn't change. -python (seemingly) doesn't like when the pyc files are changed and *not* regenerated. You will see the message: "Bad marshalling data" The solution is to not place .py[co] files in binpkgs Commit http://overlays.gentoo.org/proj/alt/changeset/56120 "fixes" it for python. But introduces another problem (as pointed out by solar), if .pyc files are not in the image dir, then you will not be in CONTENTS/vdb and as such portage won't track them (and won't remove them). This is a band-aid fix anyway because we were under that false impression that python was the only package that installed pyc files. That is simply not true. %% find $EPREFIX/var/db/pkg -name CONTENTS | xargs grep \\.pyc|grep -v "dev-lang/python"|wc -l 25 %% So, back to the drawing board. New Proposal: If a binpkg is created, drop .pyc files in the tbz2 only. This can be achieved in portage in the function that builds the tbz2. But, I can't find it yet. Alternative, is to exlude it when it is extracted. Is that sane? %% tar -tf test.tar test/ test/foo.s test/foo.o test/foo.c %% tar --exclude=*.c -xvf test.tar test/ test/foo.s test/foo.o %%
Created attachment 216429 [details] --exclude=*.pyc --exclude=*.pyo
Created attachment 216430 [details, diff] --exclude=*.pyc --exclude=*.pyo of course, a missing space.
I find it weird that python_mod_optimize is run /after/ installing in that case. Because that means you don't get them in your VDB, while in fact they're still out there. @python team: how is this supposed to work? (the VDB contents and .py[co] files) If this is all on purpose, I propose to either patch compileall and friends to have a force mode, used by python_mod_optimize, or if that is infeasible, to simply set mtime to Jan 1st 1970 or something, so make sure they will be regenerated.
Comment on attachment 216430 [details, diff] --exclude=*.pyc --exclude=*.pyo It doesn't make sense to create this different install approach for binpkgs vs source packages. Patch is obsolete. Don't consider it.
python_mod_optimize() already supports '-f' option. -f: force rebuild even if timestamps are up-to-date
ok, that's nice to know. Arfrever how do you feel about some packages having .py[co] files in CONTENTS file/VDB, while others don't? Should all have that, or should no package ever list it in VDB? From another perspective, why doesn't python_mod_optimize generate it inside ${D} instead of on the live filesystem after installing?
Maybe I'm wrong here, but IIRC python optimized binaries created in $D do not work in $ROOT. Iff this is really true, .py[co] in vdb are plain wrong.
prefix bits are irrelevant here, bug too polluted.
