Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 701878

Summary: dev-python/pycparser-2.19-r1 - src_compile(): python3_7: ModuleNotFoundError: No module named 'lextab'
Product: Gentoo Linux Reporter: N. Andrew Walsh <n.andrew.walsh>
Component: Current packagesAssignee: Python Gentoo Team <python>
Status: RESOLVED FIXED    
Severity: normal CC: dbishop, gem, grobian, mgorny, quinn_jones, sam, tommy, wols
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=871969
https://bugs.gentoo.org/show_bug.cgi?id=613896
https://github.com/eliben/pycparser/pull/494
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 351559    
Attachments: build.log
build.log
Debugging patch
build.log with patch
build log with patch, try #2
build log dev-python/pycparser-2.21-r2
import-debug.patch
rm lextab.py ; strace -o trace.log python -B _build_tables.py
inval.patch

Description N. Andrew Walsh 2019-12-03 12:05:29 UTC
emerge pycparser fails with the following error:

---------------------
copying pycparser/ply/__init__.py -> /var/tmp/portage/dev-python/pycparser-2.19-r1/work/pycparser-2.19-python3_7/lib/pycparser/ply
copying pycparser/_c_ast.cfg -> /var/tmp/portage/dev-python/pycparser-2.19-r1/work/pycparser-2.19-python3_7/lib/pycparser
warning: build_py: byte-compiling is disabled, skipping.

Traceback (most recent call last):
  File "_build_tables.py", line 31, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
----------------------

build.log attached

Reproducible: Always




 # emerge --info
Portage 2.3.80 (python 3.6.9-final-0, default/linux/amd64/17.0, gcc-9.2.0, glibc-2.30-r3, 5.4.1-gentoo x86_64)
=================================================================
System uname: Linux-5.4.1-gentoo-x86_64-AMD_FX-tm-9590_Eight-Core_Processor-with-gentoo-2.6
KiB Mem:    32790560 total,  17309276 free
KiB Swap:     976556 total,    976556 free
Timestamp of repository gentoo: Mon, 02 Dec 2019 09:50:02 +0000
Head commit of repository gentoo: dc2ddf0786eaeab7effb3cee121d022641f2c9a5

sh bash 5.0_p11
ld GNU ld (Gentoo 2.33.1 p1) 2.33.1
ccache version 3.7.6 [disabled]
app-shells/bash:          5.0_p11::gentoo
dev-java/java-config:     2.2.0-r4::gentoo
dev-lang/perl:            5.30.1::gentoo
dev-lang/python:          2.7.17-r100::stefantalpalaru, 3.5.9::gentoo, 3.6.9::gentoo, 3.7.5-r1::gentoo, 3.8.0::gentoo
dev-util/ccache:          3.7.6::gentoo
dev-util/cmake:           3.16.0::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.6-r1::gentoo
sys-apps/openrc:          0.42.1::gentoo
sys-apps/sandbox:         2.18::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake:       1.11.6-r2::gentoo, 1.13.4-r1::gentoo, 1.15.1-r1::gentoo, 1.16.1-r2::gentoo
sys-devel/binutils:       2.33.1::gentoo
sys-devel/gcc:            9.2.0-r2::gentoo
sys-devel/gcc-config:     2.1::gentoo
sys-devel/libtool:        2.4.6-r5::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 5.3-r1::gentoo (virtual/os-headers)
sys-libs/glibc:           2.30-r3::gentoo
Repositories:

gentoo
    location: /var/portage/repos
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo
    priority: -1000

x-portage
    location: /usr/local/portage
    masters: gentoo
    priority: 0

atom
    location: /var/portage/repos/atom
    masters: gentoo
    priority: 50

bobwya
    location: /var/portage/repos/bobwya
    masters: gentoo
    priority: 50

brave-overlay
    location: /var/portage/repos/brave-overlay
    masters: gentoo
    priority: 50

cynede
    location: /var/portage/repos/cynede
    masters: gentoo
    priority: 50

farmboy0
    location: /var/portage/repos/farmboy0
    masters: gentoo
    priority: 50

hossie
    location: /var/portage/repos/hossie
    masters: gentoo
    priority: 50

pentoo
    location: /var/portage/repos/pentoo
    masters: gentoo
    priority: 50

qt
    location: /var/portage/repos/qt
    masters: gentoo
    priority: 50

stefantalpalaru
    location: /var/portage/repos/stefantalpalaru
    masters: gentoo
    priority: 50

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=bdver2 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=bdver2 -O2 -pipe"
DISTDIR="/var/portage/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=n"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://gentoo.mirror.dkm.cz/pub/gentoo/ http://ftp.fi.muni.cz/pub/linux/gentoo/ http://gentoo.mirror.web4u.cz/ ftp://gentoo.mirror.web4u.cz/ ftp://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ http://mirror.netcologne.de/gentoo/ ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo http://gentoo.mneisen.org/ ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ ftp://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp6.uni-erlangen.de/pub/mirrors/gentoo ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ ftp://mirror.netcologne.de/gentoo/ ftp://ftp6.uni-erlangen.de/pub/mirrors/gentoo ftp://ftp6.uni-muenster.de/pub/linux/distributions/gentoo http://ftp.uni-erlangen.de/pub/mirrors/gentoo http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ ftp://ftp.ipv6.uni-muenster.de/pub/linux/distributions/gentoo ftp://gentoo.inf.elte.hu/ http://gentoo.inf.elte.hu/ http://ftp.heanet.ie/pub/gentoo/ ftp://ftp.df.lth.se/pub/gentoo/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en en_US en_US.utf8 de de_DE de_DE.utf8"
MAKEOPTS="-j9"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X Xaw3d a52 aac acl acpi aim alsa amd64 apm audiofile bash-completion berkdb bzip2 cairo cddb cdinstall cdparanoia cdr cli cracklib crypt css curl curlwrappers cxx dbus directfb dri dvd dvdr encode fbcon ffmpeg fftw firefox flac fortran ftp gdbm geoip gif gimp glut graphite gstreamer gtk hddtemp iconv icq ieee1394 imagemagick imap imlib ipv6 jack java javascript joystick jpeg lame latex ldap libsamplerate libtirpc libwww lm_sensors mad matroska minimal mmx motif mp3 mpeg mplayer multilib ncurses nls nptl nsplugin offensive ogg openal opengl openmp openssl oscar pam pcre pdf perl png policykit posix python qt5 quicktime raw readline real rss scanner seccomp semantic-desktop sndfile sockets speex spell split-usr ssl suid svg symlink syslog tcl tcpd tetex theora threads tidy tiff tk translucency truetype udev unicode usb vaapi vdpau videos vorbis win32codecs wmf wxwindows x264 xattr xcomposite xetex xine xml xpm xscreensaver xulrunner xv xvid xvmc zlib" ABI_X86="64 32" ADA_TARGET="gnat_2018" ALSA_CARDS="hda-intel usb-audio" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a" CURL_SSL="openssl" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" L10N="en-US de" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python3_7 python3_8 python2_7" RUBY_TARGETS="ruby26 ruby25" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 N. Andrew Walsh 2019-12-03 12:05:49 UTC
Created attachment 598318 [details]
build.log
Comment 2 Quinn Jones 2020-05-07 15:30:11 UTC
Seconding this bug, just ran into it after an 'emerge --sync' and 'emerge -uDNav world' today.
Comment 3 Quinn Jones 2020-05-07 15:51:28 UTC
Interesting note:

On a whim I restarted emerge ("emerge -av --resume").  Resuming sets a different build order: pycparser now 2nd item in set, rebuild of dev-python/traceback2-1.4.0 comes first.  traceback2 succeeds, pycparser fails again.

Resume again, pycparser comes first.  I expect it to fail -- but it succeeds.

traceback2 doesn't include a 'lextab' component, and no other python changes were made in between, so not sure why this worked.



So, to summarize:

--- Event 1 ---

'emerge -uDNav world' fails on upgrade of dev-python/pycparser-2.20 (from dev-python/pycparser-2.19-r1), with:

warning: build_py: byte-compiling is disabled, skipping.

Traceback (most recent call last):
  File "_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'

--- Event 2 ---

'emerge --resume' starts with rebuild of dev-python/traceback2-1.4.0, followed by upgrade to dev-python/pycparser-2.20. traceback2 succeeds, but pycparser still fails:

warning: build_py: byte-compiling is disabled, skipping.

Traceback (most recent call last):
  File "_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'

--- Event 3 ---

'emerge --resume' starts with upgrade to dev-python/pycparser-2.20, and succeeds.
Comment 4 N. Andrew Walsh 2020-05-11 11:45:51 UTC
Same error today with pycparser-2.20, by the way. Any progress on this?
Comment 5 Quinn Jones 2020-05-12 01:25:03 UTC
Like OP, same error today, different machine than my previous report.

This particular machine didn't have traceback2 installed, which saved my emerge --update last week.  'emerge --resume'ing wasn't getting me anywhere.

So on a whim I installed dev-python/traceback2. It pulled in dev-python/pbr-4.2.0-r1, dev-python/linecache2-1.0.0, and dev-python/traceback2-1.4.0.

Voila! dev-python/pycparser-2.20 now emerges.
Comment 6 N. Andrew Walsh 2020-05-12 08:11:14 UTC
When I ran 'emerge --resume --skipfirst' pycparser completed without error, so apparently something does have to be emerge first for it to be able to find lextab. 

'equery b /usr/lib64/python3.7/site-packages/pycparser/lextab.py' doesn't return anything, nor for the same file under …/python3.8/…, so I'm not sure what installed it. But neither of those files was there yesterday, so I'd guess that pycparser depends on whatever that package is being installed with the same python versions used to build pycparser.
Comment 7 Gary E. Miller 2020-09-26 23:20:58 UTC
Same error on dev-python/pycparser-2.20-r1 today.
Comment 8 Gary E. Miller 2020-09-26 23:23:19 UTC
emerging this a second time and it emerged fine...
Comment 9 Mikhail Krivtsov 2020-09-27 11:30:27 UTC
ERROR: dev-python/pycparser-2.20-r1::gentoo failed (compile phase)
ModuleNotFoundError: No module named 'lextab'

It is obvious that the problem is in 'pycparser' itself:
# equery belongs /usr/lib/python3.7/site-packages/pycparser/lextab.py
 * Searching for /usr/lib/python3.7/site-packages/pycparser/lextab.py ...
dev-python/pycparser-2.20 (/usr/lib/python3.7/site-packages/pycparser/lextab.py)

I had seen similar issue in different project. It was caused by incomplete dependencies specification. Build system tried to use item which was not built yet. As workaround we disabled parallel build in that project.

Build for 'dev-python/pycparser-2.20-r1' launched with 'emerge --jobs 1' finished successfully.
Comment 10 dE 2021-08-02 11:18:36 UTC
emerged this the 2nd time and it build.
Comment 11 Fabian Groffen gentoo-dev 2021-11-21 09:19:18 UTC
concurrency issue, somehow dependency isn't expressed

same here for pycparser-2.20-python3_9
Comment 12 Fabian Groffen gentoo-dev 2021-11-21 12:23:48 UTC
not sure if setup.py uses make, in any case, buildlog is here:
https://bootstrap.prefix.bitzolder.nl/results/x86_64-apple-darwin17/20211120/portage/dev-python/pycparser-2.20-r1/temp/build.log
Comment 13 Fabian Groffen gentoo-dev 2022-01-07 09:33:52 UTC
FWIW: seems fixed in 2.21
Comment 14 Thomas Sachau gentoo-dev 2022-01-12 20:32:04 UTC
(In reply to Fabian Groffen from comment #13)
> FWIW: seems fixed in 2.21

still happens with 2.21
Comment 15 Don Bishop 2022-05-06 15:30:07 UTC
pycparser-2.21 is still failing for missing module. 

'emerge --jobs=1 pucparser' works.
Comment 16 Gary E. Miller 2022-05-24 03:16:26 UTC
> pycparser-2.21 is still failing for missing module. 

Also failing pycparser-2.21-r1

> 'emerge --jobs=1 pucparser' works.

Works for me too.  Thanks!
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2022-06-12 05:25:34 UTC
Cleanup of bugs open for old versions.
Comment 18 Don Bishop 2022-09-20 01:57:54 UTC
On a new install of Gentoo today I ran into this exact problem. I pushed the eemrge until pycparser (2-21-r1) was the first in the --resume list then used --skipfirst to bypass it. i let another 15 or so packages install (this was part of adding gnuradio, soapysdr, and kicad to my system so 80pkgs or so in total).

End result: in another terminal, while the rest of the ebuild was motoring on, I did 'emerge -O1 pycparser' and it built and installed without incident.

The trouble was definitely rooted in not being able to build lextab.

This bug persists.
Comment 19 dE 2023-02-06 05:23:10 UTC
Issue persists. Hit on a system upgrade.
Comment 20 Larry the Git Cow gentoo-dev 2023-02-06 05:27:04 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9cab7f29a9f1ecd74c9e873b344fba255cbb390f

commit 9cab7f29a9f1ecd74c9e873b344fba255cbb390f
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-02-06 05:26:31 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-02-06 05:26:31 +0000

    dev-python/pycparser: force -j1
    
    Closes: https://bugs.gentoo.org/701878
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-python/pycparser/pycparser-2.21-r1.ebuild | 3 +++
 1 file changed, 3 insertions(+)
Comment 21 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-20 05:32:14 UTC
*** Bug 871969 has been marked as a duplicate of this bug. ***
Comment 22 Gary E. Miller 2023-02-21 20:46:19 UTC
I still have this issue with dev-python/pycparser-2.21-r1:0/0::gentoo,
Comment 23 Gary E. Miller 2023-02-21 20:47:11 UTC
Created attachment 853524 [details]
build.log

build.log
Comment 24 Gary E. Miller 2023-02-22 07:22:06 UTC
This does not work for me:

emerge --jobs=1 pycparser

But oddly, this did:

emerge -O1 pycparser
Comment 25 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-22 07:26:40 UTC
(In reply to Gary E. Miller from comment #24)
> This does not work for me:
> 
> emerge --jobs=1 pycparser

--jobs=1 controls emerge-level parallelism so it shouldn't make a difference, it's not the same as MAKEOPTS.

> 
> But oddly, this did:
> 
> emerge -O1 pycparser

That shouldn't make a difference to emerging pycparser by itself. It's more that it's a race and this happened to work. If you tried it in a loop, I suspect one would eventually fail.
Comment 26 Gary E. Miller 2023-02-22 07:49:09 UTC
(In reply to Sam James from comment #25)
> > emerge -O1 pycparser
> 
> That shouldn't make a difference to emerging pycparser by itself. It's more
> that it's a race and this happened to work. If you tried it in a loop, I
> suspect one would eventually fail.

You are correct  I tried it 5 times, and it failed 4 times.
Comment 27 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-22 15:10:04 UTC
I'm looking into it.
Comment 28 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-22 15:20:31 UTC
That said, I still can't reproduce.  I've put it in a loop now, and few runs passed with no problem so far.

I've inspected the code and there doesn't seem to be anything wrong with it.  FWICS lex() should either read existing lextab (which doesn't exist, so that fails) or write one reliably, using a context manager.  Nothing random to happen there.

So far my only guess is that Python has some import caching, and it doesn't "notice" that lextab.py appeared.  That said, if I can't reproduce it I'm going to need your help debugging it.
Comment 29 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-22 15:46:42 UTC
Created attachment 853916 [details, diff]
Debugging patch

Ok, I think technically ply could fail to write lextab and then put the error in some error log that probably isn't printed.

Could you please put the attached patch into /etc/portage/patches/dev-python/pycparser/ and attach/paste the output when you manage to make it fail again?

Note that this will only work with -r1.  I'm going to push -r2 that unbundles ply shortly, and the patch will not apply to it.  Please test with -r1.
Comment 30 Gary E. Miller 2023-02-22 21:40:34 UTC
(In reply to Michał Górny from comment #29)
> 
> Could you please put the attached patch into
> /etc/portage/patches/dev-python/pycparser/ and attach/paste the output when
> you manage to make it fail again?


I applied the patch.  Failed 5 out of 5 tries.
Comment 31 Gary E. Miller 2023-02-22 21:41:30 UTC
Created attachment 854016 [details]
build.log with patch

build log with patch.
Comment 32 Gary E. Miller 2023-02-22 21:49:59 UTC
I called the patch:

/etc/portage/patches/dev-python/pycparser/patch701878

Looking at the build log, I do not see it being applied.  What did I do wrong?
Comment 33 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-22 21:50:31 UTC
(In reply to Gary E. Miller from comment #32)
> I called the patch:
> 
> /etc/portage/patches/dev-python/pycparser/patch701878
> 
> Looking at the build log, I do not see it being applied.  What did I do
> wrong?

Ah, you have to give it a .patch (or .diff suffix) or Portage will ignore it, I think.
Comment 34 Gary E. Miller 2023-02-22 21:54:57 UTC
I renamed it:

/etc/portage/patches/dev-python/pycparser/print.patch

And it seems to apply now.

Running 10 times, got 10 failures.  Build log attached.
Comment 35 Gary E. Miller 2023-02-22 21:55:51 UTC
Created attachment 854018 [details]
build log with patch, try #2

build log with patch, try #2
Comment 36 Gary E. Miller 2023-02-22 22:19:14 UTC
Just for fun I tried:

nice -n 20 emerge -1 pycparser

And:

ionice emerge -1 pycparser

Hoping to jiggle any latency effects, but no change.

Also no effect:

MAKEOPTS="-j1"
Comment 37 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-23 06:07:51 UTC
Now that's weird.  I suppose:

  find /var/tmp/portage/dev-python/pycparser-2.21-r1/work/pycparser-2.21 -name lextab.py

yields nothing?

Can you try cloning the upstream git repo and testing there?  i.e.:

  git clone https://github.com/eliben/pycparser/
  cd pycparser
  python -m build -nw

?  If that succeeds, you may want to try release_v2.21 tag too.
Comment 38 Gary E. Miller 2023-02-23 07:02:36 UTC
(In reply to Michał Górny from comment #37)
> Now that's weird.  I suppose:
> 
>   find /var/tmp/portage/dev-python/pycparser-2.21-r1/work/pycparser-2.21
> -name lextab.py
> 
> yields nothing?

Yeah, nothing.  I already tried that, and similar.

> Can you try cloning the upstream git repo and testing there?  i.e.:
> 
>   git clone https://github.com/eliben/pycparser/
>   cd pycparser
>   python -m build -nw
> 
> ?  If that succeeds, you may want to try release_v2.21 tag too.

Fails.

````
dilbert /var/tmp #  git clone https://github.com/eliben/pycparser/
Cloning into 'pycparser'...
remote: Enumerating objects: 2907, done.
remote: Counting objects: 100% (417/417), done.
remote: Compressing objects: 100% (188/188), done.
remote: Total 2907 (delta 265), reused 338 (delta 212), pack-reused 2490
Receiving objects: 100% (2907/2907), 1.24 MiB | 3.36 MiB/s, done.
Resolving deltas: 100% (1942/1942), done.
dilbert /var/tmp # cd pycparser
dilbert /var/tmp/pycparser # python -m build -nw
/usr/lib/python-exec/python3.11/python: No module named build
dilbert /var/tmp/pycparser # 

````
Comment 39 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-23 13:07:17 UTC
You need to install dev-python/build first.
Comment 40 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-23 15:16:36 UTC
commit 638fa47d385b85c97485c8afd6d99186e73420e4
Author: Michał Górny <mgorny@gentoo.org>
Date:   Wed Feb 22 16:43:08 2023 +0100

    dev-python/pycparser: Unbundle dev-python/ply

    Signed-off-by: Michał Górny <mgorny@gentoo.org>

commit b78b8bbc63df017492b555c79695c14fa518ce53
Author: Michał Górny <mgorny@gentoo.org>
Date:   Wed Feb 22 16:00:28 2023 +0100

    dev-python/pycparser: Revert "force -j1"

    This does not affect the build in any way, we need to find the actual
    issue.

    Reverts: 9cab7f29a9f1ecd74c9e873b344fba255cbb390f
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

Anyway, let us know what happens with dev-python/build, ^ shouldn't fix it yet.
Comment 41 Gary E. Miller 2023-02-23 19:42:53 UTC
I see dev-python/pycparser-2.21-r2 is released.  Fails to emerge on that one problem host (dilbert).

Same bug.
Comment 42 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-23 19:43:44 UTC
(In reply to Gary E. Miller from comment #41)
> I see dev-python/pycparser-2.21-r2 is released.  Fails to emerge on that one
> problem host (dilbert).
> 
> Same bug.

Can you share the output when you try the git thing again with build installed?
Comment 43 Gary E. Miller 2023-02-23 19:43:56 UTC
Created attachment 854196 [details]
build log dev-python/pycparser-2.21-r2

build log dev-python/pycparser-2.21-r2

dilbert.
Comment 44 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-23 22:24:59 UTC
(In reply to Gary E. Miller from comment #43)
> Created attachment 854196 [details]
> build log dev-python/pycparser-2.21-r2
> 
> build log dev-python/pycparser-2.21-r2
> 
> dilbert.

I mean reattempt https://bugs.gentoo.org/701878#c38 with dev-python/build installed.
Comment 45 Gary E. Miller 2023-02-23 23:22:54 UTC
(In reply to Sam James from comment #44)
> (In reply to Gary E. Miller from comment #43)
> > Created attachment 854196 [details]
> > build log dev-python/pycparser-2.21-r2
> > 
> > build log dev-python/pycparser-2.21-r2
> > 
> > dilbert.
> 
> I mean reattempt https://bugs.gentoo.org/701878#c38 with dev-python/build
> installed.

dilbert /var/tmp/pycparser # python -m build -nw
* Getting build dependencies for wheel...
/usr/lib/python3.11/site-packages/setuptools/config/setupcfg.py:520: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
running egg_info
creating pycparser.egg-info
writing pycparser.egg-info/PKG-INFO
writing dependency_links to pycparser.egg-info/dependency_links.txt
writing top-level names to pycparser.egg-info/top_level.txt
writing manifest file 'pycparser.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'setup.pyc'
warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
adding license file 'LICENSE'
writing manifest file 'pycparser.egg-info/SOURCES.txt'
* Building wheel...
/usr/lib/python3.11/site-packages/setuptools/config/setupcfg.py:520: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/pycparser
copying pycparser/c_generator.py -> build/lib/pycparser
copying pycparser/_ast_gen.py -> build/lib/pycparser
copying pycparser/__init__.py -> build/lib/pycparser
copying pycparser/c_lexer.py -> build/lib/pycparser
copying pycparser/plyparser.py -> build/lib/pycparser
copying pycparser/c_ast.py -> build/lib/pycparser
copying pycparser/ast_transforms.py -> build/lib/pycparser
copying pycparser/_build_tables.py -> build/lib/pycparser
copying pycparser/c_parser.py -> build/lib/pycparser
creating build/lib/pycparser/ply
copying pycparser/ply/__init__.py -> build/lib/pycparser/ply
copying pycparser/ply/ygen.py -> build/lib/pycparser/ply
copying pycparser/ply/yacc.py -> build/lib/pycparser/ply
copying pycparser/ply/ctokens.py -> build/lib/pycparser/ply
copying pycparser/ply/cpp.py -> build/lib/pycparser/ply
copying pycparser/ply/lex.py -> build/lib/pycparser/ply
copying pycparser/_c_ast.cfg -> build/lib/pycparser
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_generator.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_ast_gen.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/__init__.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_lexer.py -> build/bdist.linux-x86_64/wheel/pycparser
creating build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/__init__.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/ygen.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/yacc.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/ctokens.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/cpp.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/lex.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/plyparser.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_ast.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/ast_transforms.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_c_ast.cfg -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_build_tables.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_parser.py -> build/bdist.linux-x86_64/wheel/pycparser
running install_egg_info
running egg_info
writing pycparser.egg-info/PKG-INFO
writing dependency_links to pycparser.egg-info/dependency_links.txt
writing top-level names to pycparser.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'setup.pyc'
warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
adding license file 'LICENSE'
writing manifest file 'pycparser.egg-info/SOURCES.txt'
Copying pycparser.egg-info to build/bdist.linux-x86_64/wheel/pycparser-2.21-py3.11.egg-info
running install_scripts
Build the lexing/parsing tables
creating build/bdist.linux-x86_64/wheel/pycparser-2.21.dist-info/WHEEL
creating '/var/tmp/pycparser/dist/.tmp-z6b5rxkn/pycparser-2.21-py2.py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'pycparser/__init__.py'
adding 'pycparser/_ast_gen.py'
adding 'pycparser/_build_tables.py'
adding 'pycparser/_c_ast.cfg'
adding 'pycparser/ast_transforms.py'
adding 'pycparser/c_ast.py'
adding 'pycparser/c_generator.py'
adding 'pycparser/c_lexer.py'
adding 'pycparser/c_parser.py'
adding 'pycparser/lextab.py'
adding 'pycparser/plyparser.py'
adding 'pycparser/yacctab.py'
adding 'pycparser/ply/__init__.py'
adding 'pycparser/ply/cpp.py'
adding 'pycparser/ply/ctokens.py'
adding 'pycparser/ply/lex.py'
adding 'pycparser/ply/yacc.py'
adding 'pycparser/ply/ygen.py'
adding 'pycparser-2.21.dist-info/LICENSE'
adding 'pycparser-2.21.dist-info/METADATA'
adding 'pycparser-2.21.dist-info/WHEEL'
adding 'pycparser-2.21.dist-info/top_level.txt'
adding 'pycparser-2.21.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built pycparser-2.21-py2.py3-none-any.whl
dilbert /var/tmp/pycparser #
Comment 46 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-24 04:43:26 UTC
Can you reproduce it if you 'git checkout 'v2.21' tag?  Can you reproduce it if you think the checkout on the same filesystem /var/tmp/portage?  What filesystem are you using there?
Comment 47 Gary E. Miller 2023-02-24 19:31:10 UTC
(In reply to Michał Górny from comment #46)
> Can you reproduce it if you 'git checkout 'v2.21' tag? 

dilbert /var/tmp/pycparser # git checkout 'v2.21'
error: pathspec 'v2.21' did not match any file(s) known to git

dilbert /var/tmp/pycparser # git checkout -b 'v2.21'
Switched to a new branch 'v2.21'

dilbert /var/tmp/pycparser # python -m build -nw
* Getting build dependencies for wheel...
/usr/lib/python3.11/site-packages/setuptools/config/setupcfg.py:520: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
running egg_info
writing pycparser.egg-info/PKG-INFO
writing dependency_links to pycparser.egg-info/dependency_links.txt
writing top-level names to pycparser.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'setup.pyc'
warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
adding license file 'LICENSE'
writing manifest file 'pycparser.egg-info/SOURCES.txt'
* Building wheel...
/usr/lib/python3.11/site-packages/setuptools/config/setupcfg.py:520: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_generator.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_ast_gen.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/__init__.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_lexer.py -> build/bdist.linux-x86_64/wheel/pycparser
creating build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/__init__.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/ygen.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/yacc.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/ctokens.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/cpp.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/lex.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/plyparser.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_ast.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/ast_transforms.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_c_ast.cfg -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_build_tables.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_parser.py -> build/bdist.linux-x86_64/wheel/pycparser
running install_egg_info
running egg_info
writing pycparser.egg-info/PKG-INFO
writing dependency_links to pycparser.egg-info/dependency_links.txt
writing top-level names to pycparser.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'setup.pyc'
warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
adding license file 'LICENSE'
writing manifest file 'pycparser.egg-info/SOURCES.txt'
Copying pycparser.egg-info to build/bdist.linux-x86_64/wheel/pycparser-2.21-py3.11.egg-info
running install_scripts
Build the lexing/parsing tables
Traceback (most recent call last):
  File "/var/tmp/pycparser/build/bdist.linux-x86_64/wheel/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
    main()
  File "/usr/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
    return _build_backend().build_wheel(wheel_directory, config_settings,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/build_meta.py", line 413, in build_wheel
    return self._build_with_temp_dir(['bdist_wheel'], '.whl',
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/build_meta.py", line 398, in _build_with_temp_dir
    self.run_setup()
  File "/usr/lib/python3.11/site-packages/setuptools/build_meta.py", line 485, in run_setup
    self).run_setup(setup_script=setup_script)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/build_meta.py", line 335, in run_setup
    exec(code, locals())
  File "<string>", line 35, in <module>
  File "/usr/lib/python3.11/site-packages/setuptools/__init__.py", line 108, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.11/site-packages/setuptools/dist.py", line 1221, in run_command
    super().run_command(command)
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 360, in run
    self.run_command("install")
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.11/site-packages/setuptools/dist.py", line 1221, in run_command
    super().run_command(command)
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "<string>", line 24, in run
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 339, in execute
    util.execute(func, args, msg, dry_run=self.dry_run)
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/util.py", line 337, in execute
    func(*args)
  File "<string>", line 17, in _run_build_tables
  File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/lib/python-exec/python3.11/python', '-B', '_build_tables.py']' returned non-zero exit status 1.

ERROR Backend subprocess exited when trying to invoke build_wheel
dilbert /var/tmp/pycparser #
Comment 48 Gary E. Miller 2023-02-24 19:32:17 UTC
> Can you reproduce it
> if you think the checkout on the same filesystem /var/tmp/portage?

And on /tmp:

dilbert /tmp #  git clone https://github.com/eliben/pycparser/
Cloning into 'pycparser'...
remote: Enumerating objects: 2912, done.
remote: Counting objects: 100% (422/422), done.
remote: Compressing objects: 100% (191/191), done.
remote: Total 2912 (delta 266), reused 338 (delta 212), pack-reused 2490
Receiving objects: 100% (2912/2912), 1.24 MiB | 3.65 MiB/s, done.
Resolving deltas: 100% (1943/1943), done.
dilbert /tmp # cd pycparser
dilbert /tmp/pycparser # git checkout -b 'v2.21'
Switched to a new branch 'v2.21'
dilbert /tmp/pycparser # python -m build -nw
* Getting build dependencies for wheel...
/usr/lib/python3.11/site-packages/setuptools/config/setupcfg.py:520: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
running egg_info
creating pycparser.egg-info
writing pycparser.egg-info/PKG-INFO
writing dependency_links to pycparser.egg-info/dependency_links.txt
writing top-level names to pycparser.egg-info/top_level.txt
writing manifest file 'pycparser.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'setup.pyc'
warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
adding license file 'LICENSE'
writing manifest file 'pycparser.egg-info/SOURCES.txt'
* Building wheel...
/usr/lib/python3.11/site-packages/setuptools/config/setupcfg.py:520: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/pycparser
copying pycparser/c_generator.py -> build/lib/pycparser
copying pycparser/_ast_gen.py -> build/lib/pycparser
copying pycparser/__init__.py -> build/lib/pycparser
copying pycparser/c_lexer.py -> build/lib/pycparser
copying pycparser/plyparser.py -> build/lib/pycparser
copying pycparser/c_ast.py -> build/lib/pycparser
copying pycparser/ast_transforms.py -> build/lib/pycparser
copying pycparser/_build_tables.py -> build/lib/pycparser
copying pycparser/c_parser.py -> build/lib/pycparser
creating build/lib/pycparser/ply
copying pycparser/ply/__init__.py -> build/lib/pycparser/ply
copying pycparser/ply/ygen.py -> build/lib/pycparser/ply
copying pycparser/ply/yacc.py -> build/lib/pycparser/ply
copying pycparser/ply/ctokens.py -> build/lib/pycparser/ply
copying pycparser/ply/cpp.py -> build/lib/pycparser/ply
copying pycparser/ply/lex.py -> build/lib/pycparser/ply
copying pycparser/_c_ast.cfg -> build/lib/pycparser
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_generator.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_ast_gen.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/__init__.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_lexer.py -> build/bdist.linux-x86_64/wheel/pycparser
creating build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/__init__.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/ygen.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/yacc.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/ctokens.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/cpp.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/ply/lex.py -> build/bdist.linux-x86_64/wheel/pycparser/ply
copying build/lib/pycparser/plyparser.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_ast.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/ast_transforms.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_c_ast.cfg -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/_build_tables.py -> build/bdist.linux-x86_64/wheel/pycparser
copying build/lib/pycparser/c_parser.py -> build/bdist.linux-x86_64/wheel/pycparser
running install_egg_info
running egg_info
writing pycparser.egg-info/PKG-INFO
writing dependency_links to pycparser.egg-info/dependency_links.txt
writing top-level names to pycparser.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'setup.pyc'
warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
adding license file 'LICENSE'
writing manifest file 'pycparser.egg-info/SOURCES.txt'
Copying pycparser.egg-info to build/bdist.linux-x86_64/wheel/pycparser-2.21-py3.11.egg-info
running install_scripts
Build the lexing/parsing tables
Traceback (most recent call last):
  File "/tmp/pycparser/build/bdist.linux-x86_64/wheel/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
    main()
  File "/usr/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
    return _build_backend().build_wheel(wheel_directory, config_settings,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/build_meta.py", line 413, in build_wheel
    return self._build_with_temp_dir(['bdist_wheel'], '.whl',
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/build_meta.py", line 398, in _build_with_temp_dir
    self.run_setup()
  File "/usr/lib/python3.11/site-packages/setuptools/build_meta.py", line 485, in run_setup
    self).run_setup(setup_script=setup_script)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/build_meta.py", line 335, in run_setup
    exec(code, locals())
  File "<string>", line 35, in <module>
  File "/usr/lib/python3.11/site-packages/setuptools/__init__.py", line 108, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.11/site-packages/setuptools/dist.py", line 1221, in run_command
    super().run_command(command)
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 360, in run
    self.run_command("install")
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.11/site-packages/setuptools/dist.py", line 1221, in run_command
    super().run_command(command)
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "<string>", line 24, in run
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 339, in execute
    util.execute(func, args, msg, dry_run=self.dry_run)
  File "/usr/lib/python3.11/site-packages/setuptools/_distutils/util.py", line 337, in execute
    func(*args)
  File "<string>", line 17, in _run_build_tables
  File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/lib/python-exec/python3.11/python', '-B', '_build_tables.py']' returned non-zero exit status 1.

ERROR Backend subprocess exited when trying to invoke build_wheel
dilbert /tmp/pycparser # 


>  What
> filesystem are you using there?

dilbert /tmp/pycparser # df -h
Filesystem                Size  Used Avail Use% Mounted on
/dev/root                 431G  262G  150G  64% /
devtmpfs                   10M     0   10M   0% /dev
tmpfs                     6.3G  6.2M  6.3G   1% /run
cgroup_root                10M     0   10M   0% /sys/fs/cgroup
shm                        16G   31M   16G   1% /dev/shm
/dev/nvme0n1p2            511M  221M  291M  44% /boot
tmpfs                     3.2G   24K  3.2G   1% /run/user/1000

/var/tmp is in same partition as /.  / is ext4.

/tmp is a tmpfs, with noexec.
Comment 49 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-24 19:45:45 UTC
Sorry, I meant 'release_v2.21' tag.
Comment 50 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-24 19:47:12 UTC
(In reply to Michał Górny from comment #49)
> Sorry, I meant 'release_v2.21' tag.

…but that doesn't matter.
Comment 51 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-24 19:48:48 UTC
Do you have any "special" environment variables set, particularly anything Python-related?  If that's no secret, could you paste the output of env(1) as the user you're running the failing tests as.
Comment 52 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-24 19:50:51 UTC
Inside the git checkout, could you try:

  cd pycparser  # i.e. the pycparser subdirectory inside the repository
  python -B _build_tables.py

If it fails, paste the output of `ls`.
Comment 53 Gary E. Miller 2023-02-24 20:08:43 UTC
(In reply to Michał Górny from comment #50)
> (In reply to Michał Górny from comment #49)
> > Sorry, I meant 'release_v2.21' tag.
> 
> …but that doesn't matter.

Do I have an action item here?
Comment 54 Gary E. Miller 2023-02-24 20:10:39 UTC
(In reply to Michał Górny from comment #51)
> Do you have any "special" environment variables set, particularly anything
> Python-related?  If that's no secret, could you paste the output of env(1)
> as the user you're running the failing tests as.

I run my tests as root.  Hard to emerge otherwise...

Nothing special, just a fast laptop, running Gentoo unstable:

dilbert /var/tmp/pycparser # env
SHELL=/bin/bash
XDG_CONFIG_DIRS=/etc/xdg
LESS=-R -M --shift 5
CONFIG_PROTECT_MASK=/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/terminfo /etc/dconf /etc/ca-certificates.conf /etc/texmf/web2c /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/revdep-rebuild
VGL_READBACK=pbo
EDITOR=/usr/bin/vi
PWD=/var/tmp/pycparser
CONFIG_PROTECT=/usr/share/gnupg/qualified.txt /opt/brother/scanner/brscan4/brsanenetdevice4.cfg /usr/lib64/libreoffice/program/sofficerc
LOGNAME=root
MANPATH=/usr/share/gcc-data/x86_64-pc-linux-gnu/12/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.40/man:/usr/local/share/man:/usr/share/man:/usr/lib/rust/man:/usr/lib/llvm/15/share/man
XAUTHORITY=/root/.xauthap6nDv
OPENCL_PROFILE=nvidia
HOME=/root
LANG=en_US.utf8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.cfg=00;32:*.conf=00;32:*.diff=00;32:*.doc=00;32:*.ini=00;32:*.log=00;32:*.patch=00;32:*.pdf=00;32:*.ps=00;32:*.tex=00;32:*.txt=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.swp=00;90:*.tmp=00;90:*.dpkg-dist=00;90:*.dpkg-old=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:
OPENGL_PROFILE=xorg-x11
INFOPATH=/usr/share/gcc-data/x86_64-pc-linux-gnu/12/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.40/info:/usr/share/info
MOZ_GMP_PATH=/usr/lib64/nsbrowser/plugins/gmp-gmpopenh264/system-installed
PYTHONPATH=/usr/local/lib/python3.10/site-packages/
TERM=xterm-256color
LESSOPEN=|lesspipe %s
USER=root
MANPAGER=manpager
VISUAL=/usr/bin/vi
NPM_CONFIG_GLOBALCONFIG=/etc/npm/npmrc
DISPLAY=:0.0
SHLVL=1
PAGER=/usr/bin/less
LEX=flex
GCC_SPECS=
GSETTINGS_BACKEND=dconf
XDG_DATA_DIRS=/usr/local/share:/usr/share
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/15/bin
VBOX_APP_HOME=/usr/lib64/virtualbox
LV2_PATH=/usr/lib64/lv2
OLDPWD=/tmp/ice-1.11.14/src
LADSPA_PATH=/usr/lib64/ladspa
_=/usr/bin/env
dilbert /var/tmp/pycparser #
Comment 55 Gary E. Miller 2023-02-24 20:12:38 UTC
(In reply to Michał Górny from comment #52)
> Inside the git checkout, could you try:
> 
>   cd pycparser  # i.e. the pycparser subdirectory inside the repository
>   python -B _build_tables.py
> 
> If it fails, paste the output of `ls`.

dilbert /var/tmp/pycparser/pycparser #   python -B _build_tables.py
dilbert /var/tmp/pycparser/pycparser # 

dilbert /var/tmp/pycparser/pycparser # ls
_ast_gen.py        _c_ast.cfg      c_lexer.py   lextab.py     yacctab.py
ast_transforms.py  c_ast.py        c_parser.py  ply
_build_tables.py   c_generator.py  __init__.py  plyparser.py
dilbert /var/tmp/pycparser/pycparser #
Comment 56 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-24 20:13:08 UTC
>PYTHONPATH=/usr/local/lib/python3.10/site-packages/

This will interfere with emerges.
Comment 57 Gary E. Miller 2023-02-24 20:20:03 UTC
(In reply to Sam James from comment #56)
> >PYTHONPATH=/usr/local/lib/python3.10/site-packages/
> 
> This will interfere with emerges.

I have done that for ages, on dozens of hosts.  The only things there are gpsd and ntpsec.  I wipe site-packages frequently.

Python will look there, then continue it's normal searsh for modules.
Comment 58 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 05:53:10 UTC
Please test if the package fails to build with PYTHONPATH unset.
Comment 59 Gary E. Miller 2023-02-25 06:35:51 UTC
(In reply to Michał Górny from comment #58)
> Please test if the package fails to build with PYTHONPATH unset.

As expected, unsetting PYTHONPATH has no effect.
Comment 60 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 07:00:31 UTC
I'm afraid I have no clue what's happening here.  FWICS things work when you call _build_tables.py manually but they don't when setup.py calls it (via build frontend).

Could you perhaps could try a few times to remove lextab.py, then run _build_tables.py again and see if it perhaps fails 1 out of N attempts?
Comment 61 Gary E. Miller 2023-02-25 19:20:36 UTC
(In reply to Michał Górny from comment #60)
> I'm afraid I have no clue what's happening here.  FWICS things work when you
> call _build_tables.py manually but they don't when setup.py calls it (via
> build frontend).
> 
> Could you perhaps could try a few times to remove lextab.py, then run
> _build_tables.py again and see if it perhaps fails 1 out of N attempts?

Very interesting.  This works, repeatedly:

dilbert /var/tmp/pycparser/pycparser # python -B _build_tables.py
dilbert /var/tmp/pycparser/pycparser # python -B _build_tables.py
dilbert /var/tmp/pycparser/pycparser # python -B _build_tables.py
dilbert /var/tmp/pycparser/pycparser # python -B _build_tables.py
dilbert /var/tmp/pycparser/pycparser # python -B _build_tables.py
dilbert /var/tmp/pycparser/pycparser # rm lextab.py 
dilbert /var/tmp/pycparser/pycparser # python -B _build_tables.py
dilbert /var/tmp/pycparser/pycparser # python -B _build_tables.py


But not this:

dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser #

Curiouser, and curiouser...
Comment 62 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 19:48:53 UTC
Do I understand it correctly that it works if there's some delay between `rm` and `python` but fails if they are quickly executed one after another? oO

Does `lextab.py` exist then after the failed run?

Are you running the latest kernel?  Perhaps you're hitting some kernel bug?
Comment 63 Gary E. Miller 2023-02-25 19:58:25 UTC
(In reply to Michał Górny from comment #62)
> Do I understand it correctly that it works if there's some delay between
> `rm` and `python` but fails if they are quickly executed one after another?
> oO

Seems to be.

> Does `lextab.py` exist then after the failed run?

Interesting idea:

dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py; ls lextab.py
lextab.py
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py; ls lextab.py
lextab.py
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py; ls lextab.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
lextab.py
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py; ls lextab.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
lextab.py
dilbert /var/tmp/pycparser/pycparser # rm lextab.py ; python -B _build_tables.py; ls lextab.py
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
lextab.py

> Are you running the latest kernel?  Perhaps you're hitting some kernel bug?

I am running 6.2.0, am about to update to 6.2.1.  Can't be any specific
kernel version, I've had this isse for years.
Comment 64 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 20:11:02 UTC
How about:

  rm lextab.py ; strace python -B _build_tables.py |& tee wtf.txt

The file's gonna be a bit large but I think:

  grep -C10 lextab\.py wtf.txt

will suffice (from a failed run).
Comment 65 Gary E. Miller 2023-02-25 20:16:18 UTC
(In reply to Michał Górny from comment #64)
> How about:
> 
>   rm lextab.py ; strace python -B _build_tables.py |& tee wtf.txt
> 
> The file's gonna be a bit large but I think:
> 
>   grep -C10 lextab\.py wtf.txt
> 
> will suffice (from a failed run).

dilbert /var/tmp/pycparser/pycparser #   grep -C10 lextab\.py wtf.txt
openat(AT_FDCWD, "/var/tmp/pycparser/pycparser/../pycparser/ast_transforms.py", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=5691, ...}, AT_EMPTY_PATH) = 0
ioctl(3, TCGETS, 0x7ffebeb897b0)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=5691, ...}, AT_EMPTY_PATH) = 0
read(3, "#-------------------------------"..., 5692) = 5691
read(3, "", 1)                          = 0
close(3)                                = 0
newfstatat(AT_FDCWD, "/var/tmp/pycparser/pycparser/../pycparser", {st_mode=S_IFDIR|0755, st_size=1024, ...}, 0) = 0
openat(AT_FDCWD, "lextab.py", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_EMPTY_PATH) = 0
ioctl(3, TCGETS, 0x7ffebeb8a590)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
write(3, "# lextab.py. This file automatic"..., 1504) = 1504
write(3, "_lexstatere   = {'INITIAL': [('("..., 7050) = 7050
close(3)                                = 0
newfstatat(AT_FDCWD, "/var/tmp/pycparser/pycparser/../pycparser", {st_mode=S_IFDIR|0755, st_size=1024, ...}, 0) = 0
newfstatat(AT_FDCWD, "/var/tmp/pycparser/pycparser/../pycparser/yacctab.py", {st_mode=S_IFREG|0644, st_size=209738, ...}, 0) = 0
newfstatat(AT_FDCWD, "/var/tmp/pycparser/pycparser/../pycparser/yacctab.py", {st_mode=S_IFREG|0644, st_size=209738, ...}, 0) = 0
openat(AT_FDCWD, "/var/tmp/pycparser/pycparser/../pycparser/__pycache__/yacctab.cpython-311.pyc", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/var/tmp/pycparser/pycparser/../pycparser/yacctab.py", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=209738, ...}, AT_EMPTY_PATH) = 0
ioctl(3, TCGETS, 0x7ffebeb89e80)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
dilbert /var/tmp/pycparser/pycparser #
Comment 66 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 20:22:36 UTC
Created attachment 854656 [details, diff]
import-debug.patch

Ok, so apparently it doesn't even try to open it afterwards.

Could you apply the following patch to your git checkout and try:

  rm lextab.py ; python -B _build_tables.py

again?  It should print a bunch of import-related variables.
Comment 67 Gary E. Miller 2023-02-25 20:27:55 UTC
Created attachment 854658 [details]
rm lextab.py ; strace -o trace.log  python -B _build_tables.py

strace log of failure.
Comment 68 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 20:31:33 UTC
Another idea:

  stat . ; rm lextab.py ; stat . ; python -B _build_tables.py ; stat .
Comment 69 Gary E. Miller 2023-02-25 20:37:02 UTC
(In reply to Michał Górny from comment #66)
> Created attachment 854656 [details, diff] [details, diff]
> import-debug.patch
> 
> Ok, so apparently it doesn't even try to open it afterwards.
> 
> Could you apply the following patch to your git checkout and try:
> 
>   rm lextab.py ; python -B _build_tables.py
> 
> again?  It should print a bunch of import-related variables.


A Fail:

dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ; python -B _build_tables.py
modules = {'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': <module '_frozen_importlib' (frozen)>, '_imp': <module '_imp' (built-in)>, '_thread': <module '_thread' (built-in)>, '_warnings': <module '_warnings' (built-in)>, '_weakref': <module '_weakref' (built-in)>, '_io': <module '_io' (built-in)>, 'marshal': <module 'marshal' (built-in)>, 'posix': <module 'posix' (built-in)>, '_frozen_importlib_external': <module '_frozen_importlib_external' (frozen)>, 'time': <module 'time' (built-in)>, 'zipimport': <module 'zipimport' (frozen)>, '_codecs': <module '_codecs' (built-in)>, 'codecs': <module 'codecs' (frozen)>, 'encodings.aliases': <module 'encodings.aliases' from '/usr/lib/python3.11/encodings/aliases.py'>, 'encodings': <module 'encodings' from '/usr/lib/python3.11/encodings/__init__.py'>, 'encodings.utf_8': <module 'encodings.utf_8' from '/usr/lib/python3.11/encodings/utf_8.py'>, '_signal': <module '_signal' (built-in)>, '_abc': <module '_abc' (built-in)>, 'abc': <module 'abc' (frozen)>, 'io': <module 'io' (frozen)>, '__main__': <module '__main__' from '/var/tmp/pycparser/pycparser/_build_tables.py'>, '_stat': <module '_stat' (built-in)>, 'stat': <module 'stat' (frozen)>, '_collections_abc': <module '_collections_abc' (frozen)>, 'genericpath': <module 'genericpath' (frozen)>, 'posixpath': <module 'posixpath' (frozen)>, 'os.path': <module 'posixpath' (frozen)>, 'os': <module 'os' (frozen)>, '_sitebuiltins': <module '_sitebuiltins' (frozen)>, '_distutils_hack': <module '_distutils_hack' from '/usr/lib/python3.11/site-packages/_distutils_hack/__init__.py'>, 'site': <module 'site' (frozen)>, '_string': <module '_string' (built-in)>, 'types': <module 'types' from '/usr/lib/python3.11/types.py'>, '_operator': <module '_operator' (built-in)>, 'operator': <module 'operator' from '/usr/lib/python3.11/operator.py'>, 'itertools': <module 'itertools' (built-in)>, 'keyword': <module 'keyword' from '/usr/lib/python3.11/keyword.py'>, 'reprlib': <module 'reprlib' from '/usr/lib/python3.11/reprlib.py'>, '_collections': <module '_collections' (built-in)>, 'collections': <module 'collections' from '/usr/lib/python3.11/collections/__init__.py'>, '_functools': <module '_functools' (built-in)>, 'functools': <module 'functools' from '/usr/lib/python3.11/functools.py'>, 'enum': <module 'enum' from '/usr/lib/python3.11/enum.py'>, '_sre': <module '_sre' (built-in)>, 're._constants': <module 're._constants' from '/usr/lib/python3.11/re/_constants.py'>, 're._parser': <module 're._parser' from '/usr/lib/python3.11/re/_parser.py'>, 're._casefix': <module 're._casefix' from '/usr/lib/python3.11/re/_casefix.py'>, 're._compiler': <module 're._compiler' from '/usr/lib/python3.11/re/_compiler.py'>, 'copyreg': <module 'copyreg' from '/usr/lib/python3.11/copyreg.py'>, 're': <module 're' from '/usr/lib/python3.11/re/__init__.py'>, 'string': <module 'string' from '/usr/lib/python3.11/string.py'>, '_ast_gen': <module '_ast_gen' from '/var/tmp/pycparser/pycparser/_ast_gen.py'>, 'errno': <module 'errno' (built-in)>, '_locale': <module '_locale' (built-in)>, 'locale': <module 'locale' from '/usr/lib/python3.11/locale.py'>, 'signal': <module 'signal' from '/usr/lib/python3.11/signal.py'>, '_weakrefset': <module '_weakrefset' from '/usr/lib/python3.11/_weakrefset.py'>, 'threading': <module 'threading' from '/usr/lib/python3.11/threading.py'>, 'warnings': <module 'warnings' from '/usr/lib/python3.11/warnings.py'>, 'contextlib': <module 'contextlib' from '/usr/lib/python3.11/contextlib.py'>, 'fcntl': <module 'fcntl' from '/usr/lib/python3.11/lib-dynload/fcntl.cpython-311-x86_64-linux-gnu.so'>, '_posixsubprocess': <module '_posixsubprocess' from '/usr/lib/python3.11/lib-dynload/_posixsubprocess.cpython-311-x86_64-linux-gnu.so'>, 'select': <module 'select' from '/usr/lib/python3.11/lib-dynload/select.cpython-311-x86_64-linux-gnu.so'>, 'collections.abc': <module 'collections.abc' from '/usr/lib/python3.11/collections/abc.py'>, 'math': <module 'math' from '/usr/lib/python3.11/lib-dynload/math.cpython-311-x86_64-linux-gnu.so'>, 'selectors': <module 'selectors' from '/usr/lib/python3.11/selectors.py'>, 'subprocess': <module 'subprocess' from '/usr/lib/python3.11/subprocess.py'>, 'pycparser.ply': <module 'pycparser.ply' from '/var/tmp/pycparser/pycparser/../pycparser/ply/__init__.py'>, '_ast': <module '_ast' (built-in)>, 'ast': <module 'ast' from '/usr/lib/python3.11/ast.py'>, '_opcode': <module '_opcode' from '/usr/lib/python3.11/lib-dynload/_opcode.cpython-311-x86_64-linux-gnu.so'>, 'opcode': <module 'opcode' from '/usr/lib/python3.11/opcode.py'>, 'dis': <module 'dis' from '/usr/lib/python3.11/dis.py'>, 'importlib._bootstrap': <module '_frozen_importlib' (frozen)>, 'importlib._bootstrap_external': <module '_frozen_importlib_external' (frozen)>, 'importlib': <module 'importlib' from '/usr/lib/python3.11/importlib/__init__.py'>, 'importlib.machinery': <module 'importlib.machinery' (frozen)>, 'token': <module 'token' from '/usr/lib/python3.11/token.py'>, 'tokenize': <module 'tokenize' from '/usr/lib/python3.11/tokenize.py'>, 'linecache': <module 'linecache' from '/usr/lib/python3.11/linecache.py'>, 'inspect': <module 'inspect' from '/usr/lib/python3.11/inspect.py'>, '_struct': <module '_struct' from '/usr/lib/python3.11/lib-dynload/_struct.cpython-311-x86_64-linux-gnu.so'>, 'struct': <module 'struct' from '/usr/lib/python3.11/struct.py'>, 'binascii': <module 'binascii' from '/usr/lib/python3.11/lib-dynload/binascii.cpython-311-x86_64-linux-gnu.so'>, 'base64': <module 'base64' from '/usr/lib/python3.11/base64.py'>, 'pycparser.ply.yacc': <module 'pycparser.ply.yacc' from '/var/tmp/pycparser/pycparser/../pycparser/ply/yacc.py'>, 'pycparser.c_ast': <module 'pycparser.c_ast' from '/var/tmp/pycparser/pycparser/../pycparser/c_ast.py'>, 'weakref': <module 'weakref' from '/usr/lib/python3.11/weakref.py'>, 'copy': <module 'copy' from '/usr/lib/python3.11/copy.py'>, 'pycparser.ply.lex': <module 'pycparser.ply.lex' from '/var/tmp/pycparser/pycparser/../pycparser/ply/lex.py'>, 'pycparser.c_lexer': <module 'pycparser.c_lexer' from '/var/tmp/pycparser/pycparser/../pycparser/c_lexer.py'>, 'pycparser.plyparser': <module 'pycparser.plyparser' from '/var/tmp/pycparser/pycparser/../pycparser/plyparser.py'>, 'pycparser.ast_transforms': <module 'pycparser.ast_transforms' from '/var/tmp/pycparser/pycparser/../pycparser/ast_transforms.py'>, 'pycparser.c_parser': <module 'pycparser.c_parser' from '/var/tmp/pycparser/pycparser/../pycparser/c_parser.py'>, 'pycparser': <module 'pycparser' from '/var/tmp/pycparser/pycparser/../pycparser/__init__.py'>, 'pycparser.yacctab': <module 'pycparser.yacctab' from '/var/tmp/pycparser/pycparser/../pycparser/yacctab.py'>}
meta_path = [<_distutils_hack.DistutilsMetaFinder object at 0x7f88eaa20d10>, <class '_frozen_importlib.BuiltinImporter'>, <class '_frozen_importlib.FrozenImporter'>, <class '_frozen_importlib_external.PathFinder'>]
path = ['.', '..', '/var/tmp/pycparser/pycparser', '/usr/local/lib/python3.10/site-packages', '/usr/lib/python311.zip', '/usr/lib/python3.11', '/usr/lib/python3.11/lib-dynload', '/usr/lib/python3.11/site-packages']
path_hooks = [<class 'zipimport.zipimporter'>, <function FileFinder.path_hook.<locals>.path_hook_for_FileFinder at 0x7f88eab982c0>]
path_importer_cache = {'/usr/local/lib/python3.10/site-packages': FileFinder('/usr/local/lib/python3.10/site-packages'), '/usr/lib/python311.zip': None, '/usr/lib/python3.11': FileFinder('/usr/lib/python3.11'), '/usr/lib/python3.11/encodings': FileFinder('/usr/lib/python3.11/encodings'), '/usr/lib/python3.11/lib-dynload': FileFinder('/usr/lib/python3.11/lib-dynload'), '/usr/lib/python3.11/site-packages': FileFinder('/usr/lib/python3.11/site-packages'), '/var/tmp/pycparser/pycparser/_build_tables.py': None, '.': FileFinder('/var/tmp/pycparser/pycparser'), '..': FileFinder('/var/tmp/pycparser/pycparser/..'), '/var/tmp/pycparser/pycparser': FileFinder('/var/tmp/pycparser/pycparser'), '/usr/lib/python3.11/re': FileFinder('/usr/lib/python3.11/re'), '/usr/lib/python3.11/collections': FileFinder('/usr/lib/python3.11/collections'), '/var/tmp/pycparser/pycparser/../pycparser': FileFinder('/var/tmp/pycparser/pycparser/../pycparser'), '/var/tmp/pycparser/pycparser/../pycparser/ply': FileFinder('/var/tmp/pycparser/pycparser/../pycparser/ply')}
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 38, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
Comment 70 Gary E. Miller 2023-02-25 20:37:48 UTC
(In reply to Michał Górny from comment #68)
> Another idea:
> 
>   stat . ; rm lextab.py ; stat . ; python -B _build_tables.py ; stat .

A fail:

dilbert /var/tmp/pycparser/pycparser #   stat . ; rm lextab.py ; stat . ; python -B _build_tables.py ; stat .
  File: .
  Size: 1024      	Blocks: 2          IO Block: 1024   directory
Device: 259,4	Inode: 51741431    Links: 3
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-02-24 12:13:18.000000000 -0800
Modify: 2023-02-25 12:34:28.000000000 -0800
Change: 2023-02-25 12:34:28.000000000 -0800
 Birth: -
  File: .
  Size: 1024      	Blocks: 2          IO Block: 1024   directory
Device: 259,4	Inode: 51741431    Links: 3
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-02-24 12:13:18.000000000 -0800
Modify: 2023-02-25 12:34:56.000000000 -0800
Change: 2023-02-25 12:34:56.000000000 -0800
 Birth: -
modules = {'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': <module '_frozen_importlib' (frozen)>, '_imp': <module '_imp' (built-in)>, '_thread': <module '_thread' (built-in)>, '_warnings': <module '_warnings' (built-in)>, '_weakref': <module '_weakref' (built-in)>, '_io': <module '_io' (built-in)>, 'marshal': <module 'marshal' (built-in)>, 'posix': <module 'posix' (built-in)>, '_frozen_importlib_external': <module '_frozen_importlib_external' (frozen)>, 'time': <module 'time' (built-in)>, 'zipimport': <module 'zipimport' (frozen)>, '_codecs': <module '_codecs' (built-in)>, 'codecs': <module 'codecs' (frozen)>, 'encodings.aliases': <module 'encodings.aliases' from '/usr/lib/python3.11/encodings/aliases.py'>, 'encodings': <module 'encodings' from '/usr/lib/python3.11/encodings/__init__.py'>, 'encodings.utf_8': <module 'encodings.utf_8' from '/usr/lib/python3.11/encodings/utf_8.py'>, '_signal': <module '_signal' (built-in)>, '_abc': <module '_abc' (built-in)>, 'abc': <module 'abc' (frozen)>, 'io': <module 'io' (frozen)>, '__main__': <module '__main__' from '/var/tmp/pycparser/pycparser/_build_tables.py'>, '_stat': <module '_stat' (built-in)>, 'stat': <module 'stat' (frozen)>, '_collections_abc': <module '_collections_abc' (frozen)>, 'genericpath': <module 'genericpath' (frozen)>, 'posixpath': <module 'posixpath' (frozen)>, 'os.path': <module 'posixpath' (frozen)>, 'os': <module 'os' (frozen)>, '_sitebuiltins': <module '_sitebuiltins' (frozen)>, '_distutils_hack': <module '_distutils_hack' from '/usr/lib/python3.11/site-packages/_distutils_hack/__init__.py'>, 'site': <module 'site' (frozen)>, '_string': <module '_string' (built-in)>, 'types': <module 'types' from '/usr/lib/python3.11/types.py'>, '_operator': <module '_operator' (built-in)>, 'operator': <module 'operator' from '/usr/lib/python3.11/operator.py'>, 'itertools': <module 'itertools' (built-in)>, 'keyword': <module 'keyword' from '/usr/lib/python3.11/keyword.py'>, 'reprlib': <module 'reprlib' from '/usr/lib/python3.11/reprlib.py'>, '_collections': <module '_collections' (built-in)>, 'collections': <module 'collections' from '/usr/lib/python3.11/collections/__init__.py'>, '_functools': <module '_functools' (built-in)>, 'functools': <module 'functools' from '/usr/lib/python3.11/functools.py'>, 'enum': <module 'enum' from '/usr/lib/python3.11/enum.py'>, '_sre': <module '_sre' (built-in)>, 're._constants': <module 're._constants' from '/usr/lib/python3.11/re/_constants.py'>, 're._parser': <module 're._parser' from '/usr/lib/python3.11/re/_parser.py'>, 're._casefix': <module 're._casefix' from '/usr/lib/python3.11/re/_casefix.py'>, 're._compiler': <module 're._compiler' from '/usr/lib/python3.11/re/_compiler.py'>, 'copyreg': <module 'copyreg' from '/usr/lib/python3.11/copyreg.py'>, 're': <module 're' from '/usr/lib/python3.11/re/__init__.py'>, 'string': <module 'string' from '/usr/lib/python3.11/string.py'>, '_ast_gen': <module '_ast_gen' from '/var/tmp/pycparser/pycparser/_ast_gen.py'>, 'errno': <module 'errno' (built-in)>, '_locale': <module '_locale' (built-in)>, 'locale': <module 'locale' from '/usr/lib/python3.11/locale.py'>, 'signal': <module 'signal' from '/usr/lib/python3.11/signal.py'>, '_weakrefset': <module '_weakrefset' from '/usr/lib/python3.11/_weakrefset.py'>, 'threading': <module 'threading' from '/usr/lib/python3.11/threading.py'>, 'warnings': <module 'warnings' from '/usr/lib/python3.11/warnings.py'>, 'contextlib': <module 'contextlib' from '/usr/lib/python3.11/contextlib.py'>, 'fcntl': <module 'fcntl' from '/usr/lib/python3.11/lib-dynload/fcntl.cpython-311-x86_64-linux-gnu.so'>, '_posixsubprocess': <module '_posixsubprocess' from '/usr/lib/python3.11/lib-dynload/_posixsubprocess.cpython-311-x86_64-linux-gnu.so'>, 'select': <module 'select' from '/usr/lib/python3.11/lib-dynload/select.cpython-311-x86_64-linux-gnu.so'>, 'collections.abc': <module 'collections.abc' from '/usr/lib/python3.11/collections/abc.py'>, 'math': <module 'math' from '/usr/lib/python3.11/lib-dynload/math.cpython-311-x86_64-linux-gnu.so'>, 'selectors': <module 'selectors' from '/usr/lib/python3.11/selectors.py'>, 'subprocess': <module 'subprocess' from '/usr/lib/python3.11/subprocess.py'>, 'pycparser.ply': <module 'pycparser.ply' from '/var/tmp/pycparser/pycparser/../pycparser/ply/__init__.py'>, '_ast': <module '_ast' (built-in)>, 'ast': <module 'ast' from '/usr/lib/python3.11/ast.py'>, '_opcode': <module '_opcode' from '/usr/lib/python3.11/lib-dynload/_opcode.cpython-311-x86_64-linux-gnu.so'>, 'opcode': <module 'opcode' from '/usr/lib/python3.11/opcode.py'>, 'dis': <module 'dis' from '/usr/lib/python3.11/dis.py'>, 'importlib._bootstrap': <module '_frozen_importlib' (frozen)>, 'importlib._bootstrap_external': <module '_frozen_importlib_external' (frozen)>, 'importlib': <module 'importlib' from '/usr/lib/python3.11/importlib/__init__.py'>, 'importlib.machinery': <module 'importlib.machinery' (frozen)>, 'token': <module 'token' from '/usr/lib/python3.11/token.py'>, 'tokenize': <module 'tokenize' from '/usr/lib/python3.11/tokenize.py'>, 'linecache': <module 'linecache' from '/usr/lib/python3.11/linecache.py'>, 'inspect': <module 'inspect' from '/usr/lib/python3.11/inspect.py'>, '_struct': <module '_struct' from '/usr/lib/python3.11/lib-dynload/_struct.cpython-311-x86_64-linux-gnu.so'>, 'struct': <module 'struct' from '/usr/lib/python3.11/struct.py'>, 'binascii': <module 'binascii' from '/usr/lib/python3.11/lib-dynload/binascii.cpython-311-x86_64-linux-gnu.so'>, 'base64': <module 'base64' from '/usr/lib/python3.11/base64.py'>, 'pycparser.ply.yacc': <module 'pycparser.ply.yacc' from '/var/tmp/pycparser/pycparser/../pycparser/ply/yacc.py'>, 'pycparser.c_ast': <module 'pycparser.c_ast' from '/var/tmp/pycparser/pycparser/../pycparser/c_ast.py'>, 'weakref': <module 'weakref' from '/usr/lib/python3.11/weakref.py'>, 'copy': <module 'copy' from '/usr/lib/python3.11/copy.py'>, 'pycparser.ply.lex': <module 'pycparser.ply.lex' from '/var/tmp/pycparser/pycparser/../pycparser/ply/lex.py'>, 'pycparser.c_lexer': <module 'pycparser.c_lexer' from '/var/tmp/pycparser/pycparser/../pycparser/c_lexer.py'>, 'pycparser.plyparser': <module 'pycparser.plyparser' from '/var/tmp/pycparser/pycparser/../pycparser/plyparser.py'>, 'pycparser.ast_transforms': <module 'pycparser.ast_transforms' from '/var/tmp/pycparser/pycparser/../pycparser/ast_transforms.py'>, 'pycparser.c_parser': <module 'pycparser.c_parser' from '/var/tmp/pycparser/pycparser/../pycparser/c_parser.py'>, 'pycparser': <module 'pycparser' from '/var/tmp/pycparser/pycparser/../pycparser/__init__.py'>, 'pycparser.yacctab': <module 'pycparser.yacctab' from '/var/tmp/pycparser/pycparser/../pycparser/yacctab.py'>}
meta_path = [<_distutils_hack.DistutilsMetaFinder object at 0x7fdbeb7c0d10>, <class '_frozen_importlib.BuiltinImporter'>, <class '_frozen_importlib.FrozenImporter'>, <class '_frozen_importlib_external.PathFinder'>]
path = ['.', '..', '/var/tmp/pycparser/pycparser', '/usr/local/lib/python3.10/site-packages', '/usr/lib/python311.zip', '/usr/lib/python3.11', '/usr/lib/python3.11/lib-dynload', '/usr/lib/python3.11/site-packages']
path_hooks = [<class 'zipimport.zipimporter'>, <function FileFinder.path_hook.<locals>.path_hook_for_FileFinder at 0x7fdbeb9982c0>]
path_importer_cache = {'/usr/local/lib/python3.10/site-packages': FileFinder('/usr/local/lib/python3.10/site-packages'), '/usr/lib/python311.zip': None, '/usr/lib/python3.11': FileFinder('/usr/lib/python3.11'), '/usr/lib/python3.11/encodings': FileFinder('/usr/lib/python3.11/encodings'), '/usr/lib/python3.11/lib-dynload': FileFinder('/usr/lib/python3.11/lib-dynload'), '/usr/lib/python3.11/site-packages': FileFinder('/usr/lib/python3.11/site-packages'), '/var/tmp/pycparser/pycparser/_build_tables.py': None, '.': FileFinder('/var/tmp/pycparser/pycparser'), '..': FileFinder('/var/tmp/pycparser/pycparser/..'), '/var/tmp/pycparser/pycparser': FileFinder('/var/tmp/pycparser/pycparser'), '/usr/lib/python3.11/re': FileFinder('/usr/lib/python3.11/re'), '/usr/lib/python3.11/collections': FileFinder('/usr/lib/python3.11/collections'), '/var/tmp/pycparser/pycparser/../pycparser': FileFinder('/var/tmp/pycparser/pycparser/../pycparser'), '/var/tmp/pycparser/pycparser/../pycparser/ply': FileFinder('/var/tmp/pycparser/pycparser/../pycparser/ply')}
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 38, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
  File: .
  Size: 1024      	Blocks: 2          IO Block: 1024   directory
Device: 259,4	Inode: 51741431    Links: 3
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-02-24 12:13:18.000000000 -0800
Modify: 2023-02-25 12:34:56.000000000 -0800
Change: 2023-02-25 12:34:56.000000000 -0800
 Birth: -
dilbert /var/tmp/pycparser/pycparser #
Comment 71 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 20:44:59 UTC
Ok, so here's my guess: Python is caching the file list in the directory and using mtime of the directory to verify whether the cache is up-to-date.  You are using only second-precision timestamps, so what basically happens is:

1. You remove the file, so mtime of the directory is set to the current second.

2. The Python script writes lextab.py within the same second, so mtime stays the same.

3. Python determines mtime didn't change, so doesn't update the cache.

I can't reproduce it because I have sub-second timestamps.  Ideally, you could confirm it by doing something like:

  rm lextab.py ; touch -r / . ; python -B _build_tables.py

Since it's resetting mtime of the current directory prior to starting the script, Python should start invalidating the cache.
Comment 72 Gary E. Miller 2023-02-25 20:54:00 UTC
(In reply to Michał Górny from comment #71)
> Ok, so here's my guess: Python is caching the file list in the directory and
> using mtime of the directory to verify whether the cache is up-to-date.  You
> are using only second-precision timestamps, so what basically happens is:
> 
> 1. You remove the file, so mtime of the directory is set to the current
> second.
> 
> 2. The Python script writes lextab.py within the same second, so mtime stays
> the same.
> 
> 3. Python determines mtime didn't change, so doesn't update the cache.

I have seen that with other build systems.  scons can use the timestamp, and
a hash, to see when files changes.

> > I can't reproduce it because I have sub-second timestamps.

Odd, how do I see the timestamp size?  How do I get sub-seconds on ext4?

> Ideally, you
> could confirm it by doing something like:
> 
>   rm lextab.py ; touch -r / . ; python -B _build_tables.py
> 
> Since it's resetting mtime of the current directory prior to starting the
> script, Python should start invalidating the cache.

Yeah, that always works.  Nice touch.
Comment 73 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 21:00:18 UTC
Created attachment 854660 [details, diff]
inval.patch

Ok, as a final request: could you `git checkout .` to undo our changes, reproduce the failure again, then apply the attached patch and confirm that it works with it applied?  If it does, I'm going to send it upstream and have this resolved once and for all.
Comment 74 Gary E. Miller 2023-02-25 21:10:33 UTC
(In reply to Michał Górny from comment #73)
> Created attachment 854660 [details, diff] [details, diff]
> inval.patch
> 
> Ok, as a final request: 

How many times have I heard that??  :-)

> could you `git checkout .` to undo our changes,
> reproduce the failure again, then apply the attached patch and confirm that
> it works with it applied? 

Original, still fails:

dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
Traceback (most recent call last):
  File "/var/tmp/pycparser/pycparser/_build_tables.py", line 35, in <module>
    import lextab
ModuleNotFoundError: No module named 'lextab'

Patch:

dilbert /var/tmp/pycparser/pycparser # wget 'https://bugs.gentoo.org/attachment.cgi?id=854660'
--2023-02-25 13:07:12--  https://bugs.gentoo.org/attachment.cgi?id=854660
Resolving bugs.gentoo.org... 2001:470:e849:1::4, 204.187.15.4
Connecting to bugs.gentoo.org|2001:470:e849:1::4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://701878.bugs.gentoo.org/attachment.cgi?id=854660 [following]
--2023-02-25 13:07:12--  https://701878.bugs.gentoo.org/attachment.cgi?id=854660
Resolving 701878.bugs.gentoo.org... 2001:470:e849:1::4, 204.187.15.4
Connecting to 701878.bugs.gentoo.org|2001:470:e849:1::4|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 574 [text/plain]
Saving to: ‘attachment.cgi?id=854660’

attachment.cgi?id=8 100%[===================>]     574  --.-KB/s    in 0s      

2023-02-25 13:07:12 (540 MB/s) - ‘attachment.cgi?id=854660’ saved [574/574]

dilbert /var/tmp/pycparser/pycparser # mv 'attachment.cgi?id=854660' inval.patch
dilbert /var/tmp/pycparser/pycparser # patch < inval.patch 
patching file _build_tables.py

Always works!

dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser #   rm lextab.py ;  python -B _build_tables.py 
dilbert /var/tmp/pycparser/pycparser # 

>  If it does, I'm going to send it upstream and
> have this resolved once and for all.

How many times have I heard that??  :-)
Comment 75 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-25 21:16:50 UTC
re improving accuracy: I can't find much yet. I found e.g. https://stackoverflow.com/questions/14392975/timestamp-accuracy-on-ext4-sub-millsecond?rq=1 but nobody seems to talk about how to actually fix it or what causes it. Is it a really old ext4 partition or perhaps with tweaked parameters at creation time?
Comment 76 Gary E. Miller 2023-02-26 01:22:50 UTC
(In reply to Sam James from comment #75)
> re improving accuracy: I can't find much yet. I found e.g.
> https://stackoverflow.com/questions/14392975/timestamp-accuracy-on-ext4-sub-
> millsecond?rq=1 but nobody seems to talk about how to actually fix it or
> what causes it. Is it a really old ext4 partition or perhaps with tweaked
> parameters at creation time?

I found there is the old ext2 and ext3 compatable ext4 inode, and a newer one, that is 2x the size, that implements the sub-seconds.

The volume was created 2018-11-14.  Must be before the change?  But I have much older that seem unaffected.

Here is the caps for subseconds ext4:

capabilities: multi journaled extended_attributes large_files huge_files dir_nlink recover 64bit extents ext4 ext2 initialized

And non-subseconds:

capabilities: journaled extended_attributes large_files huge_files dir_nlink recover 64bit extents ext4 ext2 initialized

The same, except for multi.  But that is off topic, I'll work on that.

Let's get this 3+ year old bug fixed!  Hot on the trail!
Comment 77 Gary E. Miller 2023-02-26 01:42:23 UTC
For ext4 to support nanoseconds, it needs a larger inode.  The is controlled by the extra_isize feature, present since Linux 2.6.28.

Oddly, dumpe2fs shows extra_isize on the seconds only ext4:

Filesystem features:      has_journal ext_attr dir_index filetype needs_recovery
 meta_bg extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nl
ink extra_isize

On the ext that has working sub-seconds:

Filesystem features:      has_journal ext_attr resize_inode dir_index filetype n
eeds_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink e
xtra_isize metadata_csum

Both have identical /etc/mke2fs.conf

No hookah smoking caterpillars yet.
Comment 78 Larry the Git Cow gentoo-dev 2023-02-26 15:25:15 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=06668ccb9dd2920b55fc01fb5bc6a24129e69421

commit 06668ccb9dd2920b55fc01fb5bc6a24129e69421
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2023-02-26 15:06:16 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2023-02-26 15:25:10 +0000

    dev-python/pycparser: Backport lextab.py build failure fix
    
    Closes: https://bugs.gentoo.org/701878
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 .../files/pycparser-2.21-lextab-cache.patch        | 66 ++++++++++++++++++++++
 dev-python/pycparser/pycparser-2.21-r1.ebuild      |  5 ++
 dev-python/pycparser/pycparser-2.21-r2.ebuild      |  5 ++
 3 files changed, 76 insertions(+)
Comment 79 Gary E. Miller 2023-02-26 21:09:29 UTC
It does indeed seem to be fixed.  Thanks!

I wish that whenever an ebuild changed, that the rev bumped.  Seems to me this should be dev-python/pycparser-2.21-r3::gentoo, so I don't have to read the commits or bugs.gentoo to know things changed.
Comment 80 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-26 21:15:33 UTC
(In reply to Gary E. Miller from comment #79)
> It does indeed seem to be fixed.  Thanks!
> 
> I wish that whenever an ebuild changed, that the rev bumped.  Seems to me
> this should be dev-python/pycparser-2.21-r3::gentoo, so I don't have to read
> the commits or bugs.gentoo to know things changed.

The counterpart to that is "why do I have to rebuild things when nothing changed?" We usually only revbump if the installed files change or dependencies.