Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 933646

Summary: dev-embedded/esptool-4.7.0-r2 doesn't work
Product: Gentoo Linux Reporter: Luke-Jr <luke-jr+gentoobugs>
Component: Current packagesAssignee: Martin Dummer <martin.dummer>
Status: RESOLVED FIXED    
Severity: major CC: flow, hyedad, kangie, martin.dummer, mgorny, proxy-maint, python
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description Luke-Jr 2024-06-05 18:44:04 UTC
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.12/esptool.py", line 34, in <module>
    import esptool
...repeating
  File "/usr/lib/python-exec/python3.12/esptool.py", line 34, in <module>
    import esptool
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1262, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1528, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1502, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1635, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1590, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 828, in spec_from_file_location
  File "<frozen importlib._bootstrap_external>", line 187, in _path_abspath
RecursionError: maximum recursion depth exceeded

python3.11 has the same issue.

dev-embedded/esptool-4.6.2 seems to work fine
Comment 1 Karl Hakimian 2024-06-17 01:18:44 UTC
The previous version also works for me. The following also works

python3.12 /usr/bin/esptool.py

So it seems it is the python-exec stuff that is screwed up. I don't know enough about what that is trying to do to debug further.
Comment 2 Martin Dummer 2024-06-17 20:15:03 UTC
I can confirm both reports - executing

esptool.py

ends up with the RecursionError, and


python3.12 /usr/bin/esptool.py

is working.

As I am no real python expert, I seek for help and call back....
Comment 3 Matt Jolly gentoo-dev 2024-07-01 22:36:39 UTC
This package obviously needs to be a single-impl python thing.

My workaround:

```
python3.11 $(which esptool.py) --baud 115200 --port /dev/ttyUSB0 write_flash 0x0 firmware-heltec-v2_1-2.3.13.83f5ba0.bin 
```
Comment 4 Martin Dummer 2024-07-02 12:34:43 UTC
(In reply to Matt Jolly from comment #3)
> This package obviously needs to be a single-impl python thing.
> 


I confirm that re-adding DISTUTILS_SINGLE_IMPL=1 gives the functionality back immediately.

But @flow has removed exactly this in commit f1fac2d9fe with the comment:

    There seems to be no reason why esptool is python single-impl.
    
    Being python single-impl causes issues for consumer of the
    dev-embedded/esptool dependency. For example, dev-embedded/esp-idf (from
    ::guru), currently declares the dev-embedded/esptool dependency without
    [${PYTHON_USEDEP}] as consequence of this.

So there is the question if there is a solution for both options - but I must say that's beyond my personal knowledge.

Python experts, please have an eye on this here!
Comment 5 Florian Schmaus gentoo-dev 2024-07-03 07:12:16 UTC
(In reply to Martin Dummer from comment #4)
> So there is the question if there is a solution for both options - but I
> must say that's beyond my personal knowledge.

I do not see an immediate solution either. The launchers for esptool binaries could basically employ the workaround mentioned here. Maybe there is even existing Gentoo Python tooling for such cases. But I am not aware of something like that (which does not mean mean that it does not exists).

The root cause seems to be a sys.module modification that esptool's "executables" do. If I understand the code comment correctly, it's to allow importing the esptool module.

This seems a little bit odd, but maybe it is idiomatic to Python? Or is this something that could be fixed upstream (which is always the preferable solution)?


> Python experts, please have an eye on this here!

Yes, this is smells like something for Gentoo's Python experts. I would also suggest to approach them in #gentoo-python. Not sure if a request for help on this bug can summon them.
Comment 6 Martin Dummer 2024-07-03 21:34:09 UTC
  
>     Being python single-impl causes issues for consumer of the
>     dev-embedded/esptool dependency. For example, dev-embedded/esp-idf (from
>     ::guru), currently declares the dev-embedded/esptool dependency without
>     [${PYTHON_USEDEP}] as consequence of this.
> 

To be  honest, I tend to rate a common usage of esptool (that means with DISTUTILS_SINGLE_IMPL=1 in the ebuild) higher than one other package that wants to import esptool as a module, but does not properly declare [${PYTHON_USEDEP}]

@flow please 
- either find a solution that is suitable for both usecases,  
- or revert your commit f1fac2d9fe

The workaround to start esptool like

python3.11 $(which esptool.py) --baud 115200 --port /dev/ttyUSB0 write_flash 0x0 firmware-heltec-v2_1-2.3.13.83f5ba0.bin 

is IMHO not acceptable.
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-07-03 22:37:36 UTC
I don't think we should leave it broken. It's not clear to me why esp-idf even needs to be multi-impl, especially in light of this.
Comment 8 Larry the Git Cow gentoo-dev 2024-07-03 22:38:31 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b3386e04d713abff3bfe4071b22a67ad0b9077ae

commit b3386e04d713abff3bfe4071b22a67ad0b9077ae
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-07-03 22:37:33 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-07-03 22:37:59 +0000

    Revert "dev-embedded/esptool: make python multi-impl"
    
    This reverts commit f1fac2d9fe291f036e60efab036f5c02b1791c25.
    
    Bug: https://bugs.gentoo.org/933646
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-embedded/esptool/esptool-4.7.0-r2.ebuild | 1 +
 1 file changed, 1 insertion(+)
Comment 9 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-07-03 22:41:47 UTC
Ugh, sorry, my revbump got lost. Fixing that.
Comment 10 Larry the Git Cow gentoo-dev 2024-07-03 22:42:52 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6fdec87512d25412575852c8f00d41f23e564120

commit 6fdec87512d25412575852c8f00d41f23e564120
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-07-03 22:42:14 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-07-03 22:42:14 +0000

    dev-embedded/esptool: add revbump for single impl
    
    Sorry, I lost this somehow.
    
    Closes: https://bugs.gentoo.org/933646
    Fixes: b3386e04d713abff3bfe4071b22a67ad0b9077ae
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-embedded/esptool/{esptool-4.7.0-r2.ebuild => esptool-4.7.0-r3.ebuild} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
Comment 11 Florian Schmaus gentoo-dev 2024-07-04 06:58:38 UTC
(In reply to Martin Dummer from comment #6)
> @flow please 
> - or revert your commit f1fac2d9fe

I told sam yesterday that I would do that as soon as I had access to my OpenPGP keys again.

So no worries, we where on the same page.

I just hoped that someone would point out a solution that would get us the best of both worlds. Instead it looks like most of the espressif toolchain should be Python singl-impl. Which is unfortunate, since a large part of it is in ::guru.  This means that it will likely break as soon as Gentoo switches PYTHON_SINGLE_TARGET, as the ::guru packages probably leak behind (at least, that was the case recently and one motivation for this change).

> The workaround to start esptool like

> python3.11 $(which esptool.py) --baud 115200 --port /dev/ttyUSB0 write_flash 0x0 firmware-heltec-v2_1-2.3.13.83f5ba0.bin 

FWIW, the idea was to deploy the workaround as wrapper script. Not that users had to start esptool this way.