| Summary: | dev-python/selenium-4.12.0: "Unable to obtain driver for firefox using Selenium Manager" | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Henrik Johansson <henjo456> |
| Component: | Current packages | Assignee: | Python Gentoo Team <python> |
| Status: | UNCONFIRMED --- | ||
| Severity: | normal | CC: | blc+gentoo, captaincrutches, graaff, henjo456, mgorny |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
Does it help to install dev-util/selenium-manager ? I have the same problem, it also happens when using Chrome - same traceback.
Installing dev-util/selenium-manager does not help.
It looks like it's specifically looking for the manager binary inside the install directory of selenium itself...
From /usr/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py:
> path = Path(__file__).parent.joinpath(directory, file)
which resolves to /usr/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager
which doesn't exist.
It seems selenium-manager is required for selenium now or at least it seems to be implied upon cursory inspection of the code, should it be a dependency or at least default dependency subject to a USE flag? I upgraded from 4.5 to 4.13 and got a huge frag bomb waiting for me, breaking my selenium scripts that I need to run daily. I'm currently waiting for selenium-manager to build, looks like more rusty stuff which will take a while on this box *sigh* Oh finished earlier than I thought. dev-util/selenium-manager a large amount of files and thought it would take a while to build, now after running selenium-manager this seems like a script that's way bigger than it needs to be... Anyway I was able to restore operations after merging it and running my script with a slightly different invocation that worked before without the hack: CONDA_PREFIX=/usr ./my-selenium-script I guess this will need to be a workaround for now, but this probably isn't ideal. Is there anyone who got this working without hacks? I'm afraid that none from python@ is really knowledgeable on selenium. We're merely maintaining it because it's needed by random test suites (dev-python/django, noticeably) and the current package works for that purpose and passes its own tests. We probably ought to add some kind of postinst message about dev-util/selenium-manager, or even a dependency. However, it'd really be helpful if you could find some specific documentation/explanation for us, as I'm overwhelmed with other work. @graaff, perhaps dev-util/selenium-manager could install an env.d file setting SE_MANAGER_PATH? Apparently this is the first envvar that selenium uses to find the executable (and unlike CONDA_PREFIX, it is unlikely to have side effects). Hmm, yeah that's probably a better solution compared to the hack inspired by the debug trace.
It looks like selenium's driver model can be manually overridden but this is an API change ("It *was* working...") since Se 4.6 and selenium-manager was meant to "simplify" things https://www.selenium.dev/blog/2022/introducing-selenium-manager/ . It seems that it was meant to have multiple versions and download/update the driver (geckodriver, chromedriver, etc.).
Thus there should be some now new mandatory use API options within Se 4.6 and newer to specify the location of the driver, so selenium-manager is technically unnecessary, and that's probably what django's doing. I couldn't find documentation about this API change either, where it had been automagically working in the past. This is ridiculous as for me geckodriver is in $PATH (heck it's in /usr/bin) and there is no reason why it couldn't find it - must use some other path now that's no longer based on $PATH. Grr.
Then there's the old documentation out there that's no longer relevant. Grr^2.
The crappy thing is that API changes in Se is not new... I had to rewrite when they went from Se 3.x to Se 4.x, so this is technically minor compared to that.
I'm not an expert on this either, just wanted my scraper scripts working :)
(In reply to Captain Crutches from comment #2) > From > /usr/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager. > py: > > path = Path(__file__).parent.joinpath(directory, file) > which resolves to > /usr/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium- > manager > which doesn't exist. The same thing happens in dev-ruby/selenium-webdriver. In that case I've patched the code to return /usr/bin/selenium-mananger instead of this path and depend on dev-util/selenium-manager. It doesn't seem this code path is fully tested. If I set
SE_MANAGER_PATH=/usr/bin/selenium-manager ./my-selenium-script
I get
File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 60, in get_binary
if not path.is_file() and os.environ["CONDA_PREFIX"]:
^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'is_file'
Now I'm not an expert pythoneer so I'm not sure how additional methods is added to str objects for path/file detection, but apparently this was not done here, and looks like I have to continue using the hack through CONDA_PREFIX for now...
btw would this be the correct code for line 60?
...
if not os.path.isfile(path) and os.environ["CONDA_PREFIX"]:
...
Above issue was marked in https://github.com/SeleniumHQ/selenium/issues/12929 after all, so expect it in next upstream version... The upstream bug states that that issue has been resolved in version 4.15.1 but I still have the same problem in version 4.16.0 in portage. With selenium-4.19.0 I get a similar but distinct failure with `SE_MANAGER_PATH` set:
```
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 142, in run
output = json.loads(stdout)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/driver_finder.py", line 38, in get_path
path = SeleniumManager().driver_location(options) if path is None else path
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 106, in driver_location
output = self.run(args)
^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 145, in run
raise WebDriverException(f"Unsuccessful command executed: {command}") from err
selenium.common.exceptions.WebDriverException: Message: Unsuccessful command executed: /usr/bin/selenium-manager --browser chrome --language-binding python --output json
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/captaincrutches/Playground/./run_e2e_test.py", line 19, in <module>
main()
File "/home/captaincrutches/Playground/./run_e2e_test.py", line 12, in main
browser = webdriver.Chrome()
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/usr/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 49, in __init__
self.service.path = DriverFinder.get_path(self.service, options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/driver_finder.py", line 41, in get_path
raise NoSuchDriverException(msg) from err
selenium.common.exceptions.NoSuchDriverException: Message: Unable to obtain driver for chrome using Selenium Manager.; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location
```
Running that unsuccessful command myself yields:
```
/usr/bin/selenium-manager --browser chrome --language-binding python --output json
error: unexpected argument '--language-binding' found
```
It looks like this argument was added in selenium-manager 4.17.0: https://github.com/SeleniumHQ/selenium/commit/a8468f65a0469be85dffd41b15f8d0b32b445dba
But the latest selenium-manager in gentoo currently is 4.15.0, and the oldest selenium is 4.18.1, so they won't match up here.
I think selenium-manager-4.14 and 4.15 are basically wasting space in portage at this point. As an experiment I hacked up a wrapper for selenium-manager to remove the unknown option and it got a bit further but failed parsing json... so it looks the json output of the old selenium-manager is no longer compatible. Staring at the ebuild for selenium-manager 4.15 and it looks like one needs to know not only python but rust to figure out deps ... is there an easy way to get the rust dep list to get an ebuild to build a newer version? hmm... might need to ask fgo or something for a wider audience, or perhaps this bug is a selenium-manager issue and not specifically selenium. I retract my previous comment and have a workaround for selenium-4.21 and selenium-manager-4.15. I screwed up my wrapper script. Fixing my wrapper script, I was finally able to use selenium. This is my wrapper script: ------------------ #!/bin/bash BUILD=$1 while shift; do arg=$1 if [ "$arg" == "--language-binding" ]; then shift; else BUILD="$BUILD $arg" fi done /usr/bin/selenium-manager $BUILD ------------------ I used SE_MANAGER_PATH=/path/to/above/script my_selenium_script.py Hope this enables some people... I can confirm that removing "--language-binding python" solves this problem running dev-python/selenium-4.22.0-r1 and dev-util/selenium-manager-4.15.0. Right, I forgot to mention that I had to create the following symlink in order for it to work as well: /usr/bin/selenium-manager /usr/lib/python3.12/site-packages/selenium/webdriver/common/linux/selenium-manager Selenium-4.24 and selenium-manager-4.24 appears to have just been stabilized and appears to be working for me now with my legacy scripts and no longer need workarounds... anyone else see the same? (In reply to Ben from comment #16) > Selenium-4.24 and selenium-manager-4.24 appears to have just been stabilized > and appears to be working for me now with my legacy scripts and no longer > need workarounds... anyone else see the same? I'm running dev-python/selenium-4.26.1 and dev-util/selenium-manager-4.26.0 and no longer have this problem and there is no need for any workaround. Headless does not work anymore, however, does this have anything to do with making geckodriver its own package by any chance? |
Updating dev-python/selenium from 4.10.0 to 4.12.0 seems to break the Firefox driver. I'm using dev-lang/python-3.11.5 The following piece of code executes on 4.10.0 but gives the error message below using 4.12.0. ========= #!/usr/bin/python from selenium import webdriver driver = webdriver.Firefox() driver.get("https://www.gentoo.org") ========= Traceback (most recent call last): File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/driver_finder.py", line 38, in get_path path = SeleniumManager().driver_location(options) if path is None else path ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 78, in driver_location args = [str(self.get_binary()), "--browser", browser] ^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/selenium_manager.py", line 57, in get_binary if not path.is_file() and os.environ["CONDA_PREFIX"]: ~~~~~~~~~~^^^^^^^^^^^^^^^^ File "<frozen os>", line 679, in __getitem__ KeyError: 'CONDA_PREFIX' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/hrnick/misc/./test", line 3, in <module> driver = webdriver.Firefox() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/selenium/webdriver/firefox/webdriver.py", line 59, in __init__ self.service.path = DriverFinder.get_path(self.service, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/selenium/webdriver/common/driver_finder.py", line 41, in get_path raise NoSuchDriverException(msg) from err selenium.common.exceptions.NoSuchDriverException: Message: Unable to obtain driver for firefox using Selenium Manager.; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location