During update of media-libs/gst-plugins-base, I hit the following problem: undefined reference to `gst_audio_channel_get_fallback_mask' It turned out, for some reason, the package added -L/usr/lib64 to the gcc command line, ending up to link against /usr/lib64/libgstaudio-1.0.so instead of the newly built $WORKDIR/gst-plugins-base-1.14.4-abi_x86_64.amd64/gst-libs/gst/audio/.libs/libgstaudio-1.0.so /usr/lib64/libgstaudio-1.0.so was old enough to not define gst_audio_channel_get_fallback_mask. Removing that file, the emerge completed successfully. In order to detect (and ideally work around) those problems, sandbox should prevent access to already installed files of the package currently being installed. So, on any attempt to open/stat such files, sandbox should return ENOENT, and on any readdir (if neccessary), sandbox should hide that files. It is possibly not a wise idea, to error out on problems like that, because on a first install by a gentoo developer, the problem can't show up at all, and I don't assume, gentoo developer will always reinstall a new ebuild right after a successfull install just to test this kind of problems (even if this would be made mandatory, it would be error prone).
How will you build sys-devel/gcc without using files from already installed sys-devel/gcc? :)
Working around is far from 'ideal' here. We should fix broken software, not shove the problems under the carpet and look the other way.
@Arfrever Frehtes Taifersar Arahesis Obviously, there are some rare exceptions. You just found one. Keep up the good work, and complete the list of exceptions, so they can be white listed. @Michał Górny > Working around is far from 'ideal' here. I like to disagree. If I as a user want to merge something, I do not want to wait for someone to fix such issues. This feature can be configurable. Attempt to read will be blocked or causes an access violation error. Developers can then configure their system to error out here. > We should fix broken software, not shove the problems under the carpet and look the other way. True. But obviously, it's a task, that fails regularly. Nobody asks you to not fix problems. I only ask you to help users to get around problems.
What is the gst-plugins-base specific bug, so I can look into it? If one doesn't exist, please create one to solve it, independent of the feature request here in this bug entry. Regarding this request, how would it actually work? How would portage/sandbox know what files to hide or error on access or such? Deny access to the installed version files, if it's not a fresh install? Deny access to everything that isn't files from recursive RDEPEND and (B)DEPENDs CONTENTS?
(In reply to Mart Raudsepp from comment #4) > What is the gst-plugins-base specific bug, so I can look into it? I'm converting to Funtoo, so the ebuild is a Funtoo ebuild media-libs/gst-plugins-base-1.14.4-r1::media-kit. I can file a report here anyways, probably the same problem is in Gentoo as well, but I can't be perfectly sure about that. So, if you want me to file it anyways, I'll go for it. > Regarding this request, how would it actually work? How would > portage/sandbox know what files to hide or error on access or such? Deny > access to the installed version files, if it's not a fresh install? Deny > access to everything that isn't files from recursive RDEPEND and (B)DEPENDs > CONTENTS? Actually, I would recommend to simulate a fresh install for each ebuild with smallest dep tree possible. That would mean much more runtime overhead (so it should be configurable). Just denying access to everything listed in CONTENTS would definitively be a first easy to implement and low overhead solution. However the fresh install solution would also solve all cases of missing dependencies. (It wouldn't solve "too old dependency" problems thou.)
This is not a new idea (which should be pretty obvious at this point). However, you're entirely missing a lot of complex intricacies that make this kind of thing almost entirely impossible. Starting with a lot of modularity with system's configuration (imagine this filtering killing some file needed by user's nss configuration), and ending at the flexibility Gentoo users want to exercise.
(In reply to Michał Górny from comment #6) > However, you're entirely missing a lot of complex intricacies that make this > kind of thing almost entirely impossible. Oh, then ask Google, they solved other impossible problems already ;) > Starting with a lot of modularity with system's configuration (imagine > this filtering killing some file needed by user's nss configuration), Easy: Besides system and world, add a new set like important, where the user adds those packets. > and ending at the flexibility Gentoo users want to exercise. Like installing Gnome 3 without systemd? Which is the fight, I was doing the past years until I finally decided to turn over to Funtoo? Please do not cherry pick some "flexibility" arguments only to not solve problems. All I hear sound like excuses. Yes, there might be problems like how to merge gcc, how to deal with special setups, and there are probably more corner cases, but none of them can't be solved by white listing packaged and files. Also, nobody would be required to use this feature, collision-protect is configurable as well, isn't it? So, if this feature really doesn't work for 20 Users, they can just disable it and stay where they are today already. So what's the real point you're talking about here? It's not that this feature causes some problems for some guys. It's something more deep.
The first problem is that you can't predict all problems. That's why I didn't work on it so far. I can predict it's going to be a lot, and I don't think it's worth it. The second problem is that the list will grow. I don't think you can even reliably move the list of workarounds to configuration, unless you want to maintain a huge package list there and usually broaden the scope beyond necessity, effectively reducing the usefulness of the feature. The third problem is that it hides some of the problems instead of reporting them. Yes, it wouldn't be obligatory but still a lot of people would fall into the trap and hide in a safe place rather than reporting problems that affect other Gentoo users. And some devs will also fall into one trap or another and go on wondering why they can't reproduce some failures. One example of problems you missed are plugin-based packages that do not specify their plugins by dependencies but require them once they're installed (e.g. via configuration). And of course, the whole lot of packages depending on files that aren't owned by any package.
One of problems which you have not predicted is that for bigger packages you would exhaust allowed limit of environment: $ python -c 'import subprocess; subprocess.call(["true"], env={"A": "A" * 2 ** 16})' $ python -c 'import subprocess; subprocess.call(["true"], env={"A": "A" * 2 ** 17})' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib64/python3.6/subprocess.py", line 267, in call with Popen(*popenargs, **kwargs) as p: File "/usr/lib64/python3.6/subprocess.py", line 709, in __init__ restore_signals, start_new_session) File "/usr/lib64/python3.6/subprocess.py", line 1344, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) OSError: [Errno 7] Argument list too long: 'true' $ python -c 'import subprocess; f=open("/var/db/pkg/dev-db/postgresql-11.2/CONTENTS"); contents=f.readlines(); f.close(); contents=[" ".join(x.split(" ")[1:-2]) for x in contents if x.startswith("obj ")]; sandbox_deny=":".join(contents); subprocess.call(["true"], env={"SANDBOX_DENY": sandbox_deny})' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib64/python3.6/subprocess.py", line 267, in call with Popen(*popenargs, **kwargs) as p: File "/usr/lib64/python3.6/subprocess.py", line 709, in __init__ restore_signals, start_new_session) File "/usr/lib64/python3.6/subprocess.py", line 1344, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) OSError: [Errno 7] Argument list too long: 'true' Currently information is passed from package manager to sandbox in environment variables: SANDBOX_DENY SANDBOX_READ SANDBOX_WRITE SANDBOX_PREDICT So you would need to design and implement new way of passing information...
(In reply to Michał Górny from comment #8) > 1. you can't predict all problems. No, but once this feature is implemented, first stage could be to make it a warning that doesn't cause builds to fail. 99.9% of all problems will be detected in this stage. > 2. the list will grow. [...] maintain a huge package list [...] $ ls -ld /var/db/pkg/*/* | wc -l 1400 $ ls -ld /var/cache/funtoo/kits/*/*/* | wc -l 21981 Let's say, 10% of all packages will have a problem that needs exception from this feature. This will be 2198 packages. Maybe, both of us have different understandings of what the word "huge" means, but I don't call a list with 2198 packages "huge". All packages, which will be identified by Gentoo Devs as being problematic will get their own flag in the ebuild, so with 99.9% of all problems being detected in stage 1 of this feature implementation, there will be at most a hand full of packages left, that I as an end user will have to add to such a list. > 3. hide problems instead of reporting; And some devs will also fall into > one trap or another If a package tried to install against a previously installed instance and you just don't provide that package with a previously installed instance, then there is no problem anymore, that could potentially be hidden away. I don't know how committing ebuild work, but maybe you can even enforce a check, that the dev successfully tried to install and reinstall the package with this feature turned off. This will then end up being a feature, that enhances quality of ebuilds (and not decrease the quality). > plugin-based packages that [...] require them once they're [...] configured If those packages start to be needed due to configuration, add a hint in those configuration files (or better yet: Add those packages to exception lists). BTW: Can you give me one example for such plugin-based packages, where plugins are needed (or at least desired) during emerge? > And of course, the whole lot of > packages depending on files that aren't owned by any package. Create one special package, listing all files, that belong to Gentoo but are not owned by any specific package.
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #9) > One of problems which you have not predicted is that for bigger packages you > would exhaust allowed limit of environment: I didn't "predict" it, because it doesn't exist. man grep [...] -f FILE, --file=FILE Obtain patterns from FILE, one per line. [...] [...] man tar [...] -T, --files-from=FILE Get names to extract or create from FILE. Unless specified otherwise, the FILE must contain a list of names [...] one name per line [...] [...]
