Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 889340 - sci-visualization/paraview: file collision with sci-libs/vtk
Summary: sci-visualization/paraview: file collision with sci-libs/vtk
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal
Assignee: Matthias Maier
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-01-02 06:23 UTC by Jordi PM
Modified: 2024-11-24 21:57 UTC (History)
11 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jordi PM 2023-01-02 06:23:44 UTC
I've tried updating my @world and I get:

>>> Installing (1 of 1) sci-visualization/paraview-5.11.0_rc2-r1::gentoo

 * checking 8730 files for package collisions
 * This package will overwrite one or more files that may belong to other
 * packages (see list below). You can use a command such as `portageq
 * owners / <filename>` to identify the installed package that owns a
 * file. If portageq reports that only one package owns a file then do
 * NOT file a bug report. A bug report is only useful if it identifies at
 * least two or more packages that are known to install the same file(s).
 * If a collision occurs and you can not explain where the file came from
 * then you should simply ignore the collision since there is not enough
 * information to determine if a real problem exists. Please do NOT file
 * a bug report at https://bugs.gentoo.org/ unless you report exactly
 * which two packages install the same file(s). See
 * https://wiki.gentoo.org/wiki/Knowledge_Base:Blockers for tips on how
 * to solve the problem. And once again, please do NOT file a bug report
 * unless you have completely understood the above message.
 *
 * Detected file collision(s):
 *
 *      /usr/lib64/qt5/qml/VTK.9.2/plugins.qmltypes
 *      /usr/lib64/qt5/qml/VTK.9.2/qmldir
 *      /usr/lib64/qt5/qml/VTK.9.2/libqmlvtkplugin.so
 *
 * Searching all installed packages for file collisions...
 *
 * Press Ctrl-C to Stop
 *
 * sci-libs/vtk-9.2.2-r1:0::gentoo
 *      /usr/lib64/qt5/qml/VTK.9.2/libqmlvtkplugin.so
 *      /usr/lib64/qt5/qml/VTK.9.2/plugins.qmltypes
 *      /usr/lib64/qt5/qml/VTK.9.2/qmldir
 *
 * Package 'sci-visualization/paraview-5.11.0_rc2-r1' NOT merged due to
 * file collisions. If necessary, refer to your elog messages for the
 * whole content of the above message.

I'm trying to use vtk-9.2.2-r1 and paraview-5.11.0_rc2-r1



Reproducible: Always

Steps to Reproduce:
1. Install vtk and paraview on the same system
Actual Results:  
1 Package NOT merged due to file collisions

Expected Results:  
Both packages merged

emerge -pv --nodeps vtk paraview

These are the packages that would be merged, in order:

[ebuild   R   ~] sci-libs/vtk-9.2.2-r1:0/9.2::gentoo  USE="boost ffmpeg imaging java logging mpi mysql openmp python qt5 rendering sdl threads -all-modules -cuda -debug -doc -examples -freetype -gdal -las -odbc -openvdb -pdal -postgres (-qt6) -tbb -test -tk -views -vtkm -web" PYTHON_SINGLE_TARGET="python3_10 -python3_8 -python3_9" VIDEO_CARDS="-nvidia" 0 KiB
[ebuild  N    ~] sci-visualization/paraview-5.11.0_rc2-r1::gentoo  USE="ffmpeg mpi openmp python qt5 sqlite webengine -boost -cg -doc -examples -nvcontrol -offscreen -plugins -test -tk" PYTHON_SINGLE_TARGET="python3_10 -python3_8 -python3_9" 0 KiB
Comment 1 Bernd 2023-01-03 18:32:22 UTC
Just checked their GitHub mirror, it's still true, that they don't fully support using external VTK, like I mentioned in bug #253881#c11.

There's an overseeable number of rdeps for VTK. Would it be an option to add a blocker?
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-03 23:10:53 UTC
(In reply to Bernd from comment #1)
> Just checked their GitHub mirror, it's still true, that they don't fully
> support using external VTK, like I mentioned in bug #253881#c11.
> 
> There's an overseeable number of rdeps for VTK. Would it be an option to add
> a blocker?

It's an option but I'm not sure it's a good idea. It would mean nobody can install both paraview and VTK.

I feel like the people who want to use both of them is a non-empty set.
Comment 3 François Bissey 2023-01-03 23:27:49 UTC
It's been more than a decade since I was maintainer of paraview in the science overlay and upstream was promising distro that you could use an external vtk - soon.

Seriously, it would just takes proper will on their part to just stop making vtk a git submodule of the paraview git tree and have to eat their own dog food properly. If they want to continue to bundle vtk, they have to install the internal copy in a fully internal location.
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-03 23:29:52 UTC
(In reply to François Bissey from comment #3)
> It's been more than a decade since I was maintainer of paraview in the
> science overlay and upstream was promising distro that you could use an
> external vtk - soon.
> 
> Seriously, it would just takes proper will on their part to just stop making
> vtk a git submodule of the paraview git tree and have to eat their own dog
> food properly. If they want to continue to bundle vtk, they have to install
> the internal copy in a fully internal location.

Absolutely. It's either bundled or it's not. Not this frustrating hybrid :(
Comment 5 Bernd 2023-01-04 09:33:01 UTC
(In reply to François Bissey from comment #3)
> It's been more than a decade since I was maintainer of paraview in the
> science overlay and upstream was promising distro that you could use an
> external vtk - soon.
> 
> Seriously, it would just takes proper will on their part to just stop making
> vtk a git submodule of the paraview git tree and have to eat their own dog
> food properly. If they want to continue to bundle vtk, they have to install
> the internal copy in a fully internal location.

I agree with you. Don't know the ParaView build system well enough, maybe there's even a way to install the VTK components internally.

We also might check, if the VTK installed by ParaView is usable by VTK consumers and we could use an any-of dependency operator on them. IIRC ParaView uses a specific configuration of VTK, but still allows to pass some options down to VTK to customize the VTK config. If they also install the cmake config files, this might or might not be an option, depending on the users needs for VTK and the possibility to pass the needed options down to VTK.

For ::gentoo consumers of VTK this seems pretty well doable. They all use rather basic build options (rendering, views, imaging, qt5, python) which are all high-level options to VTK and configure general settings.

The situation is different however, if VTK is used for development of a software, which is not in ::gentoo and requires a more unusual configuration.
Comment 6 François Bissey 2023-01-04 09:54:35 UTC
If those are the only colliding files, then it seems it happens mainly because they want to install properly some qt5 components. Any chance that they could be installed under `.../qt5/qml/Paraview.5.11.0/`, even if manually, instead?
Comment 7 Bernd 2023-01-05 17:26:16 UTC
It looks like these are the only files. We already pass CMAKE_INSTALL_QMLDIR to both packages. Dropping this for paraview, those files get installed into `/VTK-9.2`. IIRC for VTK they will get installed into `/usr/$(get_libdir)/VTK-9.2` if this parameter isn't passed. So we could set this to the value you have specified, but I'm not sure if this is Qt compliant. It will install the files under `/usr/$(get_libdir)/qt5/qml/paraview-5.11/VTK-9.2`.

I found some other issues regarding installation. The cmake config files are installed under `/usr/$(get_libdir)/${P}/cmake/${P}` instead of `...LIBDIR/cmake/${P}`. The python files are as well installed under `...LIBDIR/${P}/python3.10/site-packages`, not in `/usr/lib/python3.10/site-packages/`. If installed in the usual location they would cause another collision, due to some toplevel python files installed directly in site-packages, so all those files would need to go into a subdir of site-packages.

It looks however, the libraries wouldn't need to get into `...LIBDIR/paraview-511`, because all VTK libraries have a `-pv5.11` suffix.
Comment 8 François Bissey 2023-01-06 01:35:59 UTC
Considering cmake and paraview are from the same company, the fact that it doesn't adhere to standard is just bizarre. The python issue is well know and that has been the case as far back as paraview 3.10. It is on purpose to avoid collision with vtk as far as I know.

The default shipping of paraview is as a superbuild installed in its own prefix and it shows. Someone seriously needs to send upstream https://youtu.be/NSemlYagjIU and make sure they watch until the end.
Comment 9 Bob Johnson 2023-01-09 19:52:43 UTC
BTW, it appears they recently moved the release candidates into an 'RCs' subdirectory, so the current ebuild won't even fetch the tarball at the moment.
Comment 10 Bernd 2023-01-09 21:18:10 UTC
Maybe they moved it, because 5.11 is already released?
Comment 11 alex Mezey 2023-01-23 11:17:32 UTC
(In reply to Bernd from comment #10)
> Maybe they moved it, because 5.11 is already released?

 I made an ebuild for the new release see https://bugs.gentoo.org/show_bug.cgi?id=890493 - the vtk problem persists.

Is there a workaround? Which ebuild's files should I have in that place?
As FreeCAD needs both packages for visualization a chroot is not an option.
Comment 12 Bernd 2023-01-23 18:43:04 UTC
(In reply to François Bissey from comment #8)
> Considering cmake and paraview are from the same company, the fact that it
> doesn't adhere to standard is just bizarre. The python issue is well know
> and that has been the case as far back as paraview 3.10. It is on purpose to
> avoid collision with vtk as far as I know.
> 
Well if it works for ParaView, we IMO don't need to change it.
The same is possibly true for the cmake config files. As we don't yet have revdeps of paraview, we don't need to take much care of packages that need to find the provided cmake config files if they are not installed into the usual locations.

(In reply to alex Mezey from comment #11)
>  I made an ebuild for the new release see
> https://bugs.gentoo.org/show_bug.cgi?id=890493 - the vtk problem persists.
> 
Yes, if you follow the thread, there has been a very long tradition of collisions for these two packages.

> Is there a workaround? Which ebuild's files should I have in that place?
> As FreeCAD needs both packages for visualization a chroot is not an option.
There might be some workarounds by passing certain CMAKE_* options in the ebuild. Comment #7 lists the most important one. But it needs some testing.
Comment 13 Matthias Maier gentoo-dev 2023-01-24 03:45:44 UTC
*blerg*

The only solution I see is to install paraview similarly to how we do it for firefox or other "non compliant" software: Simply using /usr/$(get_libdir)/paraview/, adding a symlink to /usr/bin and an ld.co.conf entry for the dynamic loader.

We tried multiple times to debundle VTK from Paraview but this is simply not feasible downstream. And I doubt that this will ever be supported upstream. At least it hasn't happened in about 10 years.
Comment 14 Matthias Maier gentoo-dev 2023-01-24 03:48:26 UTC
Actually that's also creating a potential mess with the dynamic loader. Fun.
Comment 15 Etienne Lorriaux 2023-06-16 07:59:37 UTC
Hello,

the situation does not seem to move regarding that 5.11 version, there are still files collisions when trying to emerge it.

But I'm surprised that all previous versions have been removed from portage tree, shouldn't we maintain a 5.10 ebuild in portage tree to keep a working version of paraview in official tree?
Comment 16 Attila Tóth 2024-03-27 08:01:52 UTC
Still happening.
Comment 17 silver_ghost 2024-04-29 21:21:11 UTC
(In reply to Sam James from comment #2)
> (In reply to Bernd from comment #1)
> > Just checked their GitHub mirror, it's still true, that they don't fully
> > support using external VTK, like I mentioned in bug #253881#c11.
> > 
> > There's an overseeable number of rdeps for VTK. Would it be an option to add
> > a blocker?
> 
> It's an option but I'm not sure it's a good idea. It would mean nobody can
> install both paraview and VTK.
> 
> I feel like the people who want to use both of them is a non-empty set.
Nobody can install them both now. So those who want to use them both just spend much more time by unsuccessful build to find that. Seems like a good reason to add a blocker.
Comment 18 Fernando B. Nascimento 2024-11-24 21:57:00 UTC
I also have this collision with qt6, but managed to install them simultaneously by using a different version of vtk from the one used on paraview.

May be other problems arise for some users, I masked the VTK version 9.3* and got

[ebuild     UD ] dev-libs/pegtl-2.8.3-r1::gentoo [3.2.7::gentoo] USE="-test%" 0 KiB

I didn't really use any feature that use the colliding libraries, it would be good if someone that knows how to actually test it install them this way and test it.

If everything goes well, this can be a temporary workaround, to mask the specific versions on the ebuilds, by the comments may be as short as 10 years workaround (or 20+ who knows?) if it doesn't break at some time.

### The collision:

 * Messages for package sci-libs/vtk-9.3.0-r3:

 * This package will overwrite one or more files that may belong to other
 * packages (see list below).
 *
 * Detected file collision(s):
 *
 *      /usr/lib64/qt6/qml/VTK.9.3/libqmlvtkplugin.so
 *      /usr/lib64/qt6/qml/VTK.9.3/qmldir
 *      /usr/lib64/qt6/qml/VTK.9.3/plugins.qmltypes
 *
 * Searching all installed packages for file collisions...
 *
 * Press Ctrl-C to Stop
 *
 * sci-visualization/paraview-5.13.0:0::gentoo
 *      /usr/lib64/qt6/qml/VTK.9.3/libqmlvtkplugin.so
 *      /usr/lib64/qt6/qml/VTK.9.3/plugins.qmltypes
 *      /usr/lib64/qt6/qml/VTK.9.3/qmldir
 *
 * Package 'sci-libs/vtk-9.3.0-r3' NOT merged due to file collisions. If
 * necessary, refer to your elog messages for the whole content of the
 * above message.

### Create file

/etc/portage/package.mask/workaround-vtk-paraview:
# Workaround on the collision between vtk-9.3 and paraview 5.13, edit this when these versions change.
=sci-libs/vtk-9.3*

### Result:

~> emerge -pv --nodeps vtk paraview

Local copy of remote index is up-to-date and will be used.

These are the packages that would be merged, in order:

[ebuild   R    ] sci-libs/vtk-9.2.5-r2:0/9.2::gentoo  USE="ffmpeg logging openmp qt6 rendering sdl threads -all-modules -boost (-cuda) (-debug) -doc -examples -freetype -gdal -imaging -java -las -mpi -mysql -odbc -openvdb -pdal -postgres -python -qt5 -tbb -test -tk -views -vtkm -web" PYTHON_SINGLE_TARGET="-python3_10 -python3_11" VIDEO_CARDS="nvidia" 0 KiB
[ebuild   R   ~] sci-visualization/paraview-5.13.0::gentoo  USE="ffmpeg openmp qt6 sqlite webengine -boost -cg -doc -examples -mpi -nvcontrol -offscreen -plugins -python -test -tk" PYTHON_SINGLE_TARGET="python3_12 -python3_10 -python3_11" 0 KiB

Total: 2 packages (2 reinstalls), Size of downloads: 0 KiB