This idea tries to solve two problems: a) Some packages can only run tests successfully when being already installed. b) Some packages create circular dependencies because they depend on each other to run tests successfully. Solution for a) with package app-foo/bar: TDEPEND="app-foo/bar" PM installs app-foo/bar first, disabling FEATURES="test" and installs it again, enabling FEATURES="test". Solution for b) with package app-foo/bar which RDEPENDs on app-foo/baz, while app-foo/baz needs app-foo/bar to run tests successfully. app-foo/bar: RDEPEND="app-foo/baz" app-foo/baz: TDEPEND="app-foo/bar" PM installs app-foo/baz, installs app-foo/bar, installs and runs tests on app-foo/baz again. Possible extensions: - noop non-src_test phases to avoid compilation and installation times (if possible) - Solve problems like app-foo/baz DEPEND="bar? ( app-foo/bar )" and app-foo/bar DEPEND="baz? ( app-foo/baz )": Add a variable FIDIUSE="bar" (First Install Disabled IUSE), that disables selected USE flags on the first install (and enforces a second install). This no longer needs TDEPEND then, just FIDIUSE="test" would do the same.
(In reply to Manuel Rüger from comment #0) > This idea tries to solve two problems: > > a) Some packages can only run tests successfully when being already > installed. Which is non-sense since they will run the tests against the previous version when upgrading... > b) Some packages create circular dependencies because they depend on each > other to run tests successfully. This doesn't justify adding another variable. > Solution for a) with package app-foo/bar: > > TDEPEND="app-foo/bar" > > PM installs app-foo/bar first, disabling FEATURES="test" and installs it > again, enabling FEATURES="test". > > Solution for b) with package app-foo/bar which RDEPENDs on app-foo/baz, > while app-foo/baz needs app-foo/bar to run tests successfully. > > app-foo/bar: > RDEPEND="app-foo/baz" > app-foo/baz: > TDEPEND="app-foo/bar" > > PM installs app-foo/baz, installs app-foo/bar, installs and runs tests on > app-foo/baz again. Both of this can be done with current 'test?'.
(In reply to Michał Górny from comment #1) > (In reply to Manuel Rüger from comment #0) > > This idea tries to solve two problems: > > > > a) Some packages can only run tests successfully when being already > > installed. > > Which is non-sense since they will run the tests against the previous > version when upgrading... > You can always do things like TDEPEND="=app-foo/bar-${PV}". I agree, that packages should avoid this behaviour and need to be fixed. > > b) Some packages create circular dependencies because they depend on each > > other to run tests successfully. > > This doesn't justify adding another variable. > > > Solution for a) with package app-foo/bar: > > > > TDEPEND="app-foo/bar" > > > > PM installs app-foo/bar first, disabling FEATURES="test" and installs it > > again, enabling FEATURES="test". > > > > Solution for b) with package app-foo/bar which RDEPENDs on app-foo/baz, > > while app-foo/baz needs app-foo/bar to run tests successfully. > > > > app-foo/bar: > > RDEPEND="app-foo/baz" > > app-foo/baz: > > TDEPEND="app-foo/bar" > > > > PM installs app-foo/baz, installs app-foo/bar, installs and runs tests on > > app-foo/baz again. > > Both of this can be done with current 'test?'. Probobaly I missed that app-foo/baz and app-foo/bar will be installed with disabled tests and then reinstalled afterwards (and run tests). If this is possible today, I'd like to know how you would do that?
At the moment, this can be done manually by user. If you wish, provide a patch for portage to handle that. Creating another variable that does the same will not make code appear magically.
I don't see how a PM can detect if tests need to be disabled to avoid circular dependencies. Being explicit using an additional variable is more efficient than backtracking here.
And how is exactly 'DEPEND=test?' different from 'TDEPEND'? It just differs in the name...
If TDEPEND is defined, PM knows that it has to drop FEATURES="test" on first install.
So if the circular dependency is mutual, e.g. A has TDEPEND on B, and B has TDEPEND on A, portage will never be able to solve it anyway. Unless it figures out that either of the packages is installed... which means we're back to square one since TDEPEND doesn't change a thing.
If neither package A nor B is installed and you run a command to install A B, the PM installs A without depending on B and without running tests, B without depending on A and without running tests. After each TDEPEND has been satisfied the PM installs A and runs the tests, installs B and runs the tests. I don't see the problem here?
But if PM needs to know whether TDEPEND is satisfied or not... it is no different from exception for DEPEND=test?. So you're steeping the learning curve for no real advantage. Technicalities aside, I'm not convinced that silently disabling tests is a good idea. If I want tests done, I don't want to have untested packages installed behind my back. You don't provide any clean strategy for handling test failures following the install. So, you install foo without tests, then try to reinstall it with tests and they fail. But the package is installed already, so the damage is done...
(In reply to Manuel Rüger from comment #2) > You can always do things like TDEPEND="=app-foo/bar-${PV}". I agree, that > packages should avoid this behaviour and need to be fixed. Won't work if USE flags have been changed.
After the hell I am hitting for upgrading with new python targets 3.8 and lots of python packages with circular deps due to tests... I agree that pretending us to go package per package installing it without tests to unbreak the cycle is terrible. I would even be fine if portage could simply output the whole list of packages that is trying to merge to allow me to simply merge them temporally with USE="-test". I am spending literally *hours* to solve that (as each time is re-emerging some package without USE test, then re-run emerge -uDN world, then repeat the same again and again)
But I agree that maybe adding a special handling for test? or doc? would be enough over adding one more DEP for each ... :/ Also, as the logic to run automatically emerge without the USE and later with the USE enabled would be a bit complicated to implement... simply listing the package atoms that are being pulled would be ok for me to let me merge them manually in one round
(In reply to Pacho Ramos from comment #11) > After the hell I am hitting for upgrading with new python targets 3.8 and > lots of python packages with circular deps due to tests... I agree that > pretending us to go package per package installing it without tests to > unbreak the cycle is terrible. I would even be fine if portage could simply > output the whole list of packages that is trying to merge to allow me to > simply merge them temporally with USE="-test". But it does. "USE=-test emerge ..." will emerge them all without test deps. However, that obviously won't solve conflicts that occur purely between test dependencies.
But that will rebuild a different world set, a world set not including the many of the conflicting packages but including lots of non-conflicting ones that, later, will need to get rebuilt again (sometimes bug things like libreoffice). And the worst is that, after rebuilding that, you try to enable tests again globally and you need to, again, resolve many packages manually and one by one. On the other hand, if portage would simply have an option to list all the packages pulled by a USE="test" emerge -uDN world, we would, indeed, be able to run USE="-test" over that packages
I presume you've tried '--with-test-deps y' (possibly with USE=-test) and it didn't resolve the problem in question?
I didn't know that option, the problem is that now I think that I re-emerged all the packages with this conflicts :S But I will try to remember for the next time thanks... ;)
The thing is, I'm all for TDEPEND *if* we can make some meaningful change through it. Say, if the --with-test-deps were able to treat TDEPEND like PDEPEND, and install all of them without cycles.
One advantage of approach in bug #697914 is that it doesn't require any changes to PMS to work, so can work with any EAPI. IME, the current use of USE="test" to gate this is adequate information to direct a package manager how to do things. The only real debate IME is *When* the package manager should do this, as clearly for some people, the default mechanism of allowing packages to be installed without testing, when they've specified FEATURES="test", is undesirable. So instead of lumping those who want that behaviour, with those who don't want that behaviour, a package-manager feature that toggles between the two seems the best approach.
(In reply to Michał Górny from comment #17) > The thing is, I'm all for TDEPEND *if* we can make some meaningful change > through it. Say, if the --with-test-deps were able to treat TDEPEND like > PDEPEND, and install all of them without cycles. Actually, I suspect portage can do that today with minimal patching. But as-is, --with-test-deps doesn't introduce real cycles as long as the dependencies themselves aren't USE="test" As much as I *like* the semantic clarity of an explicit TDEPEND, as far as the data is concerned, the data to make this decision is already present, as long as portage has the capacity to control whether the USE="test" flag is actually activated when building the package. Portage would be effectively violating PMS in some regards to do this, but not in a way that has any meaningful impact.
And something that bothers me with TDEPEND as a proposal is you still need a mechanism to determine "is testing going to actually happen for this package", so you still need the capacity in the package to do `if use test ; then ... ` ( For instance, I have many packages where I have to conditionally patch things based on whether or not tests will run, as if tests aren't getting run, not patching the sources can result in build-time excessive dependencies that aren't actually used for anything, due to limitations in upstream tooling ) If TDEPEND supplants DEPEND=test? , then by proxy, it removes the need for IUSE="test" But without IUSE="test", you have no mechanism to answer this question. So a TDEPEND proposal without specifying how packages query "are tests getting run" is an incomplete proposal.
(In reply to Michał Górny from comment #17) > The thing is, I'm all for TDEPEND *if* we can make some meaningful change > through it. Say, if the --with-test-deps were able to treat TDEPEND like > PDEPEND, and install all of them without cycles. The --with-test-deps PDEPEND-like behavior is implemented now (bug 703348), but only for USE=-test since otherwise it would violate PMS. (In reply to Pacho Ramos from bug 703348 comment #6) > Could it also allow circular deps when USE=test *AND* FEATURES="-test" ? > Otherwise I need to rebuild monsters as webkit-gtk two times :/ It's a reasonable idea, but PMS does not specify any special relationship between FEATURES=-test and USE=test, so it would technically violate PMS. TDEPEND would solve this problem by eliminating all interaction with USE=test.
Realistically, we may need TDEPEND and TBDEPEND.
