For example: x4 ~ # emerge -pv scala These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild N ] dev-java/javatoolkit-0.3.0-r9::gentoo PYTHON_TARGETS="python2_7" 17 KiB [ebuild N ] app-eselect/eselect-java-0.2.0-r1::gentoo 14 KiB [ebuild N ] app-eselect/eselect-scala-0.1.1::gentoo 2 KiB [ebuild N ] sys-apps/baselayout-java-0.1.0::gentoo 71 KiB [ebuild N ] dev-java/java-config-2.2.0-r3:2::gentoo USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 (-python3_3) -python3_5" 51 KiB [ebuild N ] dev-java/icedtea-bin-3.1.0:8::gentoo USE="alsa cups gtk webstart -doc -examples -headless-awt (-multilib) -nsplugin -pulseaudio (-selinux) -source" 60,243 KiB [ebuild N ] virtual/jdk-1.8.0-r3:1.8::gentoo 0 KiB [ebuild N ] virtual/jre-1.8.0-r1:1.8::gentoo 0 KiB [ebuild N ] dev-java/icedtea-web-1.6.2::gentoo USE="-doc -javascript -nsplugin -tagsoup {-test}" 1,782 KiB [ebuild N ] dev-java/ant-core-1.9.2::gentoo USE="-doc -source" 3,338 KiB [ebuild N ] dev-java/hawtjni-runtime-1.10::gentoo USE="-doc -source" 1,566 KiB [ebuild N ] dev-java/bcel-6.0_rc3::gentoo USE="-doc -source {-test}" 909 KiB [ebuild N ] dev-java/xml-commons-external-1.4.01-r1:1.4::gentoo USE="-doc -source" 642 KiB [ebuild N ] dev-java/jakarta-oro-2.0.8-r4:2.0::gentoo USE="-doc -examples -source" 338 KiB [ebuild N ] dev-java/jzlib-1.0.7-r2::gentoo USE="-doc -examples -source" 50 KiB [ebuild N ] dev-java/jsch-0.1.52::gentoo USE="zlib -doc -examples -source" 356 KiB [ebuild N ] dev-java/commons-logging-1.2-r1::gentoo USE="-avalon-framework -avalon-logkit -doc -log4j -servletapi -source {-test}" 185 KiB [ebuild N ] dev-java/ant-nodeps-1.9.2::gentoo 0 KiB [ebuild N ] dev-java/xjavac-20110814:1::gentoo 3 KiB [ebuild N ] dev-java/xml-commons-resolver-1.2::gentoo USE="-doc -source" 257 KiB [ebuild N ] dev-java/commons-cli-1.2:1::gentoo USE="-doc -source {-test}" 67 KiB [ebuild N ] dev-java/commons-net-3.2::gentoo USE="-doc -examples -source" 370 KiB [ebuild N ] dev-java/commons-collections-3.2.1-r1::gentoo USE="-doc -source {-test} -test-framework" 596 KiB [ebuild N ] dev-java/commons-codec-1.7::gentoo USE="-doc -source {-test}" 285 KiB [ebuild N ] dev-java/commons-httpclient-3.1-r1:3::gentoo USE="-doc -examples -source {-test}" 1,839 KiB [ebuild N ] dev-java/xerces-2.11.0-r1:2::gentoo USE="-doc -examples -source" 1,751 KiB [ebuild N ] dev-java/commons-vfs-1.0::gentoo USE="-doc -source" 273 KiB [ebuild N ] dev-java/ant-ivy-1.4.1-r1::gentoo USE="-doc -examples -source {-test}" 735 KiB [ebuild N ] dev-java/ant-contrib-1.0_beta3-r1::gentoo USE="-doc -source" 3,221 KiB [ebuild N ] dev-lang/scala-2.11.8:2.11/2.11.8::gentoo USE="-binary -doc -emacs -source" 106,406 KiB Total: 30 packages (30 new), Size of downloads: 185,352 KiB =========================================================================== A few seconds later an older version is chosen: x4 ~ # emerge -pv scala These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild N ] dev-java/javatoolkit-0.3.0-r9::gentoo PYTHON_TARGETS="python2_7" 17 KiB [ebuild N ] app-eselect/eselect-java-0.2.0-r1::gentoo 14 KiB [ebuild N ] app-eselect/eselect-scala-0.1.1::gentoo 2 KiB [ebuild N ] sys-apps/baselayout-java-0.1.0::gentoo 71 KiB [ebuild N ] dev-java/java-config-2.2.0-r3:2::gentoo USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 (-python3_3) -python3_5" 51 KiB [ebuild N ] dev-java/icedtea-bin-7.2.6.7:7::gentoo USE="alsa cups gtk webstart -cjk -doc -examples -headless-awt (-multilib) -nsplugin -nss -pulseaudio (-selinux) -source" 51,877 KiB [ebuild N ] virtual/jdk-1.7.0-r2:1.7::gentoo 0 KiB [ebuild N ] virtual/jre-1.7.0-r2:1.7::gentoo 0 KiB [ebuild N ] dev-java/icedtea-web-1.6.2::gentoo USE="-doc -javascript -nsplugin -tagsoup {-test}" 1,782 KiB [ebuild N ] dev-java/ant-core-1.9.2::gentoo USE="-doc -source" 3,338 KiB [ebuild N ] dev-java/hawtjni-runtime-1.10::gentoo USE="-doc -source" 1,566 KiB [ebuild N ] dev-java/bcel-6.0_rc3::gentoo USE="-doc -source {-test}" 909 KiB [ebuild N ] dev-java/xml-commons-external-1.4.01-r1:1.4::gentoo USE="-doc -source" 642 KiB [ebuild N ] dev-java/jakarta-oro-2.0.8-r4:2.0::gentoo USE="-doc -examples -source" 338 KiB [ebuild N ] dev-java/jzlib-1.0.7-r2::gentoo USE="-doc -examples -source" 50 KiB [ebuild N ] dev-java/jsch-0.1.52::gentoo USE="zlib -doc -examples -source" 356 KiB [ebuild N ] dev-java/commons-logging-1.2-r1::gentoo USE="-avalon-framework -avalon-logkit -doc -log4j -servletapi -source {-test}" 185 KiB [ebuild N ] dev-java/ant-nodeps-1.9.2::gentoo 0 KiB [ebuild N ] dev-java/xml-commons-resolver-1.2::gentoo USE="-doc -source" 257 KiB [ebuild N ] dev-java/xjavac-20110814:1::gentoo 3 KiB [ebuild N ] dev-java/commons-cli-1.2:1::gentoo USE="-doc -source {-test}" 67 KiB [ebuild N ] dev-java/commons-collections-3.2.1-r1::gentoo USE="-doc -source {-test} -test-framework" 596 KiB [ebuild N ] dev-java/commons-net-3.2::gentoo USE="-doc -examples -source" 370 KiB [ebuild N ] dev-java/commons-codec-1.7::gentoo USE="-doc -source {-test}" 285 KiB [ebuild N ] dev-java/commons-httpclient-3.1-r1:3::gentoo USE="-doc -examples -source {-test}" 1,839 KiB [ebuild N ] dev-java/xerces-2.11.0-r1:2::gentoo USE="-doc -examples -source" 1,751 KiB [ebuild N ] dev-java/commons-vfs-1.0::gentoo USE="-doc -source" 273 KiB [ebuild N ] dev-java/ant-ivy-1.4.1-r1::gentoo USE="-doc -examples -source {-test}" 735 KiB [ebuild N ] dev-java/ant-contrib-1.0_beta3-r1::gentoo USE="-doc -source" 3,221 KiB [ebuild N ] dev-lang/scala-2.11.8:2.11/2.11.8::gentoo USE="-binary -doc -emacs -source" 106,406 KiB Total: 30 packages (30 new), Size of downloads: 176,986 KiB
I haven't seen this random behaviour before but I do know that Portage usually picks virtual/jdk-1.7 over virtual/jdk-1.8 simply because the underlying icedtea(-bin) version is newer for 1.7. This is due to a change in the icedtea versioning scheme. This is contrary to what you would expect and we probably would have taken a different approach had we known this would cause so much trouble. Strangely I don't think we've had a bug report filed about it yet. I consider this to be a Portage bug so I'm reassigning it but I suspect it won't be fixed by the time we remove Java 7, which is on the current agenda.
This random behavior is interesting behavior. I suspect that having =virtual/jdk-1.7* to the left of =virtual/jdk-1.8* in the scala dependencies may help to trigger the buggy portage behavior: || ( =virtual/jdk-1.7* =virtual/jdk-1.8* ) Generally, preferred versions should be on the left. Anyway, we should certainly fix portage to handle this better.
I can reproduce this very reliably. The probability of either outcome seems to be 50%.
Created attachment 451955 [details] jdk-1.7 debug log (portage-2.3.2)
Created attachment 451957 [details] jdk-1.8 debug log (portage-2.3.2)
(In reply to Zac Medico from comment #4) > Created attachment 451955 [details] > jdk-1.7 debug log (portage-2.3.2) The problem occurs when processing the ant-core dependencies: Parent: (dev-java/ant-core-1.9.2:0/0::gentoo, ebuild scheduled for merge) Depstring: || ( >=virtual/jdk-1.5 dev-java/gcj-jdk ) I'm able to trigger the problem 100% of the time when I emerge ant-core in a stage3.
I've isolated the incorrect behavior to a problem in the construction of the _dep_choice.cp_map dictionary inside the dep_zapdeps function, with this input: ['||', ['dev-java/icedtea-bin:8', '=virtual/jdk-1.8.0-r3', '>=virtual/jdk-1.5'], ['dev-java/icedtea-bin:7', '=virtual/jdk-1.7.0-r2', '>=virtual/jdk-1.5']] The cp_map dictionaries end up as follows: For virtual/jdk-1.8.0-r3: {'dev-java/icedtea-bin': <Package ('ebuild', '/', 'dev-java/icedtea-bin-3.1.0', 'merge', 'gentoo')>, 'virtual/jdk': <Package ('ebuild', '/', 'virtual/jdk-1.8.0-r3', 'merge', 'gentoo')>} For virtual/jdk-1.7.0-r2: {'dev-java/icedtea-bin': <Package ('ebuild', '/', 'dev-java/icedtea-bin-7.2.6.7', 'merge', 'gentoo')>, 'virtual/jdk': <Package ('ebuild', '/', 'virtual/jdk-1.8.0-r3', 'merge', 'gentoo')>} The cp_map for virtual/jdk-1.7.0-r2 erroneously contains virtual/jdk-1.8.0-r3 because that's the highest virtual/jdk matched by the '>=virtual/jdk-1.5' atom. When the versions are compared, the virtual/jdk-1.7.0-r2 cp_map appears to be the best choice because: * it has a higher version of dev-java/icedtea-bin (7.2.6.7 is higher than 3.1.0) * the version of virtual/jdk has erroneously been set to virtual/jdk-1.8.0-r3
(In reply to Zac Medico from comment #2) > This random behavior is interesting behavior. I suspect that having > =virtual/jdk-1.7* to the left of =virtual/jdk-1.8* in the scala dependencies > may help to trigger the buggy portage behavior: > > || ( =virtual/jdk-1.7* =virtual/jdk-1.8* ) > > Generally, preferred versions should be on the left. Anyway, we should > certainly fix portage to handle this better. It turns out that the ordering is significant because dev-java/icedtea-bin:8 has a lower version than dev-java/icedtea-bin:7. This reverse version ordering defeat's portage's automatic re-ordering, so it's very important to put =virtual/jdk-1.8* on the left side.
I've posted this patch for review: https://archives.gentoo.org/gentoo-portage-dev/message/8026baee76ff2fd8be8c9face5467793 https://github.com/gentoo/portage/pull/65
(In reply to Zac Medico from comment #8) > It turns out that the ordering is significant because dev-java/icedtea-bin:8 > has a lower version than dev-java/icedtea-bin:7. This reverse version > ordering defeat's portage's automatic re-ordering, so it's very important to > put =virtual/jdk-1.8* on the left side. That sounds like a bug worth reporting. I'm pretty sure it will confuse the hell out of paludis which assumes versions grow monotonically.
(In reply to Michał Górny from comment #10) > (In reply to Zac Medico from comment #8) > > It turns out that the ordering is significant because dev-java/icedtea-bin:8 > > has a lower version than dev-java/icedtea-bin:7. This reverse version > > ordering defeat's portage's automatic re-ordering, so it's very important to > > put =virtual/jdk-1.8* on the left side. > > That sounds like a bug worth reporting. I'm pretty sure it will confuse the > hell out of paludis which assumes versions grow monotonically. Maybe rename the icedtea 7 packages to dev-java/icedtea7 and dev-java/icedtea7-bin for the remainder of their lives.
(In reply to Zac Medico from comment #9) > I've posted this patch for review: > > https://archives.gentoo.org/gentoo-portage-dev/message/ > 8026baee76ff2fd8be8c9face5467793 > https://github.com/gentoo/portage/pull/65 In the master branch now: https://gitweb.gentoo.org/proj/portage.git/commit/?id=cd864267a463769cbd40e058611d2488cc15bf70
Wow, fast work, Zac! Sorry for saying we would probably remove Java 7 first, you totally proved me wrong there. :D I can't remember all the cases where I've seen this problem but ant-core was one of them and I can confirm that your fix works here. Java team doesn't really touch the Scala stuff but I have told the maintainer not to use || with the virtuals because if confuses the Java eclasses. Looks like I'll have to remind him. ;) ant-core is a special case. We don't support gcj-jdk as a general compiler so it's not in the virtuals but we need to build ant-core with it in order to bootstrap icedtea:7. This can go away soon because icedtea:8 isn't built with ant. As for the icedtea versioning mess, there seems little point in changing tack now. I want 7 gone ASAP. If Paludis fumbled it that badly, I think we would have heard about it by now.
Fixed in portage-2.3.3.
