Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 606730

Summary: dev-texlive/texlive-basic-2016 and co hard blockers (workaround specified in Whiteboard field)
Product: Gentoo Linux Reporter: Nick Sarnie <sarnex>
Component: Current packagesAssignee: Portage team <dev-portage>
Status: CONFIRMED ---    
Severity: major CC: ag+services, alarig, bureau, gentoo, gentoo, gentoo, gentoo, harrisl, itumaykin+gentoo, jesse, leonard, mark+gentoobugs, Martin.vGagern, pacho, teika, tex, tokenring, tomaszg, wschlich
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: https://wiki.gentoo.org/wiki/TeX_Live#Upgrading
See Also: https://bugs.gentoo.org/show_bug.cgi?id=621308
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 250286    
Bug Blocks:    
Attachments: build log

Description Nick Sarnie gentoo-dev 2017-01-21 19:21:34 UTC
Hi, after the texlive 2016 ebuild update, I get the following emerge error when I try to update.

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] media-libs/audiofile-0.3.6-r3 [0.3.6-r2]
[ebuild     U  ] app-text/texlive-core-2016 [2015-r1]
[ebuild     U  ] dev-texlive/texlive-basic-2016 [2015]
[ebuild     U  ] dev-texlive/texlive-fontutils-2016 [2015]
[ebuild     U  ] dev-texlive/texlive-genericrecommended-2016 [2015]
[ebuild     U  ] dev-texlive/texlive-fontsrecommended-2016 [2015]
[ebuild     U  ] dev-texlive/texlive-latex-2016 [2015]
[ebuild     U  ] dev-texlive/texlive-latexrecommended-2016 [2015-r1]
[ebuild     U  ] app-text/texlive-2016 [2015]
[blocks B      ] <dev-texlive/texlive-basic-2016 ("<dev-texlive/texlive-basic-2016" is hard blocking dev-texlive/texlive-basic-2016)
[blocks B      ] <dev-texlive/texlive-latex-2016 ("<dev-texlive/texlive-latex-2016" is hard blocking dev-texlive/texlive-latex-2016)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  (dev-texlive/texlive-basic-2016:0/0::gentoo, ebuild scheduled for merge) pulled in by
    >=dev-texlive/texlive-basic-2016 required by (dev-texlive/texlive-fontutils-2016:0/0::gentoo, ebuild scheduled for merge)
    >=dev-texlive/texlive-basic-2016 required by (app-text/texlive-2016:0/0::gentoo, ebuild scheduled for merge)
    >=dev-texlive/texlive-basic-2016 required by (dev-texlive/texlive-genericrecommended-2016:0/0::gentoo, ebuild scheduled for merge)
    >=dev-texlive/texlive-basic-2016 required by (dev-texlive/texlive-fontsrecommended-2016:0/0::gentoo, ebuild scheduled for merge)
    >=dev-texlive/texlive-basic-2016 required by (dev-texlive/texlive-latex-2016:0/0::gentoo, ebuild scheduled for merge)

  (dev-texlive/texlive-latex-2016:0/0::gentoo, ebuild scheduled for merge) pulled in by
    >=dev-texlive/texlive-latex-2016 required by (app-text/texlive-2016:0/0::gentoo, ebuild scheduled for merge)
    dev-texlive/texlive-latex required by (dev-tex/xcolor-2.11:0/0::gentoo, installed)
    >=dev-texlive/texlive-latex-2016 required by (dev-texlive/texlive-latexrecommended-2016:0/0::gentoo, ebuild scheduled for merge)



Thanks,
Sarnex

Reproducible: Always
Comment 1 Harris Landgarten 2017-01-21 20:05:11 UTC
Same here.
Comment 2 Aric Belsito 2017-01-21 23:33:03 UTC
Same.

Please fix DEPEND of texlive-basic and texlive-latex
Comment 3 Alexis Ballier gentoo-dev 2017-01-22 04:45:14 UTC
It is not a circular dep, but !! blockers.

In short, due to a few changes and file moves, building tl-basic-2016 with tl-basic-2015 installed or tl-latex-2016 with tl-latex-2015 installed will fail at buildtime. Hence the !! blockers in both of them. I'm not sure if portage has or should have the ability to automatically resolve this by removing offending packages first, reassigning to portage team.
Comment 4 Harris Landgarten 2017-01-22 05:28:43 UTC
In order to get paludis to stop complaining about unsolvable order I had to remove texlive-basis-2015,  texlive-latex-2015 and texlive-lang-english-2015.

Then texlive-basic-2016 fails with:

50 preloaded fonts
No pages of output.
Transcript written on dviluatex.log.
fmtutil [INFO]: /var/tmp/paludis/dev-texlive-texlive-basic-2016/work/texmf-var/web2c/luatex/dviluatex.fmt installed.

Error:
  * In program cave perform install --hooks --managed-output --output-exclusivity with-others =dev-texlive/texlive-basic-2016:0::gentoo --destination installed --x-of-y 1 of 14:
  * When installing 'dev-texlive/texlive-basic-2016:0::gentoo':
  * When running an ebuild command on 'dev-texlive/texlive-basic-2016:0::gentoo':
  * Install failed for 'dev-texlive/texlive-basic-2016:0::gentoo' (paludis::ActionFailedError)

fmtutil [INFO]: Successfully rebuilt formats: 5
fmtutil [INFO]: Failed to build: 3 (pdftex/pdftex pdftex/pdfetex pdftex/etex)
fmtutil [INFO]: Total formats: 8
fmtutil [INFO]: exiting with status 3
fmtutil [ERROR]: running `pdftex -ini   -jobname=pdftex -progname=pdftex -translate-file=cp227.tcx *pdfetex.ini </dev/null' return status 1
fmtutil [ERROR]: return error due to options --strict
fmtutil [ERROR]: running `pdftex -ini   -jobname=pdfetex -progname=pdfetex -translate-file=cp227.tcx *pdfetex.ini </dev/null' return status 1
fmtutil [ERROR]: return error due to options --strict
fmtutil [ERROR]: running `pdftex -ini   -jobname=etex -progname=etex -translate-file=cp227.tcx *etex.ini </dev/null' return status 1
fmtutil [ERROR]: return error due to options --strict

!!! ERROR in dev-texlive/texlive-basic-2016::gentoo:
!!! In texlive-module_src_compile at line 4901
!!! failed to build format texmf-dist/fmtutil/format.texlive-basic.cnf

!!! Call stack:
!!!    * texlive-module_src_compile (/var/tmp/paludis/dev-texlive-texlive-basic-2016/temp/loadsaveenv:4901)
!!!    * src_compile (/var/tmp/paludis/dev-texlive-texlive-basic-2016/temp/loadsaveenv:4020)
!!!    * ebuild_f_compile (/usr/libexec/paludis/2/src_compile.bash:56)
!!!    * ebuild_main (/usr/libexec/paludis/ebuild.bash:675)
!!!    * main (/usr/libexec/paludis/ebuild.bash:698)

diefunc: making ebuild PID 15525 exit with error
die trap: exiting with error.

Failed install to / for dev-texlive/texlive-basic-2016:0::gentoo
Comment 5 Harris Landgarten 2017-01-22 05:33:51 UTC
Created attachment 460878 [details]
build log

build log
Comment 6 Alexis Ballier gentoo-dev 2017-01-22 06:07:17 UTC
(In reply to Harris Landgarten from comment #5)
> Created attachment 460878 [details]
> build log
> 
> build log

This log shows it fails because it is building english hyphenations, provided by tl-langenglish. I think paludis didnt remove properly texlive-langenglish. (check /etc/texmf/language.d*/*langenglish* ; there should be no such file after removing it)
Comment 7 Aric Belsito 2017-01-22 07:20:49 UTC
@aballier
Thanks for the clarification, I figured the !! was a typo.
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-01-22 08:24:25 UTC
(In reply to Alexis Ballier from comment #3)
> It is not a circular dep, but !! blockers.
> 
> In short, due to a few changes and file moves, building tl-basic-2016 with
> tl-basic-2015 installed or tl-latex-2016 with tl-latex-2015 installed will
> fail at buildtime. Hence the !! blockers in both of them. I'm not sure if
> portage has or should have the ability to automatically resolve this by
> removing offending packages first, reassigning to portage team.

Portage doesn't handle '!!' blockers automatically, so it's really PITA -- you basically have to figure out a 'safe' way to unmerge conflicting packages (i.e. without killing them off @world), then install/upgrade new ones.

No chance to fix the build problem somehow? It doesn't sound like a good idea to require all Gentoo texlive users to jump through hoops upgrading it.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-01-22 08:57:51 UTC
*** Bug 606726 has been marked as a duplicate of this bug. ***
Comment 10 Mathy Vanvoorden 2017-01-22 13:14:00 UTC
I was bitten by the same issue, and stumped because the blocker was on package versions that were actually in the list to be installed.

Perhaps a news article could be helpful?

I fixed it by


grep texlive /var/lib/portage/world > latex.txt

emerge -C `qlist -I texlive`

emerge -a `cat latex.txt` && emerge -uDaN world
Comment 11 Mathy Vanvoorden 2017-01-22 13:15:51 UTC
Just noticed the --deselect=n in mgorny his comment, didn't know that existed, so there you go, that's even easier ;-)
Comment 12 Harris Landgarten 2017-01-22 15:20:36 UTC
Paludis did leave lang stuff in etc. removing it solved the build failure in texlive-basis.
Comment 13 Zac Medico gentoo-dev 2017-01-22 20:00:03 UTC
(In reply to Alexis Ballier from comment #3)
> Hence the !! blockers in both of them. I'm not sure if
> portage has or should have the ability to automatically resolve this by
> removing offending packages first, reassigning to portage team.

Automatic solving of hard blockers is not implemented yet (bug 250286).

(In reply to Michał Górny from comment #8)
> Portage doesn't handle '!!' blockers automatically, so it's really PITA --
> you basically have to figure out a 'safe' way to unmerge conflicting
> packages (i.e. without killing them off @world), then install/upgrade new
> ones.

Yeah, it has some overlap with bug 199856 since it's likely to break the dependencies of installed packages, and we have to be especially careful to avoid breaking system packages or their dependencies.
Comment 14 Nick Sarnie gentoo-dev 2017-01-22 20:13:48 UTC
Mathy's solution worked for me.

Thanks,
Sarnex
Comment 15 Martin von Gagern 2017-01-22 22:01:57 UTC
It would be nice to inform users about how to deal with this upgrade, i.e. advise them to uninstall then reinstall but use --deselect=n for the former. Perhaps release a portage news item when you stabilize TL 2016?

I tried to resolve a bunch of conflicts manually, over several iterations, and finally got portage to emerge something only to have texlive-latexrecommended-2016 conflict with texlive-xetex-2015 for a bunch of font and doc files. So apparently the set of blockers, annoying as it is, is even too small! (Which in turn means that if portage COULD handle these, it might still do so incorrectly.) At that point I got annoyed enough to use the emerge -C approach myself, too, so I don't know whether I would have encountered any more conflicts on my system.
Comment 16 Tomasz Golinski 2017-01-23 17:05:09 UTC
There is a related problem with dev-texlive/texlive-mathextra being replaced by
dev-texlive/texlive-mathscience which also causes additional nasty blocks.
Comment 17 Kristian Fiskerstrand (RETIRED) gentoo-dev 2017-06-11 17:30:48 UTC
Given that this is now going stable, should a news item be posted with the -C --deselect=n line on how to upgrade?
Comment 18 quazgar 2017-06-11 20:39:17 UTC
Hmm, is there really no more elegant way?
Comment 19 Jesse Adelman 2017-06-12 23:00:17 UTC
Ah, I think this is now "CONFIRMED". :) And, no news to alert and guide the masses through the sea of blockers on a STABLE upgrade? :/
Comment 20 Zac Medico gentoo-dev 2017-06-16 16:06:58 UTC

*** This bug has been marked as a duplicate of bug 250286 ***
Comment 21 Pacho Ramos gentoo-dev 2017-06-22 10:54:28 UTC
Can we keep the bug opened? otherwise, as there is no new item telling people how to deal with this latex blockers like command specified in Whiteboard does, people have no trace about what is happening and will likely report more duplicate bugs without watching this one.
Comment 22 Sebastian Thomae 2017-06-27 11:36:16 UTC
My Boss just told me he could resolve this with a emerge -NDauvt --backtrack=100 @world
Comment 23 Teika kazura 2017-06-29 00:03:13 UTC
The sole correct fix is to write down something like
------------------------------------------------------------------------
pkg_pretend{
  # Probably has_version doesn't work this way. Sorry I don't understand ebuild (5)
  # Bug #606730
  if has_version "<dev-texlive/texlive-basic-2016"; then
     eerror "You have to manually resolve hard blocking. Sorry for inconvenience."
     eerror "See https://wiki.gentoo.org/wiki/TeX_Live#Upgrading"
     die
  fi
}
------------------------------------------------------------------------
in the ebuilds of texlive-basic, texlive-latex, and texlive-langcjk. (These 3 packages are all that contain the !! operator.)

The Wiki page section mentioned in the eerror above not only gives the workaround, but also explains what hard-block is.

BTW please:
1. Add the following to this bug's url: https://wiki.gentoo.org/wiki/TeX_Live#Upgrading
2. Remove the "*" from "texlive-latex*" in whiteboard. texlive-latexextra and texlive-latexrecommend are irrelevant. (Or you can clean whiteboard once the above url is given. Current whiteboard is truncated, and the whole is shown only as a tooltip.)

(In reply to the last comment #22)
No. Auto-resolving of hard block is lacking from portage. See comment #13.

@Pacho Ramos: +1
@Zac Medico: Please understand that what perplexes users (and even not few developers in this case!) is a bug. ("Hey, this is the situation from the technical viepoint. You understand?" is not a solution. Microsoft was something like that some decades ago. ;-)

Thanks a lot Gentoo Devs and users.
Comment 24 Zac Medico gentoo-dev 2017-06-29 00:33:09 UTC
(In reply to Sebastian Thomae from comment #22)
> My Boss just told me he could resolve this with a emerge -NDauvt
> --backtrack=100 @world

No amount of backtracking solves hard blockers (since bug 250286 is not implemented), unless it decides to skip the upgrade, which is probably not what you want.

(In reply to Teika kazura from comment #23)
> The sole correct fix is to write down something like
> ------------------------------------------------------------------------
> pkg_pretend{

That won't help because pkg_pretend is not executed after it bails out due to hard blockers.

> ------------------------------------------------------------------------
> in the ebuilds of texlive-basic, texlive-latex, and texlive-langcjk. (These
> 3 packages are all that contain the !! operator.)
> 
> The Wiki page section mentioned in the eerror above not only gives the
> workaround, but also explains what hard-block is.
> 
> BTW please:
> 1. Add the following to this bug's url:
> https://wiki.gentoo.org/wiki/TeX_Live#Upgrading

Done. Thanks!

> 2. Remove the "*" from "texlive-latex*" in whiteboard. texlive-latexextra
> and texlive-latexrecommend are irrelevant. (Or you can clean whiteboard once
> the above url is given. Current whiteboard is truncated, and the whole is
> shown only as a tooltip.)

I've just removed the whole thing because the URL is probably enough.

> (In reply to the last comment #22)
> No. Auto-resolving of hard block is lacking from portage. See comment #13.

Right.

> @Pacho Ramos: +1
> @Zac Medico: Please understand that what perplexes users (and even not few
> developers in this case!) is a bug. ("Hey, this is the situation from the
> technical viepoint. You understand?" is not a solution. Microsoft was
> something like that some decades ago. ;-)

Is there something you want me to do besides fix bug 250286? The emerge output should already include this link, but unfortunately I don't see any mention of hard blockers there:

https://wiki.gentoo.org/wiki/Handbook:X86/Working/Portage#Blocked_packages

I also don't see anything about hard blockers here:

https://wiki.gentoo.org/wiki/Knowledge_Base:Blockers
Maybe we dould
Comment 25 Pacho Ramos gentoo-dev 2017-07-01 08:19:35 UTC
(In reply to Pacho Ramos from comment #21)
> Can we keep the bug opened? otherwise, as there is no new item telling
*news item
Comment 26 Teika kazura 2017-07-03 23:15:26 UTC
@Zac: Thanks a lot.

> Is there something you want me to do besides fix bug 250286?

Let's distinguish hard-block in general and the texlive-specific situation, and also finish what can be finished. I propose:

* Make an ebuild emit the message like "When you upgrade texlive next time, it may fail due to hard-blocks. In that case see https://wiki.gentoo.org/wiki/TeX_Live#Upgrading" in pkg_postinst. (I'm not sure which ebuild is the best. app-text/texlive? Each that contains "!!" operator?)


About hard-blocks: Maybe I should repeat the following at bug #250286, but let it go:

1. As you noticed, the doc on hard-block is lacking. As far as I know, the only place where the term "hard block" appears is the emerge (1) output, i.e. <portage src>/pym/_emerge/resolver/output.py. (As a comment, it is also found in pym/portage/tests/resolver/test_merge_order.py)

In man (5) ebuild, the !! operator (more exactly an "atom prefix") is explained.

man (1) emerge should explain hard-block, or refer to Handbook.

About Handbook/Portage (https://wiki.gentoo.org/wiki/Handbook:X86/Working/Portage#Blocked_packages)
This section is "chatty", not right to the point. Improvement about that aspect is desired, too. (I can prepare a draft if you ask.)

I'm thinking of improving https://wiki.gentoo.org/wiki/Knowledge_Base:Blockers page. It ought to describe practical solutions users have to take, if such thing (see the next item) is there.

2. For texlive, the solution in the wiki works. But in general, what's the purpose of hard-blocks? How are they used? A survey will be nice, asking maintainers of the packages that use !!. (But you shouldn't wait for it. The above point 1 needs to be fixed first.) Have users experienced any problem with hard blocks?

3. As I wrote in the wiki, (soft or hard) blocking in Gentoo is only "one-way", contrary to intuition and the conventional explanation. It has to be described, too, as "more details".

More specifically, texlive-hebrew-2012 can be re-emerged after upgrading texlive-basic etc, although their ebuilds say "!!<texlive-hebrew-2016". One-way blocking may be exceptional, but for them, the message "A is blocking B" is confusing. (I've always wondered which is the "blocker". "Shunner" or "disliker" may be more self-explanatory, but I don't think it's good to change an already deeply fixed term.) Maybe ebuild authors know it - otherwise you wouldn't be able to handle package merger/splitting - but for me, just a user, it was a surprise.

> Is there something you want me to do besides fix bug 250286?

Ah, well, in EAPI=7, "pkg_first_of_all" such that runs in "emerge -p", so that the eerror I proposed in comment #23 can be put.

You know, perl upgrading often fails with subslot conflicts. This led to a wrong updating instruction of a glsa in last Feb, and I asked to put a link to the Wiki page "Perl" (bug #614392. See also bug 608168, comment 5) So "pkg_firt_of_all " can be used for perl, too.

If you think it's worth serious consideration, you or I can open a bug for it..

Regards.
Comment 27 Teika kazura 2017-07-05 22:59:59 UTC
More on EAPI 7.

Sorry for moving, but on my second thought, let me replace pkg_first_of_all with
1. pkg_before_ask: This is run even in "emerge -p", or *before* yes/no confirmation for "emerge -a". It's the best place for CONFIG_CHECK (of linux-info.eclass) for example.

2. pkg_catch_dep_fail: It's run when the dependency that the ebuild requires fails. (Meaning it is also run in "emerge -p".) It should also be able to catch the "type" like hard-block, subslots, etc. It can be used at least for texlive and dev-lang/perl. For these two cases, pkg_catch_dep_fail will print an appropriate message like "see the Wiki"; it suffices there.