Version bump to 2.6.0 coming up in a minute…
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e4b19b25c4e401cfaec6c19d6e82fce5905ad7ad commit e4b19b25c4e401cfaec6c19d6e82fce5905ad7ad Author: Sebastian Pipping <sping@gentoo.org> AuthorDate: 2024-02-06 17:42:54 +0000 Commit: Sebastian Pipping <sping@gentoo.org> CommitDate: 2024-02-06 17:43:23 +0000 dev-libs/expat: 2.6.0 (fixing CVE-2023-52425) Bug: https://bugs.gentoo.org/923951 Signed-off-by: Sebastian Pipping <sping@gentoo.org> dev-libs/expat/Manifest | 1 + dev-libs/expat/expat-2.6.0.ebuild | 95 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+)
FYI, CPython is broken with 2.6.0. I'm going to investigate later today. Hopefully it's some unrelated change and not the CVE fix.
(In reply to Michał Górny from comment #2) > FYI, CPython is broken with 2.6.0. I'm going to investigate later today. > Hopefully it's some unrelated change and not the CVE fix. I have created a pull request https://github.com/python/cpython/pull/115138 now to fix the CPython test suite for Expat >=2.6.0. Let's see how CPython upstream likes it.
How about dev-lang/python hard deps on <2.6? dev-libs/expat:0 (dev-libs/expat-2.6.0:0/0::gentoo, ebuild scheduled for merge) USE="unicode -examples -static-libs -test" pulled in by =dev-libs/expat-2.6.0 (Argument) (dev-libs/expat-2.5.0:0/0::gentoo, installed) USE="unicode -examples -static-libs -test" pulled in by <dev-libs/expat-2.6:0/0= required by (dev-lang/python-3.11.8:3.11/3.11::gentoo, installed) USE="ensurepip gdbm ncurses pgo readline sqlite ssl verify-sig -bluetooth -build -debug -examples -libedit -test -tk -valgrind"
(In reply to David Carlos Manuelda from comment #4) > How about dev-lang/python hard deps on <2.6? > > dev-libs/expat:0 Did you miss the above discussion...?
(In reply to Sam James from comment #5) > (In reply to David Carlos Manuelda from comment #4) > > How about dev-lang/python hard deps on <2.6? > > > > dev-libs/expat:0 > > Did you miss the above discussion...? To be fair, that change in dev-lang/python was news to me also before it was brought up here, and it was not mentioned above like that, I re-checked; given that it was tests breaking rather than Python-based libraries or applications, an effectively system-wide blocker seems a bit drastic to me. One can err on both sides, sure — being to optimistic or too pessimistic — but this <2.6.0 dependency that was not communicated is not ideal. On a side note, many other distros have upgraded to Expat 2.6.0 already (https://repology.org/project/expat/history) without any blockers or bug reports that I would be aware of, as of today.
(In reply to Sebastian Pipping from comment #6) > (In reply to Sam James from comment #5) > > (In reply to David Carlos Manuelda from comment #4) > > > How about dev-lang/python hard deps on <2.6? > > > > > > dev-libs/expat:0 > > > > Did you miss the above discussion...? > > To be fair, that change in dev-lang/python was news to me No, I know, but I meant they commented after those comments already existed, so I wasn't sure if they missed it or were asking something else. also before it was > brought up here, and it was not mentioned above like that, I re-checked; > given that it was tests breaking rather than Python-based libraries or > applications, an effectively system-wide blocker seems a bit drastic to me. Possibly -- although one can't really know if it's serious until upstream respond, by which point damage could've been done.
(In reply to Sam James from comment #7) > > given that it was tests breaking rather than Python-based libraries or > > applications, an effectively system-wide blocker seems a bit drastic to me. > > Possibly -- although one can't really know if it's serious until upstream > respond, by which point damage could've been done. That is the pessimistic approach. I would understand that approach for Gentoo stable but for Gentoo testing?
(In reply to Sebastian Pipping from comment #8) > (In reply to Sam James from comment #7) > > > given that it was tests breaking rather than Python-based libraries or > > > applications, an effectively system-wide blocker seems a bit drastic to me. > > > > Possibly -- although one can't really know if it's serious until upstream > > respond, by which point damage could've been done. > > That is the pessimistic approach. I would understand that approach for > Gentoo stable but for Gentoo testing? We don't really want to leave something in ~arch if we have an indication that it's broken. Tests failing for something critical to the system running are a good indication something is broken.
(I appreciate that your perspective is that it isn't really broken, and it probably isn't, just explaining how this is kind of standard for how things go when a broken library appears if there's test failures in something critical. I suspect mgorny wasn't aware you're also the upstream maintainer.)
(In reply to Sebastian Pipping from comment #8) > That is the pessimistic approach. I would understand that approach for > Gentoo stable but for Gentoo testing? These were mostly stable Python versions. And we're talking of security bug, so it was reasonable to expect it would go stable as well.
(In reply to Michał Górny from comment #11) > (In reply to Sebastian Pipping from comment #8) > > That is the pessimistic approach. I would understand that approach for > > Gentoo stable but for Gentoo testing? > > These were mostly stable Python versions. As far as I can see, literally all versions of dev-lang/python are blocking Expat 2.6.0 right now (commit 3fc8fd73d22ddfa2fece31ef7326ff51a8a646ec): # ls -1 *.ebuild | grep -F -v -f <(grep -lF '<dev-libs/expat-2.6' *.ebuild) <no output> For my own ~amd64 dev machine, Expat was downgraded to 2.5.0 by emerge also. Could you elaborate what you mean by mostly stable versions? > And we're talking of security > bug, so it was reasonable to expect it would go stable as well. Correct, but isn't this very bug here the place to discuss/initiate/stop/delay stabilization rather than blocking Expat 2.6.0 even for testing? Can we have Expat 2.6.0 unblocked for Gentoo testing please, or at least discuss when to unblock it? Debian sid (which is not Debian experimental) has Expat 2.6.0, why don't we in Gentoo testing?
(In reply to Sebastian Pipping from comment #12) > Could you elaborate what you mean by mostly stable versions? I mean that all of them, except for 3.11.8, 3.12.2 and 3.13* are stable? > Correct, but isn't this very bug here the place to > discuss/initiate/stop/delay stabilization rather than blocking Expat 2.6.0 > even for testing? > > Can we have Expat 2.6.0 unblocked for Gentoo testing please, or at least > discuss when to unblock it? Debian sid (which is not Debian experimental) > has Expat 2.6.0, why don't we in Gentoo testing? I'm against making any further changes until upstream decides how to proceed.
I would like to note that because of the central role of dev-lang/python in Gentoo, blocker dependency "<dev-libs/expat-2.6" in dev-lang/python has the same effect as an entry in package.mask except that entries in package.mask can be overridden via package.unmask by users locally with ease, while "<dev-libs/expat-2.6" in dev-lang/python cannot be bypassed (ignoring customer overlay stunts) and hence the current approach dominates all stability targets and all users, taking away user choice and further testing of dev-libs/expat-2.6.0 in Gentoo. The least we should do is to turn this into a package.mask entry and get upper bound "<dev-libs/expat-2.6" out of the testing dev-lang/python ebuilds. That or just dropping the upper bound, depending on how pessimistic you want to go. You would lose nothing, it only takes seconds to make that change, and everyone would get choice back. On a side note, I'm happy to team up with the CPython project on exposing the >=2.6.0 API XML_SetReparseDeferralEnabled (and other unexposed API) but it requires understanding of the non-trivial CPython implementation and is not something I can do economically in isolation and without someone with merge power in the CPython project. I can jump on a voice call with CPython project members on this topic any time. But that's not a Gentoo matter, and outside of broken CPython tests there is no evidence that this would be a bugfix (with regard to any Python-based library or application) rather than an improvement, as of today.
(In reply to Sebastian Pipping from comment #12) > Can we have Expat 2.6.0 unblocked for Gentoo testing please, or at least > discuss when to unblock it? Debian sid (which is not Debian experimental) > has Expat 2.6.0, why don't we in Gentoo testing? It's not immediately obvious that just because another distro is okay with software that doesn't pass tests, Gentoo should be as well. In fact, distros often make different decisions and tradeoffs? Sometimes that has been a reason to be extremely worried about whether it is at all safe to run certain distros (naming no names here but I'm sure everyone here knows of some stories). (But also, Debian sid has different stability requirements than ~arch does so I think the comparison is flawed anyway.) (In reply to Sebastian Pipping from comment #14) > I would like to note that because of the central role of dev-lang/python in > Gentoo, blocker dependency "<dev-libs/expat-2.6" in dev-lang/python has the > same effect as an entry in package.mask except that entries in package.mask > can be overridden via package.unmask by users locally with ease, while > "<dev-libs/expat-2.6" in dev-lang/python cannot be bypassed (ignoring > customer overlay stunts) and hence the current approach dominates all > stability targets and all users, taking away user choice and further testing > of dev-libs/expat-2.6.0 in Gentoo. This is all fine and well but on the other hand if the dev-lang/python maintainers do not *know* whether the failing testsuite is problematic, then the safe action is to require a version that allows the tests to pass. It's not particularly great to allow a version for which you've been given some strong hints that it is incompatible. Even if it later turns out the answer is it isn't a problem, that's a matter of adapting to new information. Also: upstream for this is cpython, not expat, so it's understandable that dev-lang/python maintainers are more concerned about the evaluation of cpython than anything else, and wish to see cpython publish an official patch. (In reply to Sebastian Pipping from comment #14) > The least we should do is to turn this into a package.mask entry and get > upper bound "<dev-libs/expat-2.6" out of the testing dev-lang/python > ebuilds. That or just dropping the upper bound, depending on how > pessimistic you want to go. You would lose nothing, it only takes seconds > to make that change, and everyone would get choice back. IMO it's unfair to treat "the dev-lang/python maintainers wish to make sure the software has passing tests" like some attempt at suppressing user choice. Both sides here are trying to do the best they can for the sake of users, and "testsuites pass" isn't a particularly hot take. The fact that python is a core system component and you cannot upgrade expat by depcleaning python isn't really the fault of python. In fact this is actually an argument in favor of the dev-lang/python maintainers doing exactly what they are doing... it is a very core package so it would be quite bad for it to break, hence passing testsuites are incredibly important. I'm sure it is possible to ask for an update from upper bounds to masking with the benefit of additional information without seemingly implying some kind of bad faith (?) and especially given that even though upstream cpython appears to NOT like your suggested testsuite fix, a cpython developer has proposed an alternative fix that is also testsuite-only, indicating that the package should be fine at runtime.
Yeah, I'm OK with a request to change from < dep to a mask. It would just be nice to have that request without a possible insinuation that we did something wrong. I won't apologise for taking test failures in a critical system component seriously, and I don't expect mgorny to either. As an aside, I would recommend in future that Expat releases are tested with a -e @world or so with test suites enabled for some critical packages.
(In reply to Sam James from comment #16) > Yeah, I'm OK with a request to change from < dep to a mask. It would just be > nice to have that request without a possible insinuation that we did > something wrong. > > I won't apologise for taking test failures in a critical system component > seriously, and I don't expect mgorny to either. Thanks. Wrong is subjective and political here: it's becoming more and more clear that (as I understand, correct me) your focus is keeping even testing rocksolid while mine is unblocking security fixes and treating testing as something that is different from stable — sure not a playground — but different from stable for good reasons; both of these motivations are valid, understandable, noble in general — I believe we agree on that. My critique is not about you, it's been about the implementation and process, about not being informed beyond "I'm going to investigate later today" about steps taken in Gentoo, more even about not getting/having a voice in this, about the practical end-user consequences, about commit 93480b9b09b2a0c0adfed10a1c9e90c1e38a9040, and this not being the first time. I believe I do understand your side, Sam — I'm not expecting any apology from you —, I hope you also understand mine. That's likely all I want to say on public record on this topic.
You do have a voice -- it's just that your request for moving to a mask got mixed up with the suggestion that it shouldn't really have been done at all, and I don't think anything done w/ cpython was really wrong here. It's just a tricky situation because both are critical system components, I guess. Normally, we wouldn't hurry and/or it wouldn't matter so much. I don't think mgorny was aware or expecting you to be responsive/able to investigate it quickly from an upstream perspective to let us know if it was harmful - what happened here is pretty normal, the only odd thing here is that you're also involved upstream, and hence have an interest in getting it resolved quickly. If you want it changed to a mask, I think we can do that. I agree it makes sense.
commit a7613a5e5989b87bf799b0718fe0af1ad16faf03 Author: Michał Górny <mgorny@gentoo.org> Date: Sun Feb 11 15:14:13 2024 +0100 dev-lang/python: Revert dev-python/expat adjustment in 2.7.18_p16 Signed-off-by: Michał Górny <mgorny@gentoo.org> commit 7818ed782b61a64f5fb34ba5499caecc1978a3f8 Author: Michał Górny <mgorny@gentoo.org> Date: Sun Feb 11 15:13:23 2024 +0100 dev-lang/python: Bump to 3.8.18_p2 Signed-off-by: Michał Górny <mgorny@gentoo.org> commit 389e8e9fdede4600442eb870cb6c5f2e65054ddf Author: Michał Górny <mgorny@gentoo.org> Date: Sun Feb 11 15:08:26 2024 +0100 dev-lang/python: Bump to 3.9.18_p2 Signed-off-by: Michał Górny <mgorny@gentoo.org> commit 9429ff64f1320d3b755e2d1f96408ee1838d75eb Author: Michał Górny <mgorny@gentoo.org> Date: Sun Feb 11 14:59:31 2024 +0100 dev-lang/python: Bump to 3.10.13_p3 Signed-off-by: Michał Górny <mgorny@gentoo.org> commit 065635b9cd073a11bcf3fcd8ae1ca364d9e093c5 Author: Michał Górny <mgorny@gentoo.org> Date: Sun Feb 11 14:55:56 2024 +0100 dev-lang/python: Bump to 3.11.8_p1 Signed-off-by: Michał Górny <mgorny@gentoo.org> commit 7915a689a192ca0db73c6f0164586efc36bbb5c1 Author: Michał Górny <mgorny@gentoo.org> Date: Sun Feb 11 14:14:32 2024 +0100 dev-lang/python: Bump to 3.12.2_p1 Signed-off-by: Michał Górny <mgorny@gentoo.org>
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e96f72ba3ca504735d231cf4a07fc091a9bff75d commit e96f72ba3ca504735d231cf4a07fc091a9bff75d Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2024-02-11 15:55:09 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2024-02-11 15:55:48 +0000 dev-lang/python: Bump to 3.13.0_alpha3_p1 Bug: https://bugs.gentoo.org/923951 Signed-off-by: Michał Górny <mgorny@gentoo.org> dev-lang/python/Manifest | 1 + dev-lang/python/python-3.13.0_alpha3_p1.ebuild | 533 +++++++++++++++++++++++++ 2 files changed, 534 insertions(+)
Thanks for unblocking Expat 2.6.0 for Gentoo testing.
