| Summary: | net-p2p/bitcoind-0.9.1 version bump | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Matt Whitlock <gentoo> |
| Component: | Current packages | Assignee: | Anthony Basile <blueness> |
| Status: | RESOLVED DUPLICATE | ||
| Severity: | normal | CC: | luke-jr+gentoobugs, proxy-maint |
| Priority: | Normal | Keywords: | EBUILD |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: | https://bugs.gentoo.org/show_bug.cgi?id=506184 | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
bitcoind-0.9.0.ebuild
bitcoind-0.9.0.ebuild |
||
Created attachment 373404 [details]
bitcoind-0.9.0.ebuild
Sorry. I missed one thing that I should have cleaned up. Here's the clean version.
@Luke-jr Its a completely new build system. What do you think of Matt's ebuild? We already have an ebuild for 0.9.0 in the overlay. This bug should probably depend on 502716 and 505320. (In reply to Luke-Jr from comment #3) > We already have an ebuild for 0.9.0 in the overlay. This bug should probably > depend on 502716 and 505320. I got around the need for bug 505320 by disabling Bitcoin's hardening. I figured users of Gentoo Hardened (of which I am not one) would already be building libmemenv.a with -fPIC, and their flags would already be configured to build bitcoind as PIE. @Luke-Jr: FWIW, where is this overlay you mentioned? Apparently I could have saved myself quite a bit of time. It would be nice if the ebuilds in the tree had einfo pointing to the overlay for those wanting the latest releases. (In reply to Matt Whitlock from comment #4) > (In reply to Luke-Jr from comment #3) > > We already have an ebuild for 0.9.0 in the overlay. This bug should probably > > depend on 502716 and 505320. > > I got around the need for bug 505320 by disabling Bitcoin's hardening. I > figured users of Gentoo Hardened (of which I am not one) would already be > building libmemenv.a with -fPIC, and their flags would already be configured > to build bitcoind as PIE. The idea upstream at least is that Bitcoin wallets should *always* be hardened. > @Luke-Jr: FWIW, where is this overlay you mentioned? Apparently I could have > saved myself quite a bit of time. It would be nice if the ebuilds in the > tree had einfo pointing to the overlay for those wanting the latest releases. https://gitorious.org/bitcoin/gentoo/ layman -a bitcoin (In reply to Luke-Jr from comment #5) > The idea upstream at least is that Bitcoin wallets should *always* be > hardened. I agree that hardening is a good idea, but if upstream intends for it to be mandatory, then why are there separate --enable-wallet and --enable-hardening configure flags? > layman -a bitcoin Thanks. (In reply to Matt Whitlock from comment #6) > (In reply to Luke-Jr from comment #5) > > The idea upstream at least is that Bitcoin wallets should *always* be > > hardened. > > I agree that hardening is a good idea, but if upstream intends for it to be > mandatory, then why are there separate --enable-wallet and > --enable-hardening configure flags? Obviously it's debatable. If the leveldb compat gets resolved first, we can disable (just) PIE by default... I should add a wallet USE flag... (In reply to Luke-Jr from comment #5) > https://gitorious.org/bitcoin/gentoo/ I'm taking a look at bitcoind-0.9.0.ebuild and have noticed a couple of things. 1. [You just addressed this point.] You give the user no option of whether to enable the wallet feature. One of the best improvements in 0.9.0 is that the wallet is now optional. It would be nice to expose that option to the user as a USE flag, which I did in my ebuild. 2. Bitcoind works fine with the latest LevelDB in Portage. You don't need the versioned dependency. 3. Why use ./autogen.sh when Portage has the autotools eclass with eautoreconf? 4. Should src_test call "emake check" as I did in my ebuild rather than directly invoking src/test/test_bitcoin, which seems more fragile? (In reply to Matt Whitlock from comment #8) > 2. Bitcoind works fine with the latest LevelDB in Portage. You don't need > the versioned dependency. bitcoind always needs this dependency versioned, since upstream (and Gentoo) LevelDB does not meet the standards required for consensus system software. We can bump the version to <=1.15, but 1.14+ break backward compatibility, so that needs to be addressed (or given an "okay to ignore"). > 3. Why use ./autogen.sh when Portage has the autotools eclass with > eautoreconf? Sounds like an okay change to me. Merge request? :) > 4. Should src_test call "emake check" as I did in my ebuild rather than > directly invoking src/test/test_bitcoin, which seems more fragile? Maybe. See if it works. :) (In reply to Luke-Jr from comment #9) > (In reply to Matt Whitlock from comment #8) > > 2. Bitcoind works fine with the latest LevelDB in Portage. You don't need > > the versioned dependency. > > bitcoind always needs this dependency versioned, since upstream (and Gentoo) > LevelDB does not meet the standards required for consensus system software. > We can bump the version to <=1.15, but 1.14+ break backward compatibility, > so that needs to be addressed (or given an "okay to ignore"). I had no problems upgrading my existing bitcoind installation to use LevelDB 1.15. Do you mean 1.14+ break *forward* compatibility? I wouldn't even downgrade LevelDB, but even if I did, so what? It's not like the wallet.dat uses LevelDB, and that file is the only thing that really matters. > > 3. Why use ./autogen.sh when Portage has the autotools eclass with > > eautoreconf? > > Sounds like an okay change to me. Merge request? :) Okay. > > 4. Should src_test call "emake check" as I did in my ebuild rather than > > directly invoking src/test/test_bitcoin, which seems more fragile? > > Maybe. See if it works. :) I already have. It would not have been in my ebuild if it didn't work. :P (In reply to Matt Whitlock from comment #10) > I wouldn't even downgrade LevelDB I meant, "I wouldn't *ever* downgrade..." (In reply to Matt Whitlock from comment #11) > (In reply to Matt Whitlock from comment #10) > > I wouldn't even downgrade LevelDB > > I meant, "I wouldn't *ever* downgrade..." Okay, but someone with Gentoo policy authority should make the decision whether to break downgrading. (In reply to Luke-Jr from comment #9) > Sounds like an okay change to me. Merge request? :) https://gitorious.org/bitcoin/gentoo/merge_requests/14 (In reply to Luke-Jr from comment #12) > Okay, but someone with Gentoo policy authority should make the decision > whether to break downgrading. I won't dispute that point, but I will add that presently Bitcoin breaks the *upgrade* path, as its presence holds back the system-installed LevelDB to 1.12. I wonder if LevelDB can be slotted... (In reply to Luke-Jr from comment #9) > upstream (and Gentoo) LevelDB does not meet the > standards required for consensus system software. Can you be more specific about what you mean by this? (In reply to Luke-Jr from comment #12) > (In reply to Matt Whitlock from comment #11) > > (In reply to Matt Whitlock from comment #10) > > > I wouldn't even downgrade LevelDB > > > > I meant, "I wouldn't *ever* downgrade..." > > Okay, but someone with Gentoo policy authority should make the decision > whether to break downgrading. Your overlay -> your policy authority, imo. I suppose exercising some caution seems prudent, however, given the possibility, however remote, that you might blow up somebody's life savings somehow... :) One possibility is to make a use-flag to control it -- this way you could default to the current behavior, but select a version-free dependency atom (or a <= atom, or whatever you think is appropriate) upon being set. One other note: instead of using an atom like '<=cat/foo-1.2.3', you might want to use the near-equivalent 'cat/foo !!>=cat/foo-1.2.4' combo, '|| ( ~cat/foo-1.2.3 <cat/foo-1.2.3 ), or, if you really just intend to pick up that particular version, simply i.e.: "~cat/foo-1.2.3". This way, your ebuild will tolerate future point revisions to leveldb without the requirement that your ebuild make some corresponding change. This will let you remain compatible with third-party overlays, which, following the dev-manual recommendation, will often revbump ebuilds they wish to override, and avoid the very real possibility that your ebuild prevents consumers from receiving some future GLSA update, possibly, even if you have updated your overlay to account for it (i.e.: they emerge --sync but don't think/want to layman -S). -gmt (In reply to Greg Turner from comment #15) > (In reply to Luke-Jr from comment #9) > > upstream (and Gentoo) LevelDB does not meet the > > standards required for consensus system software. > > Can you be more specific about what you mean by this? A bugfix to LevelDB could be a critical security problem for Bitcoin. Consensus systems require all nodes to implement the *exact* same rules, with no toleration for variance of any kind. Even if the behaviour is a bug, it *must* be enforced, or the entire system collapses. > (In reply to Luke-Jr from comment #12) > > (In reply to Matt Whitlock from comment #11) > > > (In reply to Matt Whitlock from comment #10) > > > > I wouldn't even downgrade LevelDB > > > > > > I meant, "I wouldn't *ever* downgrade..." > > > > Okay, but someone with Gentoo policy authority should make the decision > > whether to break downgrading. > > Your overlay -> your policy authority, imo. Yes, I was talking about the context of migrating the ebuild to the main tree. > This way, your ebuild will tolerate future point revisions to leveldb > without the requirement that your ebuild make some corresponding change. That's a problem, since those point revisions could fix a bug that must be maintained. (In reply to Luke-Jr from comment #16) > A bugfix to LevelDB could be a critical security problem for Bitcoin. > Consensus systems require all nodes to implement the *exact* same rules, > with no toleration for variance of any kind. Even if the behaviour is a bug, > it *must* be enforced, or the entire system collapses. In that case, may I humbly suggest that the Bitcoin ebuilds *not* use the system-installed LevelDB? (In reply to Matt Whitlock from comment #17) > (In reply to Luke-Jr from comment #16) > > A bugfix to LevelDB could be a critical security problem for Bitcoin. > > Consensus systems require all nodes to implement the *exact* same rules, > > with no toleration for variance of any kind. Even if the behaviour is a bug, > > it *must* be enforced, or the entire system collapses. > > In that case, may I humbly suggest that the Bitcoin ebuilds *not* use the > system-installed LevelDB? That's a possibility. We could add a system-leveldb USE flag for users who wish to benefit from the reduced RAM use... (In reply to Luke-Jr from comment #18) > (In reply to Matt Whitlock from comment #17) > > (In reply to Luke-Jr from comment #16) > > > A bugfix to LevelDB could be a critical security problem for Bitcoin. > > > Consensus systems require all nodes to implement the *exact* same rules, > > > with no toleration for variance of any kind. Even if the behaviour is a bug, > > > it *must* be enforced, or the entire system collapses. > > > > In that case, may I humbly suggest that the Bitcoin ebuilds *not* use the > > system-installed LevelDB? > > That's a possibility. We could add a system-leveldb USE flag for users who > wish to benefit from the reduced RAM use... That's good by me, but I'll point out that one only benefits from reduced RAM usage if one is running other applications that make use of LevelDB, in which case, why would one want those other applications using a possibly broken, possibly insecure version of LevelDB? Essentially, Bitcoin, for better or worse, is requiring the system to have a potentially buggy library and is prohibiting deployment of any fixes. That can't ever be a good thing. I personally don't believe the RAM usage argument holds water in this light. (In reply to Matt Whitlock from comment #19) > (In reply to Luke-Jr from comment #18) > > (In reply to Matt Whitlock from comment #17) > > > (In reply to Luke-Jr from comment #16) > > > > A bugfix to LevelDB could be a critical security problem for Bitcoin. > > > > Consensus systems require all nodes to implement the *exact* same rules, > > > > with no toleration for variance of any kind. Even if the behaviour is a bug, > > > > it *must* be enforced, or the entire system collapses. > > > > > > In that case, may I humbly suggest that the Bitcoin ebuilds *not* use the > > > system-installed LevelDB? > > > > That's a possibility. We could add a system-leveldb USE flag for users who > > wish to benefit from the reduced RAM use... > > That's good by me, but I'll point out that one only benefits from reduced > RAM usage if one is running other applications that make use of LevelDB, in > which case, why would one want those other applications using a possibly > broken, possibly insecure version of LevelDB? > > Essentially, Bitcoin, for better or worse, is requiring the system to have a > potentially buggy library and is prohibiting deployment of any fixes. That > can't ever be a good thing. I personally don't believe the RAM usage > argument holds water in this light. It's a theoretical issue right now. When/if it arises, there is obviously further discussion to be had on how to resolve it. (In reply to Luke-Jr from comment #20) > > Essentially, Bitcoin, for better or worse, is requiring the system to have a > > potentially buggy library and is prohibiting deployment of any fixes. That > > can't ever be a good thing. I personally don't believe the RAM usage > > argument holds water in this light. > > It's a theoretical issue right now. When/if it arises, there is obviously > further discussion to be had on how to resolve it. Are you saying nothing else in the tree depends on dev-libs/leveldb? Even if 2.12 has zero bugs, you can't in good faith make the argument that 2.12 is just as good as 2.15. So you're penalizing users for having Bitcoin installed. (In reply to Matt Whitlock from comment #21) > (In reply to Luke-Jr from comment #20) > > > Essentially, Bitcoin, for better or worse, is requiring the system to have a > > > potentially buggy library and is prohibiting deployment of any fixes. That > > > can't ever be a good thing. I personally don't believe the RAM usage > > > argument holds water in this light. > > > > It's a theoretical issue right now. When/if it arises, there is obviously > > further discussion to be had on how to resolve it. > > Are you saying nothing else in the tree depends on dev-libs/leveldb? Even if > 2.12 has zero bugs, you can't in good faith make the argument that 2.12 is > just as good as 2.15. So you're penalizing users for having Bitcoin > installed. A bundled library is strongly to be discouraged. I do not want to see a USE flag toggle between the system leveldb and bundled. (The only exception would be a precompiled library which is unavoidable, see spideroak-bin.) If there is a bug in either the system leveldb or the bundled source, then a patch should be sent to the appropriate upstream and we hold back stabilization until it is fixed. (In reply to Anthony Basile from comment #22) > A bundled library is strongly to be discouraged. I do not want to see a USE > flag toggle between the system leveldb and bundled. (The only exception > would be a precompiled library which is unavoidable, see spideroak-bin.) If > there is a bug in either the system leveldb or the bundled source, then a > patch should be sent to the appropriate upstream and we hold back > stabilization until it is fixed. If there is a bug in the system LevelDB, then Bitcoin is going to *require* the bug. It's not simply a matter of sending a patch upstream to fix it; the bug is part of the protocol, and hard changes to the protocol require consent and participation by a super-majority of the nodes in the network, so such changes are quite difficult to effect. Meanwhile, all other installed packages depending on LevelDB will have to live with the bug too. Like I said, having net-p2p/bitcoind or net-p2p/bitcoin-qt installed is effectively a commitment to whatever bugs are presently in LevelDB. Fixes will not be allowed, so nothing else in the system can get them either. To clarify: it depends on the bug. (In reply to Luke-Jr from comment #16) > (In reply to Greg Turner from comment #15) > > (In reply to Luke-Jr from comment #9) > > > upstream (and Gentoo) LevelDB does not meet the > > > standards required for consensus system software. > > > > Can you be more specific about what you mean by this? > > A bugfix to LevelDB could be a critical security problem for Bitcoin. > Consensus systems require all nodes to implement the *exact* same rules, > with no toleration for variance of any kind. Even if the behaviour is a bug, > it *must* be enforced, or the entire system collapses. Thanks for your clarification -- I didn't know about this sense of the term "consensus system". --- (In reply to Anthony Basile from comment #22) > A bundled library is strongly to be discouraged. I do not want to see a USE > flag toggle between the system leveldb and bundled. (The only exception > would be a precompiled library which is unavoidable, see spideroak-bin.) If > there is a bug in either the system leveldb or the bundled source, then a > patch should be sent to the appropriate upstream and we hold back > stabilization until it is fixed. Anthony I understand that this is Gentoo SOP but wouldn't you agree, given what Luke-Jr is saying above about consensus-systems ("A bugfix to LevelDB could be a critical security problem for Bitcoin"), that something has to give? The problem seems to one of too many "must not" dogmas to enforce simultaneously: Anthony: > A bundled library is strongly to be discouraged. I do not want to see a USE > flag toggle between the system leveldb and bundled [in the bitcoin ebuild]. Luke-Jr: > Consensus systems require all nodes to implement the *exact* same rules, > with no toleration for variance of any kind [therefore bitcoin must not > be allowed to dynamically link to a floating leveldb revision]. Matt Whitlock > Essentially, [by preventing installation of future leveldb versions, > the bitcoin overlay's] Bitcoin [ebuild] ... is prohibiting deployment > of any [future leveldb] fixes. That can't ever be a good thing. Any two of the above people can potentially have their way, but not all three. Something's got to give. My 2 satoshis: We are required, here, to decide the lesser of several evils: the "bundled libs are bad" rule, although important, seems like the least costly of the above design objectives to break. Gentoo already makes many exceptions to this rule (perhaps not always for good reasons but that is not the bug we're trying to fix here). > My 2 satoshis: We are required, here, to decide the lesser of several evils:
> the "bundled libs are bad" rule, although important, seems like the least
> costly of the above design objectives to break. Gentoo already makes many
> exceptions to this rule (perhaps not always for good reasons but that is not
> the bug we're trying to fix here).
At least this bundled library is distributed in source form, with a build system that takes responsibility for cleanly building it into the final binary. It's not like it's some proprietary-licensed, precompiled, binary blob.
It needs to be clarified again that the LevelDB bugfix/consensus issue is *not* a real-world issue today, and is not a blocker to getting 0.9.0 into the tree. The blocking issue right now is LevelDB's backward incompatibility between <1.14 and >=1.14. Probably the solution is to patch any <1.14 in the tree to tolerate the file rename in 1.14. (In reply to Luke-Jr from comment #27) > It needs to be clarified again that the LevelDB bugfix/consensus issue is > *not* a real-world issue today, and is not a blocker to getting 0.9.0 into > the tree. > > The blocking issue right now is LevelDB's backward incompatibility between > <1.14 and >=1.14. Probably the solution is to patch any <1.14 in the tree to > tolerate the file rename in 1.14. Ah, that I didn't realize. Given that bitcoin-qt and bitcoind seem to be the only consumers of levelDB(?), and given that it works that way already, it does seem a shame not to at least optionally continue to allow that library to be shared between them. It's starting to seem to me that it would make sense to consider a slotted leveldb ebuild. Even more extreme solutions exist -- for example, one could not only slot leveldb, but additionally create a slotted virtual/bitcoin-leveldb atom, which could act as a kind of "router" between categories of *coin ebuilds and their corresponding shared leveldb dependencies (kind of like virtual/mysql) That's overkill and not something I'd propose unless things got really complicated in the future -- but it illustrates that however wacky the network of ebuilds and corresponding constraints were to get in the future, there would probably be some reasonable way for bitcoin ebuilds to have their leveldb cake and consume it via shared libraries too, without breaking bitcoin's consensus requirements. To clarify, it's a forward incompatibility. Upgrading LevelDB works just fine. (In reply to Greg Turner from comment #25) > > Anthony I understand that this is Gentoo SOP but wouldn't you agree, given > what Luke-Jr is saying above about consensus-systems ("A bugfix to LevelDB > could be a critical security problem for Bitcoin"), that something has to > give? The problem is the word "bug". In the space of all possible bugs (if such a space can be defined) most should be fixed and would not affect the consensus-system. After talking with Luke-jr, was I think is meant here is a misimplementation of some algorithm or the like, but not just any old bug. (In reply to Luke-Jr from comment #27) > It needs to be clarified again that the LevelDB bugfix/consensus issue is > *not* a real-world issue today, and is not a blocker to getting 0.9.0 into > the tree. > > The blocking issue right now is LevelDB's backward incompatibility between > <1.14 and >=1.14. Probably the solution is to patch any <1.14 in the tree to > tolerate the file rename in 1.14. I would recommend adopting the patching solution. Let me know when the overlay ebuild is ready to go to the tree. 0.9.1 has been released (In reply to Patrick Lauer from comment #32) > 0.9.1 has been released Irrelevant for us, only deps were changed. *** This bug has been marked as a duplicate of bug 505096 *** |
Created attachment 373402 [details] bitcoind-0.9.0.ebuild Bitcoind 0.9.0 has switched its build system to autotools, which significantly simplifies the ebuild. Attached, please find a new ebuild for net-p2p/bitcoind-0.9.0. Note that Bitcoind now compiles cleanly against the latest version of dev-libs/leveldb, so the version restriction on that dependency has been removed. Also note that the Bitcoin devs have made it much easier to use the system-installed leveldb, so 0.8.2-sys_leveldb.patch is no longer necessary.