While hal-0.5.7.1-r5 is fairly new, this is the version that fixes any and all outstanding reported bugs. It's the version targeted for the 2007.0 release media.
x86 stable, but still masked.
ppc64 stable
The <sys-apps/pciutils-2.2.4 'solution' for Bug 161057 has caused Bug 166812. While pciutils-2.2.4 is stable on sparc at least, this is a no-go there.
Might i add that >=pciutils-2.2.4 is required on sparc for libkudzu-1.2.57.1 for sanitized headers (gcc4).
While the solution isn't optimal, does this block this version of hal going stable? We only use libkudzu for the release materials, which won't have hal on them, anyway, where hal is used on installed systems, which shouldn't have libkudzu anyway. Even the GRP builds will pull in just pciutils 2.2.3 on sparc. I should also mention that this version of hal will actually *build* in a GRP set since it doesn't die when it cannot find a configured kernel. While a proper solution will need to come about shortly, is this something that should stop us from marking this version stable and blocking the release snapshot further?
OK. After talking with gustavoz, we're going to test this on sparc via the release snapshot. This means it won't be changed in the tree until we've verified nothing really crappy happens. Everybody else is free to continue with the stabilization, as this should only affect sparc, being the only arch with pciutils 2.2.4 stable.
x86 stable but usb pendrive is identified as harddisk not as removable media. this is ok?
sparc stable, let's just hope not many people USE=zlib which don't know how to use package.keywords.
(In reply to comment #8) > sparc stable, let's just hope not many people USE=zlib which don't know how to > use package.keywords. zlib is in make.defaults everywhere; hal is a default use flag on all desktop profiles. Good luck with this folks; I've had about 10 people asking why hal ebuild dies since yesterday on IRC. All of that because apparently we desperately need to save ~50KB of disk space for PCI IDs. So, now go re-read all the fuzz on Bug 120088, and then go read the pciutils ebuild. It has exactly *one* use flag; php has ~140 of them. I'd urge releng to package.use.mask the damned flag instead of letting vapier break users just because he can. :=(
Stable for HPPA.
Please either unstable or remove the stupid check. Unknown pciids are much less of a problem than every desktop gentoo system everywhere failing to emerge.
(In reply to comment #11) > Please either unstable or remove the stupid check. Eh? The solution has been mentioned above - drop the stupid zlib flag from *pciutils*, not the check that prevents hal from bombing out on compile. Here you go - this happens when you drop the "stupid" check in hal: <snip> configure: error: cannot find pci.ids. Use --with-hwdata to specify location </snip>
That doesn't change the fact that this should not have been stabilized until the problem is fixed, however it needs to be fixed.
(In reply to comment #13) > That doesn't change the fact that this should not have been stabilized until > the problem is fixed, however it needs to be fixed. So you want all hal ebuilds marked as unstable? Because none of them compiles with zlib-enabled pciutils; this one at least doesn't bomb out on compile. You can try and go complain to QA, good luck with that, I got banned from #gentoo-qa when asking to get vapier's pciutils junk reverted. :P
Feel free to contact QA for advice.
When installing hal-0.5.7.1-r5 it does not install the hal_unmount shell script, but the 90-hal.rules udev rule still refers to this.
(In reply to comment #15) > Feel free to contact QA for advice. One week later, the ebuild still dies on default desktop profiles' use flags... What kind of advice do you mean? You've been given a solution (heck, two even - either package.use.mask the flag or nuke it altogether plus revbump the thing) that would fix this in about 5 seconds and would harm noone. Yaaawn.
(In reply to comment #7) > x86 stable > but usb pendrive is identified as harddisk not as removable media. > this is ok? I (x86 user) have the same problem. my usb-flash-disk is identified as harddisk partition. But not allways. In rare situations as removable media. I can send output of lshal here (in 24h, both versions with hotplug true/false) if someone want it.
I've spoken with Doug about this and we've figured out that it won't be easy to make hal work with the USE=zlib pciutils. Now, what solutions do we have? - Add PCI_FILL_CLASS support to pciutils 2.2.3 (bug #160315 which is still only a temporary solution) and allow newer libkudzu to use this revision of pciutils - Remove the zlib USE flag from pciutils - package.use.mask zlib for pciutils in the profiles that also have hal - Patch hal to work with compressed pci.ids files Anything else? Which one should we do? Who is planning on doing it?
(In reply to comment #19) > Anything else? Which one should we do? Who is planning on doing it? add back amd64 once you decide
(In reply to comment #20) > (In reply to comment #19) > > Anything else? Which one should we do? Who is planning on doing it? > > add back amd64 once you decide > same for ppc
The less I can be involved in HAL, the happier I am. That being said. No version of HAL supports gzipped pci.ids or usb.ids files, not even their latest HEAD git. The current implementation of the reading of pci.ids in HAL is for support of the OLPC initiative, they're not too keen on changing this. I've mentioned it before. We are also the only distro to gzip our pci.ids file, no one else is doing this. It also breaks 11 packages (someone else quoted that number, so I have no solid facts). So the choice is yours (everyone CC'd on this bug), but I know I won't be writing the patch.
(In reply to comment #22) > It also breaks 11 packages (someone else quoted that > number, so I have no solid facts). app-misc/ddccontrol (Bug 163057) sys-apps/hwsetup (Bug 160937) sys-power/athcool (Bug 163563) sys-apps/vbetool (Bug 165064) app-laptop/smcinit (Bug 171022) sys-apps/kudzu (Bug 164329) sys-libs/libkudzu (no bug, same as kudzu) sys-apps/hal (Bug 161057 plus this one) sys-power/s2ram (not in the tree, Bug 128468) There's quite a bunch of other packages w/ DEPEND on sys-apps/pciutils, pretty likely some of them got broken as well. I can imagine it's more than 11 in total. OTOH, I have yet to see any real benefit from gzip-ing pci.ids; go figure...
ping on this bug. upstream is anti using libpci. upstreams attitude is stop gzipping a file that's 395kb and shrinking to 116kb... I have to agree with them. Gentoo won't be maintaining a custom patch to make this happen. We will be blocking sys-apps/pciutils with USE=zlib until pciutils fixes parsing from memory regions as was discussed by pjones and davidz with pciutils upstream. Once that support goes into pciutils, upstream will go to using libpci for handling pci.ids.
This has been going on for too long time now. HAL upstream wont support a zlib'ed file and this breaks for some users. Vapier: use.mask that flag for now till we have a better solution, then we can remove that. I am against small silly hacks, but we can't keep up with this...
/me waves @ qa - Hello, is there anobody out there?
do you seriously expect the QA team to do something ? they havent been effective since Halcy0n left it's on the shoulders of the maintainers to get issues resolved
(In reply to comment #27) > it's on the shoulders of the maintainers to get issues resolved Apparently; so why don't you finally package.use.mask the stupid zlib flag instead of causing unneeded breakage in the tree?
if you still arent up to speed after all of the irc conversations, then there isnt much point talking to you still
OK, let's summarize this: - this has been broken for over 5 months now - every single desktop profile is broken because both zlib and hal use flags are enabled by default on those profiles - now this borkage has moved to stable tree as sys-apps/pciutils-2.2.4-r3 got stabilized for Bug 180554, causing either hal emerge to die or hal to get broken on runtime - trying to resolve this with pciutils maintainer went nowhere despite many attempts and comments explaining that hal upstream does not support this pciutils 'feature' CCing devrel here since - breaking loads of users for the sake of saving ~200KB of disk space is plain stupid - breaking default profiles is a violation of our QA policy - maintainer apparently refuses to fix this breakage - QA failed to resolve this as well I'd really expect something more mature than this kindergarten-like attitude from vapier, oh well... :/
*** Bug 182206 has been marked as a duplicate of this bug. ***
feel free to make whatever noise you like since you're obviously unable to address the issue correctly ... i'll just do what i always do: fix it the right way
It is fixed in cvs now. package.use.mask'ed zlib.
thanks but no thanks; reverted i thought you retired
(In reply to comment #34) > thanks but no thanks; reverted So you actually like a broken tree? > i thought you retired I did. But I found this contribution a lot more valueable than other stuff right now, and since I still have commit access I wanted to contribute to our users.
this bug brings to full glory what is wrong about gentoo
(In reply to comment #36) > this bug brings to full glory what is wrong about gentoo I have to agree; this bugs shows us as a bunch of idiots unable to work with each other. It also shows that our QA doesn't work as it should. It shows that certain devs don't give a damn about users, just focusing blindly on non-existant 'right way' fixes. It also shows that certain devs are free to continue break policies no matter what. This is not the first case when vapier refuses to fix the pointless breakage he's introduced (see Bug 180440 for a recent example of this behaviour). devrel, please act.
(In reply to comment #37) > (see Bug 180440 for a recent example of this behaviour). i would like to add bug 138074 to this list; it has been closed on 2006-11-26 and is still not fixed. note that i don't want to say that it was vapiers fault that this bug occurred, but he closed without a valid reason.
Adding QA. QA, comment, please? I think this is yours.
(In reply to comment #39) > Adding QA. QA, comment, please? I think this is yours. QA says: for a temporary fix we suggest package.use.mask'ing on pciutils untill a better solution shows up.
I'm not sure this is a devrel issue, but devrel has been asked to intervene, so I am commenting. As I understand it, unless USE=zlib is package.use.mask'ed for pciutils, sys-apps/hal fails. Now, if this is a hard failure, and if other packages need hal, this presents a problem. I also see that IUSE=zlib is new with pciutils-2.2.4-r3, and that at least one arch (amd64) will not go stable on -2.2.4-r3 because of this zlib problem. SO: base-system people, what is the objection to package.use.mask zlib for pciutils until this gets sorted out? It does not really matter what package is misbehaving: We have an incompatibility here (as best as I can tell) with a simple (temporary) fix which does not really reduce functionality because USE=zlib is new with this version of pciutils and if I understand the problems with hal, cannot be used anyway. Certainly EVERY arch could package.use.mask it outside any base-system control, so why not do it globally? Especially as it seems to me that QA has final say in such issues. So, please either apply the temporary fix or explain why it's OK to leave hal broken. I understand that hal should change, but it's very hard to see standing on that as a justification for leaving things in a broken state if users are depending on a fix.
(In reply to comment #30) > OK, let's summarize this: > > - this has been broken for over 5 months now Nothing is "broken" at all. You can use hal perfectly with pciutils 2.2.4 with USE=-zlib on pciutils. This is a configuration issue. > - every single desktop profile is broken because both zlib and hal use flags > are enabled by default on those profiles Again, there is no breakage. Quit making it out to be something that it is not. > - now this borkage has moved to stable tree as sys-apps/pciutils-2.2.4-r3 got > stabilized for Bug 180554, causing either hal emerge to die or hal to get > broken on runtime This still doesn't make it broken. Users have a very simple and documented way around this "breakage" via normal portage mechanisms. > - trying to resolve this with pciutils maintainer went nowhere despite many > attempts and comments explaining that hal upstream does not support this > pciutils 'feature' How about you write a patch to hal, rather than continue beating this dead horse? > CCing devrel here since > > - breaking loads of users for the sake of saving ~200KB of disk space is plain > stupid There's nothing "broken" here, so this point is moot. > - breaking default profiles is a violation of our QA policy Again, there is nothing broken. Second, which policy? Show it to me. There isn't one. I know, because I made this same argument in the past and it was brought up then that there is no such policy. Whether there *should* be a policy in place or not is inconsequential here, since you're asking devrel to act on a non-existent policy. > - maintainer apparently refuses to fix this breakage No, the maintainer has refused to make changes to reduce the functionality in his package only to work around the inadequacies of another. > - QA failed to resolve this as well Well, I won't argue here. This issue should have been resolved some time ago. I refuse to call it a "bug" because it it not one. It is a known and incompatibility. Also, remember that configuration is the user's job. Sure, our goal is to provide sensible and usable defaults and the mechanisms to allow users to maintain their systems, but that doesn't mean going around changing everything just to keep users from using /etc/portage. > I'd really expect something more mature than this kindergarten-like attitude > from vapier, oh well... :/ Pot. Kettle. Say hi to each other boys.
why can't the hal ebuild provide it's own pci.ids if it detects a compressed one? This way everyone is happy.
(In reply to comment #42) > This still doesn't make it broken. Users have a very simple and documented way > around this "breakage" via normal portage mechanisms. Why should users work around anything? Sorry, the profiles are broken no matter how much you claim they are not. either hal emerge will fail on default profiles with default use flags, or users will get broken hal if they upgrade to now stable pciutils. You say this is no breakage? Great, makes sense... :P > How about you write a patch to hal, rather than continue beating this dead > horse? Why don't you write one? Upstream has explicitely stated they won't support this anytime soon, read this bug please. > No, the maintainer has refused to make changes to reduce the functionality in > his package only to work around the inadequacies of another. I fail to see how having the stuff non compressed reduces any functionality, sorry. Unless breaking hal is now a 'functionality'. (In reply to comment #43) > why can't the hal ebuild provide it's own pci.ids if it detects a compressed > one? This way everyone is happy. Hmmm, because that's what pciutils is for? Why should hal provide its own redundant copy, take care of its updating etc.?
hmm, what about USE masking zlib for pciutils for desktop profiles only - as users that use such a profile will most likely use hal too?
QA is working on this together with the maintainers. Wolf: yup, the ideal fix is a patch for HAL. Vapier has contacted HAL upstream afaik for such a fix, I'll wait for the response. KingTaco: I wont even comment on that. Everything you've said since Ferris comment is something we know. Right now the two only solutions is a) Fix HAL (and the other packages affected, if any.) or b) package.use.mask zlib for pciutils untill we can get solution a.
Considering I am HAL upstream, Mike is just pleading his case to the other HAL maintainers. I have stayed out of the discussion on the ML but Mike is getting the same answer he got from me months ago. HAL is designed to run on small embedded devices as well. By gzipping the pci.ids file you are saving approximately 200kb of disk space and the trade off is several MBs of dirty memory which on embedded systems is not ok. It just eats too much RAM that way. This is just not something we will support or write a patch for.
I would also like to point out that this doesn't just break HAL. It breaks all the other apps out there that use the pci.ids file as well. So you'll really need to take it up with those apps as well.
(In reply to comment #44) > (In reply to comment #43) > > why can't the hal ebuild provide it's own pci.ids if it detects a compressed > > one? This way everyone is happy. > > Hmmm, because that's what pciutils is for? Why should hal provide its own > redundant copy, take care of its updating etc.? > because hal doesn't actually use pciutils. it uses a file that was provided by pciutils. were it to use pciutils(and not it's data file) then it would link to libpci.{so,a} and there would be no problem. Thus the hal ebuild should provide it's own copy of pci.ids if it can't deal with pciutils version.
(In reply to comment #47) > HAL is designed to run on small embedded devices as well. By gzipping the > pci.ids file you are saving approximately 200kb of disk space and the trade off > is several MBs of dirty memory which on embedded systems is not ok. It just > eats too much RAM that way. This is just not something we will support or write > a patch for. Is it not realistic to make HAL accept either plaintext or .gz? That way, nobody is forced to do anything but we get compatibility all round.
But to summarize, David Zeuthen is the master of the HAL project. Here's his answer to Mike: Right, TBH I don't really see any point in using a gzipped pci.ids file at all; it's a trade-off where you trade disk space for dirty memory usage and as such it doesn't make sense on neither embedded nor desktop systems since you're much more interested in keeping memory use down on both such systems vs. saving around 300-400k of disk space... To which Mike said: perhaps, but that's the direction pciutils has gone ... so something like "if pci.ids.gz exists, then decompress it into a cache file and mmap that" would be the direction for hal to go ? To which the reply by David was: It would need to happen outside of HAL - at which point it becomes completely moot to even distribute pci.ids.gz in the first place. Frankly, I don't see any point in pci.ids.gz whatsoever - and HAL will continue to require an uncompressed pci.ids file until someone explains the benefit of pci.ids.gz. IOW, punt it to the distros... Since then Fedora/RedHat, SuSE, and Linux From Scratch have weighted in saying they don't build pciutils with zlib. A quick peek on Ubuntu shows they build pciutils for the 2.2.4 series with zlib, but then unzip the file. FreeBSD and Solaris (which are two other platforms supported by HAL) do not gzip their pci.ids file. FreeBSD even uses pciutils to fetch and update that file.
(In reply to comment #50) > > Is it not realistic to make HAL accept either plaintext or .gz? That way, > nobody is forced to do anything but we get compatibility all round. > Write a patch which does this and I will consider it.
(In reply to comment #50) > > Is it not realistic to make HAL accept either plaintext or .gz? That way, > nobody is forced to do anything but we get compatibility all round. > Also, you will have to patch the dozen or so other programs that are broken by gzipping the file.
> Thus the hal ebuild should provide it's own copy of pci.ids if it can't deal > with pciutils version. There are many packages with the same problem so it's a no-go unless you want to make a package with pci.ids only, which would be particularly funny. > Thus the hal ebuild should provide it's own copy of pci.ids if it can't deal > with pciutils version. It can't deal with gzipped pci.ids which is controlled by a use-flag... Why do we care so much about the space while we don't care about precious cpu ticks used to ungzip it everytime?! > Is it not realistic to make HAL accept either plaintext or .gz? That way, > nobody is forced to do anything but we get compatibility all round As far as I understood this bug's comments it's not realistic anytime soon.
QA has package.use.mask'ed zlib on pciutils again. I'd like to point out that if my commit is going to be reverted it'll be a violation of GLEP 48. Feel free to write a patch and make a correct solution for that. Talk with the QA team if you do so and make them remove the package.use.mask entry.
(In reply to comment #55) > QA has package.use.mask'ed zlib on pciutils again. > > I'd like to point out that if my commit is going to be reverted it'll be a > violation of GLEP 48. > > Feel free to write a patch and make a correct solution for that. Talk with the > QA team if you do so and make them remove the package.use.mask entry. > In order to cut off more argument or a commit/decommit cycle, I'll point out that under GLEP48, QA is claiming a violation of QA standards because for hal to run, pciutils must be built with USE=-zlib, but USE=zlib is the default. QA also claims that having the hal build die if pciutils is built with USE=zlib is also a QA violation, and that package.use.mask is the best short-term solution. No one claims that it is acceptable long term. Please note that QA have this authority under GLEP48. If this causes problems, there are two avenues for recourse: (1) Take it up with the QA team directly; (2) Appeal to council. Unilaterally overruling QA is not permitted under current policy, if I read GLEP48 correctly. If I do not read it correctly, please resolve the matter with QA. Thanks,
<snip> 18:46:48 <+CIA-24> wolf31o2 * gentoo-x86/profiles/default-linux/ (9 files in 5 dirs): Reverting Alexander's unwarranted masking of zlib USE on pciutils. Please read the comments before making any changes. </snip> Chris, is your penis bigger now or why are you escalating this thing which is none of your business even further? How much longer will this idiocy have to continue?
My question to wolf31o2 would be.. did you read any of the comments before making a change?
alpha is out of this bug, i'll try to get it stabilized
(In reply to comment #44) > Why should users work around anything? Sorry, the profiles are broken no matter > how much you claim they are not. either hal emerge will fail on default > profiles with default use flags, or users will get broken hal if they upgrade > to now stable pciutils. You say this is no breakage? Great, makes sense... :P Your opinion on this matter isn't very relevant. The profiles are *not* broken. Nothing is "broken" as everything is functioning *exactly* as it was intended. It just happens that those intended results cause an incompatibility. This is *not* the same as something being broken and I would *really* appreciate it if you would quit with your reposting of the exact same argument with no new proof. Sure the hal ebuild will fail. It was *made* to do that. Do you not understand that this is *expected* behavior? I know that to you everything is the end result, but we're developers here, and we would prefer to actually *solve* problems rather than work around them because a few users can't be bothered to use package.use locally. I'm sure you're going to use some argument about the number of users this affects or some other completely unquantifiable measurement, so let me just stop you right there to say that while I do agree that a solution should be made as quickly as possible, I also do not believe in putting the *wrong* solution in place just because a few people complain. > > How about you write a patch to hal, rather than continue beating this dead > > horse? > > Why don't you write one? Upstream has explicitely stated they won't support > this anytime soon, read this bug please. Why should I? I'm not the one complaining. I am perfectly happy with waiting for the HAL/pciutils guys to come to some sort of agreement. It is *you* that is impatient, pushy, and otherwise pestering people for this to get fixed *your way* and *now* so you should write up a patch. After all, this is open source. Complaining isn't contributing. Code is contributing. Contribute or kindly step aside and let the rest of us actually get some work done. > > No, the maintainer has refused to make changes to reduce the functionality in > > his package only to work around the inadequacies of another. > > I fail to see how having the stuff non compressed reduces any functionality, > sorry. Unless breaking hal is now a 'functionality'. Let's see. Now pciutils has the added functionality of being able to compress pci.ids and this functionality has been added to the ebuild. You're asking for the disabling/removing of that functionality, and you cannot see how this is a loss of functionality *in pciutils*? Upstream HAL has said that they would consider a patch to make hal actually use libpci.{a,so} rather than pci.ids directly. This is what I consider to be a proper solution.(In reply to comment #58) > My question to wolf31o2 would be.. did you read any of the comments before > making a change? I did read the comments here. I also have read GLEP48, which states that if there is disagreement within the QA team, which includes vapier and myself, that a majority rules. Well, nobody has ever asked me about this, other than my comments in this bug. At any rate, I'm asking the Council to decide on this, so leave it alone until the Council makes a decision, please.
(In reply to comment #57) > <snip> > 18:46:48 <+CIA-24> wolf31o2 * gentoo-x86/profiles/default-linux/ (9 files in 5 > dirs): Reverting Alexander's unwarranted masking of zlib USE on pciutils. > Please read the comments before making any changes. > </snip> While the rest of the comment isn't exactly what I'd support, eroyf's use-masking was perfectly valid and the best possible short-term solution we have. Now you, Wolf31o2, broke (yes, it *is* broken from a user's point of view and that's what matters) this stuff again for our users wich is, in my eyes, *totally* inacceptable. I sure hope DevRel takes appropriate action against both vapier and you.
(In reply to comment #60) > Your opinion on this matter isn't very relevant. The profiles are *not* > broken. Nothing is "broken" as everything is functioning *exactly* as it was > intended. yeah, so the default profile's intended behaviour is to *die* on emerge, or to cause runtime borkage. Way to go. > Why should I? I'm not the one complaining. And why should I? I haven't broken the profiles with a stupid flag that upstream will not support, and I've not reverted the fixes. > Let's see. Now pciutils has the added functionality of being able to compress > pci.ids and this functionality has been added to the ebuild. You're asking for > the disabling/removing of that functionality, and you cannot see how this is a > loss of functionality *in pciutils*? Sorry I'll repeat myself: Because we oh so totally need to save people from wasting ~200KB of diskspace that Gentoo council needs to deal with the crucial issue of foremost importancy!!!111! Wheeeeeeeeee, keep on moaning!
OK, enough. We have these issues: (1) hal will not work with defaults and current pciutils. There is a short term patch which base-system people will not accept for no rational reason that I can see; (2) Similarly, the hal people seem to be unwilling or unable to USE=zlib conditionally (either as an upstream patch or as Gentoo specific). Both of these positions strike me as standing on principle at the expense of user convenience, and so I am copying userrel; (3) There is apparently a split within QA as to handle this, with one faction asking to remove USE=zlib from pciutils so that hal will not die in the ebuild if pciutils was built with USE=zlib, and the other faction saying that it is fine for hal build to die because USE=zlib is somehow important to pciutils and its hal's problem anyway. QA, please resolve this internally. I note in passing that if each affected arch package.use.mask s zlib for pciutils individually, base-system has no say in the matter. I note also that each affected arch could follow amd64's lead and simply revert the offending version of pciutils to ~arch. (4) Finally, I note that I (as devrel representative on this bug) requested that disagreements be resolved within QA, not by more noise on this bug and that this request was ignored. If QA will come up with something sensible, however, this is the end of the matter. Be aware, however, that over 60 comments on how to handle USE=zlib does not make Gentoo look very professional (or even look like the developers know how to talk to each other). I can and will lock this bug if you don't take this discussion someplace more appropriate and just solve the problem. Regards, Ferris
(In reply to comment #60) > > Upstream HAL has said that they would consider a patch to make hal actually use > libpci.{a,so} rather than pci.ids directly. This is what I consider to be a > proper solution. No we didn't. libpci.{a,so} does not provide the necessary API to make this happen.
(In reply to comment #52) > Write a patch which does this and I will consider it. OK - I will do that. I'd ask that people stop taking action on this until I've had a chance to look at this - give me a couple of days. Of course, if someone else wants to jump in and try patching HAL instead of me, please go right ahead.
(In reply to comment #63) > (2) Similarly, the hal people seem to be unwilling or unable to USE=zlib > conditionally (either as an upstream patch or as Gentoo specific). > No. I will entertain a patch for this. Mike (vapier) understands exactly what must be done. A zlib wrapper for mmapping a file. That's it, it's been done before. That's the only patch that will be considered since it's the only way to provide HAL the necessary info it needs out of that file, and it also will touch the least amount of code.
(In reply to comment #64) > (In reply to comment #60) > > > > > Upstream HAL has said that they would consider a patch to make hal actually use > > libpci.{a,so} rather than pci.ids directly. This is what I consider to be a > > proper solution. > > No we didn't. libpci.{a,so} does not provide the necessary API to make this > happen. Excuse me, I read comment #52 wrong. You'll accept a patch that allows HAL to work with a gzipped pci.ids file.
Until this is resolved properly, why is it being left in a situation where users with a default install can't install what are generally considered to be key packages?
Actually, wolf did the right thing as QA members alone can't do whatever they like either. There must be a vote taken in the QA team before something like that happens especially as vapier is in the QA team as well ( I would bet he isn't after reading his comment "do you seriously expect the QA team to do something ?", but that's another thing ). That being said this issue has been escalated to the council and will be soon( ~3 days ) voted on. As you can also see dsd offered to write a patch for HAL so finally things are heading the right direction. Note that other "broken" packages just need a -lz in LDFLAGS.
Just to be 100% clear: I really think that zlib should have been p.use.masked a long time ago and vapier is unnecessarily uncompromising here, but hopefully council will make The Right(tm) decision or dsd will come up with a patch even before that.
(In reply to comment #69) > Actually, wolf did the right thing as QA members alone can't do whatever they > like either. The right thing is not to leave end users screwed over whilst people engage in a silly pissing contest. Until upstream packages are fixed, the tree should be left in a state that doesn't prevent users with a default configuration from installing key packages. It shouldn't take a QA team vote or the Council to do that.
OK so Chris said it's just a configuration issue. But I thought there was an assumption, if not policy, that the default configuration should work. package.use.mask is one way to make it work, but maybe a bit extreme, as it makes the flag kinda hard to see and people who want to use it have to unmask it. But leaving it alone, and even stabilizing it is also bad (I wouldn't mind if it broke ~arch users as they should expect it and know better how to work around configuration issues). So what about a compromise? echo "sys-apps/pciutils -zlib" >> /usr/portage/profiles/base/package.use - profile defaults will work with no problem - flag will still be visible and easily enabled, even explicit USE=zlib in make.conf will override this - breakage will only occur to people who set stuff above profile defaults, and those should be expected to solve configuration issues
amd64,ppc, you still have to do hal. The zlib thing doesn't block this stabilization. It should block the pciutils stabilization you did on bug 180554 alpha stable, thanks Tobias
(In reply to comment #73) > amd64,ppc, you still have to do hal. amd64 doesn't. we're going to wait until the issue is resolved before doing this one.
I didn't catch that -r5 aborts if pciutils was built with the zlib flag. I'm ok with -r5 going stable on amd64 once someone tests.
(In reply to comment #72) > So what about a compromise? > > echo "sys-apps/pciutils -zlib" >> /usr/portage/profiles/base/package.use > > - profile defaults will work with no problem > - flag will still be visible and easily enabled, even explicit USE=zlib in > make.conf will override this > - breakage will only occur to people who set stuff above profile defaults, > and those should be expected to solve configuration issues It will also be overrided by any USE=zlib in make.defaults in any profile that inherits from base. So for this to work properly the package.use must be added at every level where USE=zlib is set in the profiles...
(In reply to comment #75) > I didn't catch that -r5 aborts if pciutils was built with the zlib flag. I'm > ok with -r5 going stable on amd64 once someone tests. Well is that really needed when you have 0.5.9-r1 already stable? :) and ppc too...(In reply to comment #76) > (In reply to comment #72) > It will also be overrided by any USE=zlib in make.defaults in any profile that > inherits from base. So for this to work properly the package.use must be added > at every level where USE=zlib is set in the profiles... Thanks for correcting me. I was testing it with package.use in x86/2007.0/desktop profile and just assumed it would do the same in base.
Simple approach, don't gzip the damn pciutils thingie. Especially when it not only brakes hal, but also the many other apps mentioned above. This causes more trouble than doing any good, not to mention the time people have to spend in a) posting here to get this sorted out and b) wondering why their emerge suddenly "brakes". I wonder if patching hal is the right thing since the gzipped pciutils id's will still cause troubles with other packages then.
The gzipped pci.ids file does *not* break other packages. Those packages simply need to link against zlib. Look at the actual bug reports rather than regurgitating falsehoods written by people not in the know. I am trying to work with the maintainers now to fix those other packages. Currently, *only* HAL directly uses the pci.ids file. Everything else I have tested works fine once you link with zlib. Also, we really don't need any further comments from the peanut gallery. This is being worked on and will be fixed. Remember, this doesn't *break* anything. Feel free to think otherwise. That's why we're all granted the ability to formulate our own opinions. =]
(In reply to comment #79) > This is being worked on and will be fixed. Until then, why not make the one small, simple change required to prevent users using a default install from being affected by the issue?
Simple, because if they made that small change, then vapier and wolf3o12's e-peen would be smaller, and they can't have that. That said, even though wolf isn't on the bug anymore (or he is on it via one of the aliases that are on here) I fail to comprehend how gzipping a file is *adding functionality*. As you so succinctly put it. pciutils upstream won't give a stable api, and hal upstream fails to see the point of saving 200KB (You *did* read the comment here that Cardoe posted from David Z's mail right?) so they won't support it. The ebuild was written to die, because after numerous requests to have the "functionality" removed from pciutils as it would break things and vapier's steadfast refusal to do so because he needs to save 200KB for whatever reason. I fail to see the added "functionality" of gzipping the file, but if thats what you want to claim, hey, you are council and you are the almighty, so more power to ya. Or less, once the next vote comes around.
(In reply to comment #81) > The ebuild was written to die, because after numerous requests to have the > "functionality" removed from pciutils as it would break things and vapier's > steadfast refusal to do so because he needs to save 200KB for whatever reason. Not to mention the runtime borkage which has been completely ignored. sys-apps/pciutils should *refuse* to emerge w/ USE=zlib (or ignore USE=zlib) whenever sys-apps/hal is installed, otherwise you get hal broken at runtime. But - certain people will apparently still claim that it's working "*exactly* as it was intended." Sigh. :/
(In reply to comment #81) > and hal upstream fails to see the point of saving 200KB (You > *did* read the comment here that Cardoe posted from David Z's mail right?) so > they won't support it. Are they considering embedded systems where 200KB might matter. Although that's still quite small savings even for an embedded system. Depends on the device and application. On a cell phone, 200kb surely would matter. > vapier's > steadfast refusal to do so because he needs to save 200KB for whatever reason. > > I fail to see the added "functionality" of gzipping the file, Another assumed benefit aside from reduced footprint on embedded systems would be to reduce bandwidth. We are downloading this file not monthly right? Some still live in bandwidth limited areas. So if we are forcing a download on them monthly. The smaller the better. Again just assumed reasons. I have a hard time believing both Chris and Mike would take a stance on something purely based on opinion. Surely possible. Unfortunately I bet the technical reasons as to why have been disclosed in IRC. Likely beat to death. I would imagine/hope those being quiet are off dealing with problem, instead of talking about it. :) Maybe not. Either way I don't see Chris or Mike doing something against users best interest, based solely on their opinions or will.
(In reply to comment #83) > Are they considering embedded systems where 200KB might matter. Although that's > still quite small savings even for an embedded system. Depends on the device > and application. On a cell phone, 200kb surely would matter. Feel free to push this on embedded as much as you wish; this bug is about screwed desktop profiles. Presumably you won't use hal on embedded either, will you? > We are downloading this file not monthly right? We are not downloading anything unless you enable USE=network-cron ATM.
(In reply to comment #84) > > Feel free to push this on embedded as much as you wish; this bug is about > screwed desktop profiles. Just an assumption as to the motivation behind the actions. >Presumably you won't use hal on embedded either, will you? Some plan to yes, 6th comment http://www.gnome.org/press/releases/gmae-quotesheet.html "Codethink has worked hard on bringing core GNOME and Freedesktop technologies such as HAL and D-Bus into the embedded sphere and is very excited to be part of this initiative." More along those lines from 02 http://www.edn.com/article/CA240905.html Based on what HAL is and does, it very much has use and applications in embedded systems/devices. But correct that those reasons should not effect say desktops. > > We are downloading this file not monthly right? > > We are not downloading anything unless you enable USE=network-cron ATM. I don't have that use flag enabled. Yet I have seen cron email me when the downloads take place monthly. Kinds sucks since they all have the same default time. All my servers and etc go to download it at the same time, and I get a bunch of emails from each's cron.
(In reply to comment #85) > > We are not downloading anything unless you enable USE=network-cron ATM. > > I don't have that use flag enabled. Yet I have seen cron email me when the > downloads take place monthly. Kinds sucks since they all have the same default > time. All my servers and etc go to download it at the same time, and I get a > bunch of emails from each's cron. That's because portage won't nuke anything under CONFIG_PROTECT on unmerge (until 2.1.3_rcX at least); see the fine changelog :) Anyway, totally unrelated here.
(In reply to comment #77) > (In reply to comment #75) > > I didn't catch that -r5 aborts if pciutils was built with the zlib flag. I'm > > ok with -r5 going stable on amd64 once someone tests. > > Well is that really needed when you have 0.5.9-r1 already stable? :) good point.
I've posted a patch to the hal mailing list for review but their listserv is being slow
Would you mind attaching it here? I still haven't seen it show up on the ml. Thanks!
http://lists.freedesktop.org/archives/hal/2007-June/008831.html I also have another version locally which applies against 0.5.9.1, but let's wait for upstream review of the master version first.
I've done a little more research on this now. For anyone interested, the primary reason behind pciutils now installing a .gz file is because this is the way the upstream package does it *by default*. There's a lot of value in sticking to upstream defaults -- it means less Gentoo modifications to maintain, and you get respect (and hence support) from upstream developers. Feel free to disagree with Mike's execution all you will, I'd be doing exactly the same if I were in his position, based on technical decisions alone. (In reply to comment #64) > No we didn't. libpci.{a,so} does not provide the necessary API to make this > happen. I took this at face value when I wrote my patch, but I now realise the above to be incorrect - API is provided for this, and using this API would be a much more sensible approach than my patch. There's already a patch to do this: http://lists.freedesktop.org/archives/hal/2007-June/008830.html I will test it later today and provide feedback on the list. Also, I'd like to point out that Mike is the person who wrote the above patch, and he's the person who initiated the discussion on the HAL list. You remember, he's that evil guy who set out to break user desktops. He doesn't even use HAL! I know how irritating it can be when a package under my control (e.g. the kernel) changes, but other packages (e.g. external modules) are using the kernel in ways that they shouldn't and hence break -- then I'm left in the situation where I really really don't care about the package in question, but users face breakage unless I step up and fix it... So, don't go so hard on Mike :)
dsd, I appreciate the work you did, and I also understand that upstream does that by default, and that vapier initiated this topic - however, we've been trying to get it resolved for over 6 months, and it was just now brought up by him.... we tried to explain that upstream hal simply don't see the point in gzipping (honestly, even though upstream does, I still don't see the point, the savings aren't *that* great) A lot of this has to do with the frustration of dealing with it for 6 months and getting nowhere.
Well now that both the patches have been shot down due to issues with libpci. Where are we sitting. Let me provide a summary of what's been said by me (on this bug and in other places) and others have said. pjones and davidz discussed the issues with the pciutils maintainer that they had with using libpci previously. - stable API/ABI - dynamically linked library (current libpci comes static only) - use of exit() inside the library rather then providing a means to return error codes and check those error codes. - memory usage - API for id-by-name. Which appears to have been added in pcutils 2.2.x series. Due to these reasons, this made using libpci months, if not years ago impossible. And the change was made to using HALs own parser. Mike (vapier) discovered this old previous code and I believe based his patch on it. The other approach is to ungzip the file in memory and work with it there. That again is a less optimal solution since now we're wasting a ton of memory to load the file. Then a ton to uncompress it and then you need to keep that uncompressed file in memory for HAL to use. Right now it's a simple mmap(), which is clean and elegant. This is the approach Daniel (dsd) took with his patch. dsd dropped his patch in favor of vapier's patch on the HAL ML after discussion. Which was then turned down because of the issues I previously listed. So here we are back at square one. Now I understand the entire reason behind this bug and trying to patch HAL is because we're trying to follow upstream's wishes to a T and gzip the file as pciutils is doing. However, we're standing on a soap box clamoring that "UPSTREAM MUST BE FOLLOWED!" yet that involves changing another upstream and changing it in a negative and detrimental way which loses functionality. While claiming that not gzipping a little file is a "loss of functionality" (comments #42, #60 by Chris) Maybe rather then one-sidingly deciding that pciutils upstream MUST be right and their wishes MUST be followed. Maybe we can listen to what some other developers out there trying to use their software have highlighted as problems with their software. Since now we have Gentoo developers clamoring on the HAL ML for a change in HAL, maybe we can have the same interest by Gentoo developers to clamor on the pciutils ML for a change in their product to work how other developers need it to work. After all, a convenience library isn't much of a convenience when it's so broken that you can't use it. A simple Google search shows me that issues with using libpci have been discussed as far back as February 2005, yet pciutils people haven't done anything to rectify the situation.
I'm not convinced that we've exhausted all leads on the patching approaches that have been tried. At least, I will be continuing to work on this upstream and am optimistic that we'll reach a solution.
If you re-read what I said, I didn't say we've exhausted all leads. I suggested rather then standing on the "pciutils upstream is 100% right and they have no issues, HAL is where all the problems lie" platform. We explore the issues that the HAL maintainers have with pciutils and devote SOME of the resources we've been spending on HAL to conform to pciutils to make pciutils conform to some of what externals developers are looking for. Namely: - shared library support - stable API/ABI guarentees for 2.x versions. - memory usage profiling Or possibly backing of this silly idea of gzipping the file. I still haven't heard ANY reason for doing that other then "UPSTREAM DOES IT ZOMG WE MUST FOLLOW THEM!". I would love for our pciutils maintainer to talk to pciutils upstream and get a reason for why they have decided this is the best approach.
I would also like to point out that pciutils is not the holy grail of the pci.ids file. pci.ids upstream is http://pciids.sourceforge.net/ pciutils upstream is http://mj.ucw.cz/pciutils.shtml (the ebuild in Portage is wrong for the HOMEPAGE) They are not the same. pciutils is maintained by Martin Mares. While pci.ids is maintained by 10 people, Martin happens to be one of the 10. That being said, pci.ids is essentially being pulled in by the pciutils project's utilities and that's all. Who's to say pciutils isn't bastardizing the wishes of the pci.ids upstream. It's time we ask some questions of the pciutils upstream.
I have a possible solution that doesn't require patching either package. Of course, I haven't tested this. I'm just the idea man, here. What about splitting the pci.ids file into its own ebuild? Provided hal will function with a zlib-enabled pciutils (if pci.ids isn't gzipped) and pciutils will work with an uncompressed pci.ids file when built with USE=zlib, we could have a pci.ids package that provides the file, with a USE=gzip local flag to enable gzipping. Why USE=gzip? Because the pci.ids file/package doesn't need zlib to work. It's just data. It doesn't care if it is compressed or not. This will allow for USE="zlib gzip" pciutils pci-ids for those that want the gzipping and it wouldn't be forced on for those that don't. Also, gzip isn't in the default USE for anything, so users would get "magically" fixed and we can put this nasty bug behind us all. Now, I haven't slept all night and have been contemplating all kinds of things, so if any of this doesn't make any sense, please say so (preferably with a good reason and a better idea to replace it). Thanks
As a fairly long time observer of this bug. I have one question. 1. What, exactly, is the advantage of having pciutils read a gzipped pci.ids file? That is to say, what functionality does it add? If everyone can fathom that, then at least everyone will be on the same page about it, but for right now, I think that's the major point of disconnect. No offense to anyone, but all the alternative workarounds for this bug seem awfully convoluted to those of us who don't know or understand _why_ pciutils must have a gzipped pci.ids file (by default, given the USE flags in the profile). Can someone please enlighten?
Created attachment 122904 [details] sys-apps/pciids/pciids-20070623.ebuild (In reply to comment #97) > What about splitting the pci.ids file into its own ebuild? I did a joke about that in one of my previous comments, but now it really seems to be the best compromise.
(In reply to comment #98) > No offense to anyone, but all the alternative workarounds for this bug seem > awfully convoluted to those of us who don't know or understand _why_ pciutils > must have a gzipped pci.ids file (by default, given the USE flags in the > profile). "must" isn't the right word.. pciutils compiled with USE=zlib supports both compressed and uncompressed pci ids. But in this case, the choice between compressed or uncompressed is presented to the user via a USE flag. As for what functionality this adds: it allows the PCI IDS database to be compressed on disk. Here are 2 emails from Martin Mares (pciutils developer) on this subject, which explain it rather clearly. They were private emails but as they were CC'd to several developers I don't think there's any problem to post this publically: --------------------- Hello! > > Looking to any insight what made you guys decide to gzip the pci.ids > > file in pciutils 2.2.4 and higher. The hard disk savings is a mere 250kb > > while the amount of RAM necessary to handle the ungzip of the file and > > parsing in libpci far exceeds 200kb. It seems in the day and age of 1TB > > hard disks 200kb wouldn't be that much of a concern. It's also generally > > not well accepted in the embedded community to eat more RAM and CPU. First, I seriously doubt that the memory required to parse the compressed file is significantly higher than for parsing the uncompressed version. Second, if you are building an embedded system, feel free to store the file uncompressed. The current libpci is able to load both. > > The only rational I can see to making this change is the savings of that > > 250kb [1]. Yes, exactly that. Although the disks are large these days, it is still a good practice not to waste space. Also, the space savings really matter for example on installation media. > > Which backwards compat should easily trump 250kb of savings. > > > > This change also breaks with the default that has been established for > > many years on many different platforms. Namely, Linux, FreeBSD, NetBSD, > > OpenBSD, etc. On these platforms, the well established default is to use libpci for parsing the file. For every application using the library, everything is backward compatible. If you parse the file on your own, you can expect more problems than just compression, because the format of the file is evolving. --------------------- Hello! > > Considering you can simply parse the file once you've mmap() the file > > rather then having to load the file into memory and ungzip it then parse > > through that, yes gzipped the file does take more memory. However, to mmap the file you also need the memory. > > Especially > > that your argument for gzipping the file is to conserve space yet disks > > are large. Most systems have significantly less RAM then they do hard > > disk space. > > This means the ratio of hard disk space conserved by > > compressing it to the ratio of RAM used by having to uncompress it means > > that based on percentages, This does not make sense, the space overhead of decompression is an additive constant, not a multiplicative constant. At best, this sort of argument can show that the space savings do not bring much gain, but certainly not that they are making things worse. > > HAL implements it's own id-to-name parser Which is their problem, not anybody else's. > > because previously the API they wanted didn't exist. Which API? As far as I know, the name resolving API is almost unchanged since the first release of libpci. Anyway, it makes much more sense to implement the missing bits than to reinvent the wheel and implement a new parser. > > libpci only comes available as a static library and not as a shared > > library, This is easily fixable, isn't it? > > Why not bzip2 the file. Well, the reasons are obvious: first, libbz2 is not as widely available as libz; second, bzip2 needs much more memory for decompression than gzip. > > Matthew Wilcox (in the previously linked mailing > > list post) showed that bzip2 provides an even greater amount of disk > > space. -EINVAL. There are lots of compression methods which are even more efficient than bzip2. Is it a reason to use them? No. > > Given that the latest snapshot listed on the pciids.sf.net website gives > > the format of: > > > > # Syntax: > > # vendor vendor_name > > # device device_name <-- single tab > > # subvendor subdevice subsystem_name <-- two tabs Which is definitely not the whole file format, just of the section after this comment. New sections have been defined in recent versions of the pci.ids. > > And searching through BSD sources shows the format being constant for > > over a decade, and I can only go back so far because I can't find > > anything older, it seems to me unlikely this file will change without > > some serious need. The need has already arisen -- the stand-alone subsystem entries (section "S" in the current format; currently not present in the distributed file, but known to the parser for several versions and ready to be used). (Also, the really old versions of the format did not contain class entries.) > > Also I have to consider how infrequently this file changes.... When is > > the last time anyone has updated it. As you might have noticed if you looked more carefully, there are daily snapshots of the ID file available. The CVS at sf.net is used only for preparing releases of the pciutils, which really are not any frequent. So I really think that you are barking at the wrong tree. The only changes which are necessary is fixing HAL to use libpci properly (after possibly improving libpci if its API is insufficient) and making a shared version of libpci, which is already in my work queue anyway. --------------------- From a pciutils upstream perspective, It's also worth noting that the patch to add pci.ids.gz support to pciutils was both *trivial* and clean. When there is such little expense to add a potentially useful feature, it often makes sense to accept the patch.
*** Bug 183748 has been marked as a duplicate of this bug. ***
hal-0.5.9.1-r1 already stable on ppc, so removing us from cc
Looks like no more arches left, so could be closed... or renamed to reflect what this bug was really about...
What do you propose it be called? "pciutils vs hal" "Spanky vs QA" All seem appropriate.
