(This is right now just an idea quickly talked about with Robin and Brian, which I didn't get through the rest of QA yet, but I guess it might be worth to track here anyway). Basically, it would seem like a good feature if we could wire the basic QA check for visibility (i.e. dependency properness) of the tree in the process of moving from CVS to a user-visible rsync snapshot; depending on the results of the check, we might allow the sync to go through if there are just a few packages broken (say, up to 20, even though that might be too high), or stop if there are too many packages broken, to avoid having the users deal with that. Additionally, the full report of QA issues should be notified to the QA team, through gentoo-qa mailing list, optimally. (I'd personally like to have a status.net real-time interface as well but that's beside the point here; and if that doesn't sound too hard, the package maintainers' and the last committer to the package, but let's not shoot for the sky right away and let's see to get those points above implemented before anything else.)
Any thoughts on what format is best for outputting the results in, including archival? It supports both pickles and xml- pickles in this case are my preference as long as the feed is kept internal. Same feed can be pushed into packages.g.o if someone had time for example... Either way, it's doable, although we should visit this after I finish updating it for EAPI4- REQUEST_USE introduces some potentials that have to be scanned for. Ignoring eapi4, this could be done inside of a day- just need to write a consumer for the result stream (or a new reporter) that contains the algo for deciding whether or not to release the tree.
The mail output could easily be something like this: Results of the on-sync pcheck: The following packages have broken dependencies: dev-ruby/foo (missing dev-ruby/bar-0.1.3-r4) Additionally there are the following problems: net-misc/asdfg (missing IUSE) blahblahblah Really just need a quick list to ensure we know where the problems are, we can get to the details as needed.
Renamed summary, I'm not worried about getting this done *before* the service moves hosts, if that is what you meant. As a matter of fact, it should only be done after the service moves.
Sorry, the moves here wasn't intended as the hosts', but rather the "cvs->rsync" process itself. It was intended after your restructure.
The idea seems good. Do not abuse the frequency of this QA check because we will end up ignoring the email reports. I've seen problems with broken deptree on exotic arches so it might worth doing the QA check only for x86/amd64 at first.
I think it is important to receive status of broken deptree across _all_ non-dev arches. i.e. the ones that repoman wouldn't let you break anyway. That stuff has to arrive as soon as possible. A full QA sweep is probably better done once a day rather than each time, likely.
Often, as it is with Gentoo, we get hung up with the little details instead of incremental improvement... Please don't get over-concerned with the little details. Overall, I'm in support of this idea and it will happen someway/how. What I'd like to see in the end is a qa.g.o webpage with all of our little "once per day scripts that get ran" Such as: - cannot fetch (http://dev.gentoo.org/distfile-mirroring/) - broken manifests that persist over 2 sync periods (one hour total) - not yet implemented. - other crazy/good ideas But, I digress... (In reply to comment #6) > I think it is important to receive status of broken deptree across _all_ > non-dev arches. i.e. the ones that repoman wouldn't let you break anyway. That > stuff has to arrive as soon as possible. A seperate ML is one solution. A webpage is another solution. Not concerned for now. > A full QA sweep is probably better done once a day rather than each time, > likely. Agreed. We'd probably eventually want to develop "tiers" of errors. Tier 1 errors BLOCK the generation of rsync tree and all the rsync nodes stay on the previous sync that worked. We do this already if egencache fails (users get upset if the metadata doesn't work) and soon for eclass-fails-to-source (bug 239266). Tier 2 could be "x86/amd64 deptree" and decide what action to take. Tier 3 could be "non main arches deptree" .... etc. BUT, don't make this idea block icremental improvement :)
(In reply to comment #7) > Often, as it is with Gentoo, we get hung up with the little details instead of > incremental improvement... > > Please don't get over-concerned with the little details. Overall, I'm in > support of this idea and it will happen someway/how. What I'd like to see in > the end is a qa.g.o webpage with all of our little "once per day scripts that > get ran" Such as: > > - cannot fetch (http://dev.gentoo.org/distfile-mirroring/) > - broken manifests that persist over 2 sync periods (one hour total) - not yet > implemented. > - other crazy/good ideas Those are easy enough; I'll work on writing out better docs for people looking to implement custom checks in pcheck. For the tree it'll be working against, it'll have the pregenned cache, correct?
(In reply to comment #8) > For the tree it'll be working against, it'll have the pregenned cache, correct? You get to pick where to run some tests. As of right now, the steps in cvs->rsync generation are as follows: # 1) Update rsync checkouts or exports from cvs.g.o::mastermirror-staging # 1b) rsync EXPORTS/gentoo-x86 to STAGEDIR # pre2) source (bash -n) eclasses to check for syntax errors # 2) generate metadata (egencache) # 3) place dtd info in STAGEDIR # 4) place glsa's in STAGEDIR # 5) place news in STAGEDIR # 6) place herds.xml in STAGEDIR # 7) rsync from STAGEDIR to FINALDIR where if you call exit BEFORE step #7, all of the rsync nodes will be "stuck" on the last run that passed. We currently exit on step pre2 and 2 if they fail. Obviously, the most complete tree being after step #6 in STAGEDIR.
FYI: migration completed to new host. This bug will be stalled until something is presented to infra.
examples of some things we are logging but not doing anything about: regen-run-20110208-2035.log * Digest verification failed: * /var/tmp/gmirror/gentoo-x86-stage/app-arch/lzop/lzop-1.03.ebuild * Reason: Filesize does not match recorded size * Got: 1050 * Expected: 1055 regen-run-20110210-1335.log * Missing digest for '/var/tmp/gmirror/gentoo-x86-stage/sys-kernel/git-sources/git-sources-2.6.38_rc4-r1.ebuild' Error processing sys-kernel/git-sources-2.6.38_rc4-r1, continuing...
(In reply to comment #11) > examples of some things we are logging but not doing anything about: > > regen-run-20110208-2035.log > * Digest verification failed: > * /var/tmp/gmirror/gentoo-x86-stage/app-arch/lzop/lzop-1.03.ebuild > * Reason: Filesize does not match recorded size > * Got: 1050 > * Expected: 1055 > > regen-run-20110210-1335.log > * Missing digest for > '/var/tmp/gmirror/gentoo-x86-stage/sys-kernel/git-sources/git-sources-2.6.38_rc4-r1.ebuild' > Error processing sys-kernel/git-sources-2.6.38_rc4-r1, continuing... All of that is flagged by pcheck actually; yell at me within the week if I've not circled around and handed over an example config please (working on eapi4 crap atm).
(In reply to comment #12) > (In reply to comment #11) > > examples of some things we are logging but not doing anything about: > > > > regen-run-20110208-2035.log > > * Digest verification failed: > > * /var/tmp/gmirror/gentoo-x86-stage/app-arch/lzop/lzop-1.03.ebuild > > * Reason: Filesize does not match recorded size > > * Got: 1050 > > * Expected: 1055 > > > > regen-run-20110210-1335.log > > * Missing digest for > > '/var/tmp/gmirror/gentoo-x86-stage/sys-kernel/git-sources/git-sources-2.6.38_rc4-r1.ebuild' > > Error processing sys-kernel/git-sources-2.6.38_rc4-r1, continuing... > > All of that is flagged by pcheck actually; yell at me within the week if I've > not circled around and handed over an example config please (working on eapi4 > crap atm). > Flagged is not that interesting to me. Afterall, it is *already* flagged (otherwise I wouldn't be able to tell you the above details). What you guys need to do is: a) decide on severity of issues b) *report* the flagged issues somehow (for QA team to implement) At least in my opinion anyway. Now, approaching 30 days of a stalled bug (comment #10), I'm going to reassign this to qa and remove infra. Let us know when you have something ready. Thanks.
(In reply to comment #13) > > All of that is flagged by pcheck actually; yell at me within the week if I've > > not circled around and handed over an example config please (working on eapi4 > > crap atm). > > Flagged is not that interesting to me. Afterall, it is *already* flagged > (otherwise I wouldn't be able to tell you the above details). What you guys > need to do is: > a) decide on severity of issues > b) *report* the flagged issues somehow (for QA team to implement) To be clear, when I say 'flagged', I mean a check identifies the fault- that result is then handed to a reporter that decides what the hell to do with it (suppress it, display it, or flip a bit saying "disable the release"). > Now, approaching 30 days of a stalled bug (comment #10), I'm going to reassign > this to qa and remove infra. Let us know when you have something ready. Busy to say the least; update on my timelines, basically I'm finishing eapi4 prior to looping back to this, thus the lack of action right now.
We do this nowadays.
