Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 473 - Gentoo Tinderbox
Summary: Gentoo Tinderbox
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Unspecified (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Gentoo Quality Assurance Team
URL:
Whiteboard:
Keywords:
: 240199 (view as bug list)
Depends on:
Blocks:
 
Reported: 2002-02-01 08:37 UTC by Karl Trygve Kalleberg (RETIRED)
Modified: 2015-01-14 05:50 UTC (History)
15 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Karl Trygve Kalleberg (RETIRED) gentoo-dev 2002-02-01 08:37:20 UTC
Currently planned features:
- recompile all new ebuilds upon commit
- compile with all combos of the ebuild's supported use settings
- generate a tbz2 and an rpm for each of
   - the ebuild settings
   - optimization profiles (AMD, PIII, P5, 386)
- generate bug reports directly into bugzilla when builds fail
Comment 1 Brad Laue (RETIRED) gentoo-dev 2003-03-12 19:11:44 UTC
This is an excellent idea for a lot of reasons.

First it will be an excellent way to verify the compilability of ebuilds,
and second it will, given a set of standard tests, verify the functionality of
portage itself.

A tinderbox designed to bootstrap and emerge system would also be highly
advisable. On several occasions portage has been blessed with bugs that
allowed either bootstrap or emerge system to finish, but left the filesystem
in a highly inconsistent state (all symlinks dead, broken man-pages, missing library files, etc).

I have an Athlon system I can put toward this if it's needed. Please let me
know if it's useful.
Comment 2 Stewart (RETIRED) gentoo-dev 2003-03-17 03:11:37 UTC
Since being educated on the concept of a tinderbox, I think it's a very excellent idea for Gentoo. With the growing pains Gentoo is sure to endure, it would be a great step to begin thorough testing.

I'll throw my workstations' spare CPU cycles at it if need be.
Comment 3 Karl Trygve Kalleberg (RETIRED) gentoo-dev 2003-05-03 08:37:25 UTC
It's just a matter of getting around to it, as usual....
Comment 4 Karl Trygve Kalleberg (RETIRED) gentoo-dev 2005-02-17 03:36:38 UTC
Okay, so an update on the situation is in order. I actually have a design for this, and a very rudimentary prototype for it running on my setup.

It is based on the idea of distributed tasks (using a DHT), where client tinderboxes can subscribe to one central point of failure, err, server, to obtain tasks.

I've also planned, but not implemented a facility for PKI, so that we may know the trust-level of the participating tinderbox clients.

All of this is publicly available in my source directory on my laptop for now:/
Comment 5 Anthon Pang 2005-05-21 11:41:16 UTC
So this would pull in updated packages (from world) and test builds?

Have you looked at John Keiser's Tinderbox 3?

http://www.johnkeiser.com/mozilla/tbox3.html
Comment 6 Ernestas Liubarskij 2006-02-05 05:39:28 UTC
It's a really nice idea. I can help to test it on a few pcs, just let me know when it will be ready.

"- generate a tbz2 and an rpm for each of
   - the ebuild settings
   - optimization profiles (AMD, PIII, P5, 386)"

Why is this needed?
Comment 7 solar (RETIRED) gentoo-dev 2006-05-29 07:46:01 UTC
Not sure about the rpm part. But I've got the basis of several other 
aspects touched on in this bug for several profiles and arches 
going at. http://tinderbox.x86.dev.gentoo.org/

Testing every single combo of every single USE= flag on every single commit 
simply wont work as you will run into blockers and the likes.
Plus you would need a pretty damn big cluster of build hosts for that.
Comment 8 Daniel Herzog 2006-12-13 09:02:53 UTC
One could generate a few sets of "average" USE/CFLAG combinations from the (currently non-existent?) gentoo-statistics project so not every single combination is checked.

This way, one would avoid silly cflag- and/or use-combinations
Comment 9 Jakub Moc (RETIRED) gentoo-dev 2007-12-26 10:44:53 UTC
karltk retired.
Comment 10 Mark Loeser (RETIRED) gentoo-dev 2008-10-06 18:23:33 UTC
*** Bug 240199 has been marked as a duplicate of this bug. ***
Comment 11 Julian Ospald 2014-03-09 12:42:36 UTC
QA should provide a tinderbox.
Comment 12 Sergey Popov gentoo-dev 2014-03-10 11:29:36 UTC
(In reply to Julian Ospald (hasufell) from comment #11)
> QA should provide a tinderbox.

Probably, but if you want it official - we need support from Infra as well
Comment 13 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2014-03-10 15:14:27 UTC
(In reply to Sergey Popov from comment #12)
> (In reply to Julian Ospald (hasufell) from comment #11)
> > QA should provide a tinderbox.
> 
> Probably, but if you want it official - we need support from Infra as well

The last tinderbox (run by solar) wasn't 'official.' I don't want Infra support to be some sort of blocker here (we have other commitments to handle at the moment.) Get some hardware, play around, get it working. Infra can move to own if (if we even want to own it) once it is running and proven somewhat stable.

-A
Comment 14 Chris Reffett (RETIRED) gentoo-dev Security 2014-03-12 23:16:51 UTC
Depends on whether you consider build failures and stuff like that to be QA's problem. Further depends on getting hardware--I for one don't exactly have boxes lying around which have nothing better to do than build packages all day. We can discuss this at the next meeting, but I wouldn't suggest getting your hopes up.
Comment 15 Julian Ospald 2014-03-13 18:21:36 UTC
(In reply to Chris Reffett from comment #14)
> Depends on whether you consider build failures and stuff like that to be
> QA's problem. Further depends on getting hardware--I for one don't exactly
> have boxes lying around which have nothing better to do than build packages
> all day. We can discuss this at the next meeting, but I wouldn't suggest
> getting your hopes up.

I am not really sure if I understand that reasoning.

Testing is the very center of Quality Assurance (that's what I learned... correct me if I am wrong). Gentoo as a distribution ships the portage tree. If no one tests the tree on a global basis (arch testers don't), then there is not much QA overall.

The quality of our tree inherently depends on the compileability of it's packages. Further, there are a lot of use cases where a developer might want/need to request a tinderbox run with a certain package unmasked, a certain eclass changed etc.

In the end, it directly affects the user.
Comment 16 Rick Farina (Zero_Chaos) gentoo-dev 2014-04-16 14:51:20 UTC
I have a tinderbox class system.  I build 4400 packages a day, give or take.  Problem is, it's the same 4400 packages, and bug reports are entirely non-automated.  If you would like to help setup something better, here I am.
Comment 17 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-05-09 16:09:44 UTC
(In reply to Rick Farina (Zero_Chaos) from comment #16)
> I have a tinderbox class system.  I build 4400 packages a day, give or take.
> Problem is, it's the same 4400 packages, and bug reports are entirely
> non-automated.  If you would like to help setup something better, here I am.

We should fix bugs first before adding more of them; reviving Tinderbox would be nice for when we run out of bugs, but that's definitely not the case yet today.

Consider to mark this RESOLVED LATER again...
Comment 18 Julian Ospald 2014-05-09 17:07:42 UTC
(In reply to Tom Wijsman (TomWij) from comment #17)
> (In reply to Rick Farina (Zero_Chaos) from comment #16)
> > I have a tinderbox class system.  I build 4400 packages a day, give or take.
> > Problem is, it's the same 4400 packages, and bug reports are entirely
> > non-automated.  If you would like to help setup something better, here I am.
> 
> We should fix bugs first before adding more of them; reviving Tinderbox
> would be nice for when we run out of bugs, but that's definitely not the
> case yet today.
> 
> Consider to mark this RESOLVED LATER again...

lolwat?
Comment 19 Alexander Berntsen (RETIRED) gentoo-dev 2014-05-09 17:11:31 UTC
(In reply to Tom Wijsman (TomWij) from comment #17)
> We should fix bugs first before adding more of them; reviving Tinderbox
> would be nice for when we run out of bugs, but that's definitely not the
> case yet today.

Should we replace enter_bug.cgi with a website saying "sorry, we have enough bugs for now" as well?
Comment 20 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-05-09 17:34:33 UTC
QA team, please decide what we do with this bug; given that the recent answers by others are unacceptable behavior per the Gentoo CoC, this bug is private for now.
Comment 21 Alex Legler (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2014-05-09 18:04:33 UTC
How about you guys fight this out and talk to infra when there's actually something we can do except getting our inboxes filled with this nonsense.

(In reply to Tom Wijsman (TomWij) from comment #20)
> QA team, please decide what we do with this bug; given that the recent
> answers by others are unacceptable behavior per the Gentoo CoC, this bug is
> private for now.

<hat type="infra">
I don't really see anything that needs to be hidden from public view, especially not to QA. Social contract says we don't hide things, CoC says it's up to comrel to take measures, so until they think this needs to not be public, it won't be restricted.
</hat>
Comment 22 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-05-09 18:25:32 UTC
(In reply to Julian Ospald (hasufell) from comment #18)
> lolwat?

(In reply to Alexander Berntsen from comment #19)
> Should we replace enter_bug.cgi with a website saying "sorry, we have enough
> bugs for now" as well?

Can you two please appropriately use this bug and take your jokes elsewhere?

https://wiki.gentoo.org/wiki/Project:Council/Code_of_conduct

(In reply to Alex Legler from comment #21)
> it won't be restricted.

So, why do we have that checkbox then? You've just publicized a private QA vote.
Comment 23 Alex Legler (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2014-05-09 18:43:46 UTC
> (In reply to Alex Legler from comment #21)
> > it won't be restricted.
> 
> So, why do we have that checkbox then? You've just publicized a private QA
> vote.

There is no voting going on as far as I can see. If you need to hold a vote, do so on a new private bug or even better via email. Locking out the interested audience from the complete discussion is not a valid choice.
Comment 24 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-05-09 18:56:18 UTC
(In reply to Alex Legler from comment #23)
> There is no voting going on as far as I can see. If you need to hold a vote,
> do so on a new private bug or even better via email.

Okay, thank you.

> Locking out the
> interested audience from the complete discussion is not a valid choice.

True, we could only lock those out who are no longer interested; similar to what you've told in Comment #21, I'm tired as well of inboxes filling with "lolwat".
Comment 25 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2014-05-09 19:41:36 UTC
I've talked a few times on conferences about getting testing done. Part of the testing could be based on a tinderbox. Part of the tests I'd like to have are best done through a virtualization environment or involve chroots.
AFAIK, besides Ned (solar) and Diego (flameeyes), the developers that have looked at and keep doing this type of testing are Patrick (bonsaikitten) and Magnus (zorry). I suggest anyone really interested in this bug to talk to them. Rick already said he's doing the same scope of testing, so he's someone else to contact.
Comment 26 Andreas K. Hüttel gentoo-dev 2014-05-09 20:25:25 UTC
Tom, your comment about running out of bugs is just plain silly. 
Julian, please refrain from unnecessary lolwuts, that's also not helping.

Otherwise I see absolutely no business for comrel here. 

I suggest from now on only people who have actually set up a large-scale tinderbox and are willing to do that again should comment with real content. Set up a build system, build a few thousand packages, file a few hundred real bugs. Then, come back here and we get talking.

No, no "ping" reminders either.
Comment 27 Denis Dupeyron (RETIRED) gentoo-dev 2014-05-09 22:01:23 UTC
(In reply to Andreas K. Hüttel from comment #26)
> I suggest from now on only people who have actually set up a large-scale
> tinderbox and are willing to do that again should comment with real content.
> Set up a build system, build a few thousand packages, file a few hundred
> real bugs. Then, come back here and we get talking.

I understand and agree that the behaviour of Tom and Julian above was childish. But that comment raises a few questions. Is this the kind of Gentoo we want? What I mean is, just because we have a few badly behaving developers, others with ideas or constructive criticism can't contribute to a particular area because they're not part of a happy few? Is this a notion we want to extend to all of Gentoo?

Thanks,
Denis.
Comment 28 Julian Ospald 2014-05-10 13:06:47 UTC
(In reply to Andreas K. Hüttel from comment #26)
> Julian, please refrain from unnecessary lolwuts, that's also not helping.
> 

I'm sorry about that. But in fact... that was exactly what I was thinking in the moment I hit "save changes". There was no intention of derailing/trolling, it was just what I was thinking.
Comment 29 Michael Palimaka (kensington) gentoo-dev 2014-05-30 19:17:14 UTC
I've been doing some work on the collection/reporting side of this. In the qa-scripts repo I committed a portage bashrc[1] that collects and stores some information about packages.
This can be processed to produce reports like this: https://dev.gentoo.org/~kensington/tinderbox/

Testing and feedback on the script / output is of course welcomed. It's easy to combine output from multiple sources, so I'd be keen to receive output tarballs from anyone building a decent number of packages to test the parser.

1: http://git.overlays.gentoo.org/gitweb/?p=proj/qa-scripts.git;a=blob;f=tinderbox/bashrc