Bug 132173 – Automatically test the process of installation on a weekly basis

• Product: Gentoo Linux
• Status: RESOLVED WONTFIX

[Description Wiktor Wandachowicz 2006-05-03 13:59:13 UTC]

After doing several (more than 10) Gentoo installations one potentially good idea came to my mind. One that could possibly improve dramatically the experience of the installation process.

What I came up with is the proposal of setting up an automated building environment that would mimic as closely as possible the real environment in which users install Gentoo. It would take a current snapshot of the portage tree and execute the installation exactly as the Handbook says, once a week.

This could serve three purposes:

1) It would catch any breakage and regressions that may rise during
   a typical installation

2) It would provide an additional tool for QA team, substantially increasing
   the quality of the whole project

3) It would enable the automatical building of updated stage files


What frustrated me during several installations I was doing were the simple
errors that popped up, breaking the otherwise smooth "bootstrap" or
"emerge -e system" process. I thought: if the developers were doing a fresh
install at least once in a month, such simple errors wouldn't show at all.
It's called "eating your own dog food"... :-)

With the presence of said autobuilder (autotester?) Gentoo could enter
the "perpetual building" process, similar in idea/goal to the one used
by Eclipse for example. Yes, I know that this metadistribution is source
based (and I *REALLY* love it!) but at least the basic installation process
should be painless and bullet proof.

All the above could be done at least for the manual installation as described
in Handbook, graphical GLI is out of scope of this proposal. Unless one would
like to set up a stress-test compilation of a whole portage tree (or at least
most used packages) on a, say, bi-weekly basis...


So, what do you think?

[Comment 1 Chris Gianelloni (RETIRED) 2006-05-03 15:00:07 UTC]

We already do this.  The problem is the unlimited combinations of USE flags.  Also, we only support a stage3 installation now, so we don't do much with the lower stages sine we *know* that they can break at pretty much any time given the complexity of the USE flag system.

Sorry, it just isn't feasible without IBM donating us some Blue Gene hardware and us gettign a few hundred monkeys to push buttons over and over.