Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 649362 - stages for use in external continuous integration systems
Summary: stages for use in external continuous integration systems
Status: CONFIRMED
Alias: None
Product: Gentoo Release Media
Classification: Unclassified
Component: Stages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Release Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-03-02 14:21 UTC by Matthew Thode ( prometheanfire )
Modified: 2020-05-28 18:10 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2018-03-02 14:21:27 UTC
There are two separate projects I know of that wish to use stage4s as a source for CI.

There's a stage4 musl (nomultilib) that the Octavia project wishes to use.
Openstack infra would also like a stage4 systemd stage so that Gentoo support can be added to Openstack-Ansible.

The project that'd be directly consuming those stage4s (and would output a qcow2) is diskimage-builder.

In order to add those profiles to diskimage builder they require a build, preferably at least once a week (so diskimage-builder's ci testing doesn't take forever on the gentoo image).  Another general requirement is that the build is automated in the distros own CI.

The problem with the current musl stages are that they are experimental, as long as they are signed and use automation to build weekly that's fine though.

The problem with the current systemd stages is the bus factor.  While the tooling seems to exist for automating the build, It is not used.  Instead a human builds the image.  I'm not sure why.

This is blocking

Reproducible: Always
Comment 1 Anthony Basile gentoo-dev 2018-03-02 14:42:03 UTC
(In reply to Matthew Thode ( prometheanfire ) from comment #0)
> There are two separate projects I know of that wish to use stage4s as a
> source for CI.
> 
  While the
> tooling seems to exist for automating the build, It is not used.  Instead a
> human builds the image.  I'm not sure why.


The tool for building the stages is on the releng repo and it is being used.  Please see

https://gitweb.gentoo.org/proj/releng.git/tree/tools-systemd

Can you please add code to that as you did for musl stage4.  I should also point out that there are other companies that have interest in how those stages are built and so we need to find a way to address everyone's requirements, not just the one's your dealing with.

Please define "the bus factor".
Comment 2 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2018-03-02 16:43:09 UTC
Ok, we went over the setup in IRC, seems there were multiple issues.

The systemd spec files NOT in tools-systemd should not be there (they were historical), that's not fixed and they are removed (still in git history).

Also, I misunderstood what was meant by automation, blueness explained that the stages are built on the build box automatically and sent to the staging area to be signed by the releng key.  This fulfils my requirements.

I've update the tools-systemd tooling to run the stage4 for amd64.  Tools musl should be changed to do likewise.
Comment 3 Anthony Basile gentoo-dev 2018-03-04 10:50:34 UTC
stage4 systemd has hit the mirrors(In reply to Matthew Thode ( prometheanfire ) from comment #2)
> 
> I've update the tools-systemd tooling to run the stage4 for amd64.  Tools
> musl should be changed to do likewise.

stage4 systemd has hit the mirrors.

stage4 musl is not a good idea at this time because it depends on a overlay that is in flux.  as a result, you will hit frequent failures on the extra packages pulled in by a stage4.

you have not defined "the bus factor".  if this is an issue to be addressed, then you need to define it.  what are the steps to reproduce?
Comment 4 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2018-03-04 20:30:28 UTC
https://www.wikiwand.com/en/Bus_factor
Comment 5 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2018-03-04 20:33:35 UTC
How do I document steps to reproduce an external request?

1. walk up to me in a conference (second one in six months).
2. make request

I'll let him know musl isn't stable enough for any type of continuous integration though.
Comment 6 Anthony Basile gentoo-dev 2018-03-04 22:21:04 UTC
(In reply to Matthew Thode ( prometheanfire ) from comment #4)
> https://www.wikiwand.com/en/Bus_factor

clearly the bus hit you and not me because you have the root password to that box!!!  i've already spoken with robbat2 about setting things up better.

when you said bus factor, i thought you meant something was throwing a SIGBUS (see https://en.wikipedia.org/wiki/Bus_error)



(In reply to Matthew Thode ( prometheanfire ) from comment #5)
> How do I document steps to reproduce an external request?
> 
> 1. walk up to me in a conference (second one in six months).
> 2. make request
> 
> I'll let him know musl isn't stable enough for any type of continuous
> integration though.

do we want to publish something that we feel might be unstable?  i mean i could produce the stage4 but with a critical overlay (ie one without which the stage breaks), i don't feel comfortable providing it.

anyhow, can't they read and comment on this bug so i can better understand what they want.
Comment 7 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2018-03-05 13:07:40 UTC
I actually don't have the password :P  I'd just reboot into recovery to get in if I had to (and others should be able to do the same, with the same access I have).

Ya, I've been trying to get them to comment on this bug, but it's like pulling teeth :(

As for publishing these stages, I personally thing we should have some sort of 'ci' section of distfiles for stuff like this.  See https://bugs.gentoo.org/608150 for a bug about that.