Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 936914 - arm64: Short-term replacement for jiji.arm.dev.gentoo.org
Summary: arm64: Short-term replacement for jiji.arm.dev.gentoo.org
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Foundation
Classification: Unclassified
Component: Proposals (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Gentoo Board of Trustees
URL:
Whiteboard: Council: 6y 0n 1a (passes), Trustees:...
Keywords:
Depends on:
Blocks:
 
Reported: 2024-07-29 22:27 UTC by Robin Johnson
Modified: 2024-11-10 19:14 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2024-07-29 22:27:22 UTC
The WorksOnARM program is ending all sponsorships on short notice: we have until August 10 to backup and data and wipe work.

This specifically impacts Gentoo's ARM64 work, as WorksOnArm provided the jiji.arm.dev.gentoo.org system:
80 cores, 256GB RAM, 1TB NVME

The closest fast comparision is the Hetzner RX170 system:
Ampere® Altra® Q80-30
128 GB DDR4 ECC
2x 960 GB Datacenter NVMe SSD

If hosted in Finland: EUR201.11/mo
once-off setup: + EUR94.01
(germany is 10EUR/mo more)
Comment 1 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2024-07-29 22:29:09 UTC
Motion:
Approve usage of Hetzner RX170 server for 3 months. To be re-evaluated in 60 days from approval date, to hopefully find a sponsored option instead.

Cost:
201.11/mo*3mo + 94.01 = 607.34EUR
Comment 2 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2024-07-29 22:29:50 UTC
yes (trustees)
yes (council)
Comment 3 Andreas K. Hüttel archtester gentoo-dev 2024-07-29 22:31:42 UTC
yes & yes
Comment 4 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2024-07-29 22:46:24 UTC
yes (trustee)
Comment 5 David Seifert gentoo-dev 2024-07-30 15:54:57 UTC
yes (trustees)
yes (council)
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-07-30 16:25:38 UTC
Yes (Council).
Comment 7 Arthur Zamarin archtester Gentoo Infrastructure gentoo-dev Security 2024-07-30 16:28:17 UTC
yes (council)

But I want to add some more notes:

I don't want to be that one who doesn't agree to any expense or something, I just want us to consider our usage to not loose the money the project have, so just some considerations on the decision:

What stuff we do on jiji:

1. Arch Testing for arm & arm64
2. catalyst for arm & arm64
3. catalyst for some other arches, I know mips, but maybe extra
4. Debugging arm{,64} failures
5. Building gentoo-kernel-bin for arm64

We also have another devbox for arm64 (with also arm supported) which is kamaji. It is much weaker (16 cores). I believe we could move arch testing (1), debugging (4) and kernel building (5) to it. We than can give correct priority and stuff so we can divide without issues and do stuff during inactivity time.

The catalyst for other arches like mips (3), is done through qemu, so it can be moved to any amd64 host (for example reuse the resources of chromium.dev.gentoo.org which is on our super server).

We are left with arm catalyst (2), which might be able to be moved also on kamaji, maybe by making stage3 a little bit more rarer.

While maybe all of this is too much for a 16 core machine, it might enable us to select a much weaker arm64 host to "buy".

On the other hand, we don't have a lot of time to migrate, so maybe better to do it simple and easy.

---

But, I put a "yes" vote since I believe in you selecting the best solution, and I prefer dilfridge and sam to say if my ideas have weight or am I just too optimistic.
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-07-31 00:58:44 UTC
(In reply to Arthur Zamarin from comment #7)
> yes (council)
> 
> But I want to add some more notes:
> 
> I don't want to be that one who doesn't agree to any expense or something, I
> just want us to consider our usage to not loose the money the project have,
> so just some considerations on the decision:
> 
> What stuff we do on jiji:
> 
> 1. Arch Testing for arm & arm64
> 2. catalyst for arm & arm64
> 3. catalyst for some other arches, I know mips, but maybe extra
> 4. Debugging arm{,64} failures
> 5. Building gentoo-kernel-bin for arm64
> 

6. Build binpkgs for arm64 (we have full coverage for amd64+arm64)

(perhaps unwisely, as we should really separate that from the devbox side; on catbus, I try to give out access to an nspawn container at least.)

kamaji isn't going to be strong enough for that. It might be OK if we split it out though as we should.

7. Regularly testing toolchain components with large @world rebuilds

I might just have to accept we can't do this anymore. We have found loads of GCC bugs this way long before they could ever hit users though.

--

I think you're right that we shouldn't discount kamaji and what we might be able to do is move some stuff to kamaji, allowing us to buy maybe a cheaper arm64 machine for the other tasks.

In the meantime though, I vote yes (council).
Comment 9 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2024-07-31 05:32:10 UTC
This motion passes. I'm getting the server online for migration of data.

I entirely agree we can probably make it work with multiple smaller servers - but we need to see.

Gentoo was far from the only organization that used the WorksOnARM program - and many others will probably be looking for new sponsors just like us.
Comment 10 Ulrich Müller gentoo-dev 2024-07-31 09:44:28 UTC
Abstention (Council)
Abstention (Trustees)
Comment 11 Andreas K. Hüttel archtester gentoo-dev 2024-11-10 19:14:15 UTC
The replacement was used for a few months but already handed back to Hetzner.

We now have a permanent replacement from arm, hosted at OSUOSL.