This is with catalyst-2.0.6_pre9 When I try to generate stage1, catalyst have some errors (I will submit the ouput). It seems that "storedir"/tmp/*/stage1-*/tmp/stage1root has two issues: 1) "storedir"/tmp/*/stage1-*/tmp/stage1root/usr/bin/which is missing (altough it's present in the "storedir"/tmp/*/stage1-*/usr/bin) so /usr/lib/catalyst/targets/support/chroot-functions.sh:216:update_env_settings() fail. Thus the warning about that env-update is missing, which it isn't. 2) "storedir"/tmp/*/stage1-*/tmp/stage1root/usr/portage is empty, so emerge fails and complains about the missing make.profile link. I have confirmed this by putting catalyst to sleep (ctrl+z) just before, and binding the portage directory manually, and then it could continue. Reproducible: Always Steps to Reproduce: 1. generate stage1 from scratch, using gentoo 2007.0 stage3 as starting point
Created attachment 144752 [details] The relevant output from catalyst before it aborts.
Well, this was already fixed in SVN and also in the 2.0.6_pre10 version which I am just about to release. In the future, please check to make sure that the issue still exists in SVN before filing a bug report. We're in the middle of a release, so catalyst is going to change rather quickly and break several times. If you need a stable catalyst version which is known to work, use the version marked as stable.
Ok, I'm sorry about not checking the changelog. But I believe only problem 1 has been fixed. At least nothing in the changelog seems to have to do with problem 2. I even tried with 2.0.6_pre10 and it still occured; the portage dir remains empty in the new chroot (I think it should be bount to the snapshot_cach)e so emerge fails later on.
Well, it works fine for everyone else. If you can track down the problem and point out exactly what is wrong (and perhaps even whip up a patch), we'll be more than happy to take a look.
Created attachment 144915 [details, diff] fixes preclean in stage1 It seems what's causing this is the call of run_default_funcs in targets/support/chroot-functions:402. This was added in 2.0.6_pre9, and that was when this problem started to appear. It seems when that file is sourced during action sequence preclean, run_default_funcs is called which results in an emerge of ccache. This doesn't seem to work well (emerge fails since /usr/portage is empty), and seems unnecessary. ccache shouldn't be needed from now on for stage1, plus ccache was already merged earlier during this stage (stage1). Simply removing the call of run_default_funcs fixes this, see the attached patch.
It turns out that run_default_funcs is not run anywhere else (the comment above the instance I removed said it was run "everywhere" so I was fooled!). As such, distcc and ccache are never emerged and won't work. So, it need to be run _somwhere_ but not when chroot-functions is sourced. I leave figuring out when to call it to you guys, I really have no clue.
Created attachment 144940 [details, diff] makes catalyst call run_default_funcs at the right places The attached patch makes catalyst call run_default_funcs at roughly the same places as update_env_settings/setup_myfeatures (which run_deafult_funcs calls in turn) was called in earlier revisions, which was right after targets/support/chroot-functions.sh is sourced in all targets/*/*-chroot.sh scripts. I'm a bit unsure whether run_default_funcs should be present in targets/support/kmerge.sh, and if so where it should be. But I added it after targets/support/chroot-functions.sh is sourced there as well.
Arrr! My ccache go cleared so obviously something isn't all right with my patch. Any suggestions, comments?
Well, we don't use ccache when building releases. This is why we're not seeing this issue ourselves. The easy way to "fix" this is to just disable ccache in your catalyst.conf. Since it's not preventing us from building the release, we're not in any hurry to fix this, especially if we have to fix it ourselves.
So what exactly was the reasoning for moving the call from being a single line in a single file to a line in a ton of files (and you missed all of the preclean places where we actually want/need this to run)? I understand what you think is the problem, but have no idea why you chose this as a solution to it. It's definitely a step backwards in terms of functionality. What would likely be best would be to modify setup_myfeatures to be smarter about not trying to install distcc/ccache/icecream if they already exist. This has the added benefit of not destroying these packages if someone had merged them with different USE, since we force a specific USE for them.
Created attachment 144984 [details, diff] makes setup_myfeatures check if ccache/distcc/icecream is already installed before emerging them (In reply to comment #10) > So what exactly was the reasoning for moving the call from being a single line > in a single file to a line in a ton of files (and you missed all of the > preclean places where we actually want/need this to run)? Well, the patch was mainly for myself I guess, just to get catalyst to produce a working system (which it did). Obviously I wasn't smart enough to just disable ccache at the time, which would have done the trick. Posting it here at least showed you what I thought was the problem very explicitly. > I understand what you think is the problem, but have no idea why you chose this > as a solution to it. It's definitely a step backwards in terms of > functionality. What would likely be best would be to modify setup_myfeatures > to be smarter about not trying to install distcc/ccache/icecream if they > already exist. This has the added benefit of not destroying these packages if > someone had merged them with different USE, since we force a specific USE for > them. Yes, the patch and its whole approach sucked. I'm certainly no expert on the inner workings of catalyst so me writing patches for it is definitely questionable. Any way, attached is a patch (for 2.0.6_pre10) which does what you suggested. I tried generating a stage1 with it, and it worked flawlessly, even with ccache enabled (and now it wasn't emptied erroneously).
That patch would certainly work and does exactly what I asked. After looking at it, I see only one real problem, and that is that we're checking /var/db/pkg, directly. This isn't a problem right now because we only support a single package manager, portage. However, we're making efforts to try to start moving things into a more package manager agnostic fashion. We'll likely still only support portage for some time, but someone else could easily patch in support for other package managers once the framework has been abstracted. Anyway, I'm adding in --noreplace onto each line, instead. I spoke with Zac Medico and he's modifying portage to have --noreplace override --newuse (as it should have anyway). What this means is that this isn't really "FIXED" per se, but it is fixed on the catalyst side. Once a new version of portage is out with this change made, it'll all just work. <CIA-50> zmedico * r9513 portage/main/trunk/pym/_emerge/__init__.py: Make --noreplace take precedence over --newuse, as suggested by wolf31o2. *grin*
By the way, the portage patch in question is at http://dev.gentoo.org/~zmedico/tmp/noreplace.patch so you can test it out and see if it works with catalyst 2.0.6_pre16 (which has the catalyst side fixed)...
*** Bug 220999 has been marked as a duplicate of this bug. ***
I suspect that I hit the same problem but described it more telegraphically in Bug #220999. Note that I am using Catalyst, version 2.0.6_pre17 (forgot to mention it in bug #220999)
Yeah, I think I broke it again after _pre16... Can you test out the -9999 ebuild, which pulls from catalyst SVN trunk? If it's fixed, I can roll up a new pre-release version with the fix.
This is in catalyst 2.0.6, which is in the tree and stable. If there's still a problem with this, please REOPEN this bug. Thanks!
