Neither the BTRFS="yes" or "genkernel --btrfs all" do anything at all. Btrfs is still only build as a external module "M" which is the default (not built-in as is required when rootfs is on Btrfs). This happens when running from a live cd (using a different root file system than Btrfs) within chroot.
If genkernel is run from within Gentoo with Btrfs as rootfs (not from the chroot when installing) the autodetection works fine and Btrfs is set as a built-in module.
The current solution is to use "genkernel --menuconfig all" and set Btrfs as built-in manually.
I have not checked if enabling support for other file systems has the problem.
Steps to Reproduce:
1. Run a live system to install Gentoo from (within a VM). I was using SystemResque instead of Gentoo minimal
2. Install Gentoo in VM up until the genkernel kernel compilation
3. Try setting (or not setting) BTRFS="yes" in /etc/genkernel.conf or run genkernel --btrfs --menuconfig all.
4. Btrfs will remain as an external module "M", but should be built-in
I expect --btrfs or BTRFS="yes" to ensure Btrfs is compiled as a built-in module. The rootfs will not mount if Btrfs is not built-in.
"Does not do anything" is wrong, like man page is saying, the option will cause genkernel to add btrfs-progs to initramfs:
> Includes or excludes Btrfs support. When enabled, this will
> compile btrfs-progs for you.
It actually never took care of required kernel options -- it was only about "/sbin/btrfs device scan" (https://gitweb.gentoo.org/proj/genkernel.git/commit/?id=d41ebc4d721a8706d0b0f7c5cd1ed8dab42d667c) which was necessary to start multi-device BTRFS setups in the past. Since we moved back to UDEV in >=4.1.0, this isn't used and needed anymore. Today it is like --[no-]e2fsprogs or --[no-]xfsprogsm, i.e. it will only add tools necessary to be able to check/repair filsystem from initramfs if you want to. And that's what documented in man page.
I'll think about renaming option to something like --btrfs-progs or updating at least text in /etc/genkernel.conf.
Also, you assumption that it should be built-in is wrong. By default, genkernel will produce a modular kernel. Only if it will detect that given kernel.config doesn't support modules (https://gitweb.gentoo.org/proj/genkernel.git/tree/gen_configkernel.sh?h=v4.2.1#n433) it will built-in.
Of course it will respect what user has already set.
Ahh, I understand, my bad. I did not read the man pages as the /etc/genkernel.conf seemed to be well commented.
I do agree however that the text should be updated as it is confusing to set Btrfs support to Yes and then have a unbootable computer because the rootfs is Btrfs.