I would like to have the ability to mount a tmpfs on /var/tmp/portage when I am building a stage. I have attached a patch that adds this functionality to catalyst. This patch adds a new option "tmpfs" and the config value "tmpfs_opts". tmpfs_opts can be used to set the size of the tmpfs. It uses the same syntax as options passed to mount -o. For example tempfs_opts="size=1G,nr_inodes=10". Reproducible: Always
Created attachment 200106 [details, diff] Patch to add an option for mounting a tmpfs
Created attachment 200473 [details, diff] Patch to add an option for mounting a tmpfs This patch fixes a bug where a directory called tmpfs was being created in the current working directory when catalyst was run.
This is rather...specific. While this would give some speed improvements, there is also a lot of time lost on disk I/O when unpacking stage tarballs or rsyncing from the source_subpath's build dir. I'd much prefer to have automatic tmpfs support like this use a tmpfs for the entire build directory. However, it has problems of its own. After the build completes, you'd lose the build directory, and the next step in the process would have to unpack the tarball. In the case of livecd-stage1->livecd-stage2, there is no tarball to unpack. I think this particular feature would be one much better put in portage than catalyst.
What do you think about adding a feature to portage to handle this, Zac?
only trouble i see is people would probably whine when it fills up because they tried to build OOo on a system with 2GiB of memory, and they'll want portage to autodetect that situation
I recall reports that tmpfs tends to have poor behavior in low-memory situations, when the tmpfs blocks tend to displace processes from memory and leads to a ridiculous amount of swapping. I haven't tested this case, but if it's still an issue then it seems like tmpfs should be fixed be given a mount option that gives higher priority to processes for allocation of physical memory pages.
Thinking about this a bit, what about adding a PORTAGE_TMPFS_SIZE variable that allows the user to specify the maximum size of the tmpfs mount. It would default to "0", disabling portage from creating a tmpfs mount. Some packages (eg openoffice, libreoffice, icc) already have CHECKREQS_ for hard disk space and memory size in pkg_setup(). I think this information could be leveraged. Questions: Should the tmpfs mount be one large mount for /var/tmp/portage, or would making individual mounts per-package help anything? What to do in the case of package failure (prompt the user and ask before unmounting/deleting the temporary contents?)?
(In reply to comment #7) > Questions: Should the tmpfs mount be one large mount for /var/tmp/portage, or > would making individual mounts per-package help anything? I think one large mount is desirable, for the sake of simplicity. In fact, given that the user can easily configure this via /etc/fstab, I'd prefer for portage not to get involved (for the sake of simplicity). > What to do in the > case of package failure (prompt the user and ask before unmounting/deleting the > temporary contents?)? If we leave it to the user to configure via /etc/fstab, the it's no concern of portage. Users who use tmpfs for /var/tmp/portage can set FEATURES=fail-clean to have temp directories cleaned up on failure. If they'd like to do something more complex then then can use register_die_hook to make a tarball out of $PORTAGE_BUILDDIR before FEATURES=fail-clean does its thing. Something like this in /etc/portage/bashrc will do the trick: backup_after_die() { cd "$PORTAGE_BUILDDIR" || return addwrite /var/tmp/build_failures mkdir /var/tmp/build_failures/$CATEGORY tar -czf /var/tmp/failures/$CATEGORY/$PF.tar.gz . } register_die_hook backup_after_die
(In reply to comment #8) > Right, the bug report was originally filed against catalyst and then shoveled off to portage. Mounting the var/tmp/portage of various catalyst builds can't really be done, especially when most of catalyst is run automatically. Do you think perhaps that the original idea of implementing this in catalyst should be done instead of portage? It does seem to me that the code would be better off in portage itself.
(In reply to comment #9) > Do you think perhaps that the original idea of implementing this in catalyst > should be done instead of portage? Yes, I think so. Catalyst already does lots of mounting and unmounting, so it can be easily added to catalyst along with all the other mounts. OTOH, portage currently doesn't do any mounting or unmounting.
I apologize, I didn't even notice this, but I independantly wrote this due to a bug in ZFS requiring me to do so. It's in 2.0.12.1, let me know if my way isn't good enough for some reason.
