I'm a die-hard Gentoo guy. All of my systems run Gentoo. I installed Gentoo on an SD card for my 256MB RAM Beaglebone. It was easy and awesome. Then I decided I wanted to run the entire OS from a ramdisk and I slowly came to the realization that Gentoo likely could not be stripped down to a sufficient size. With a lot of work it is possible but not guaranteed. To my great dismay, I was forced to switch distros for this project and set up Angstrom. How about a super-small profile and stage for Gentoo for this sort of thing? Reproducible: Always
I forgot to add, Gentoo has truly stellar ARM support and this would be a great way to benefit from that even more.
(In reply to comment #1) > I forgot to add, Gentoo has truly stellar ARM support and this would be a > great way to benefit from that even more. Can you provide a list of packages you would want in this tiny stage.
Sir, I think we should include only the bare minimum in order to get portage working, minus the portage tree. That is what the current stages already provide from what I understand, but we need a much smaller version. I think other distros accomplish this via busybox and uclibc. There are a couple of Gentoo attempts: http://judepereira.com/blog/going-embedded-with-mgentoo/ http://en.gentoo-wiki.com/wiki/TinyGentoo Tiny Core is 8MB and the Angstrom rootfs I use is 29MB. Angstrom has a really cool rootfs generator: http://narcissus.angstrom-distribution.org/ Here's Tiny Core's ARM rootfs: http://distro.ibiblio.org/tinycorelinux/4.x/armv7/rootfs.tgz Buildroot looks to have a nice system too: http://buildroot.uclibc.org/
(In reply to comment #3) > Sir, I think we should include only the bare minimum in order to get portage > working, minus the portage tree. That is what the current stages already > provide from what I understand, but we need a much smaller version. I'd need a more specific package list. Maybe let's start from what we have and slim down. Can you take a look at http://mirrors.rit.edu/gentoo/experimental/amd64/uclibc/ and http://mirrors.rit.edu/gentoo/experimental/x86/uclibc/ which are the stages I'm currently maintaining and let me know what you would like removed? Those stages are coming in at ~100MB but a lot of that is stuff under /usr/share which could be dumped.
I wish I had the skills to know what should be dumped. You have x86 and amd64 uclibc stages and profiles around 100MB? I had no idea. If you could crank out an arm set I think it would be a big deal for Gentoo on arm. It looks like there is some sort of arm stage4 up?
(In reply to comment #5) > I wish I had the skills to know what should be dumped. > > You have x86 and amd64 uclibc stages and profiles around 100MB? I had no > idea. If you could crank out an arm set I think it would be a big deal for > Gentoo on arm. It looks like there is some sort of arm stage4 up? arm: http://mirrors.rit.edu/gentoo/experimental/arm/uclibc/ also mips: http://mirrors.rit.edu/gentoo/experimental/mips/uclibc/
May I ask what has been added to that arm stage to make it a stage4?
I'm glued to the election right now but when I get some time I will go through those stripped-down Gentoo links I posted and make a list of stuff that should be OK to dump.
I extracted stage4-armv7a-softfloat-uclibc-hardened-20120929.tar.lzma and it comes to 531MB so we should be able to reduce that a lot. I will come up with a list of stuff to throw out ASAP.
This is the stuff that can go for sure according to Jude but it only frees about 50MB: /var/db/* /var/log/* /usr/include/ /usr/lib/pkgconfig/ /usr/share/nano/ /var/lib/portage/ /var/lib/gentoo/ /var/lib/misc/ /var/cache/* /usr/share/terminfo/ /usr/share/baselayout/ /usr/share/consoletrans/ /usr/share/openrc /usr/sbin/fdformat /var/lib/misc/ /etc/logrotate.d/ /etc/portage/ He has uclibc stage3's but the latest extracts to 257MB: http://judepereira.com/blog/x86-uclibc-stages/ He claims 17MB but I'm not sure where the space savings come from. Can you tell? His instructions are very short: http://judepereira.com/blog/going-embedded-with-mgentoo/
Grant, the space savings come from the use of the most essential USE flags. That's all. I had an idea of putting all of that into a ramdisk and booting into RAM for speed, but then I never got around to doing that. Also, because uClibC is being used, with the -Os optimization, it does come upto 17MB. I had further gone down to 11MB with help from #gentoo and #gentoo-embedded on Freenode earlier. I still have such builds lying around, also a 16MB Flash card image. Would you like to see them? The 16MB flash card image works very well.
Furthermore, here is an updated list on what can be safely removed: mkdir /mounted/dev mkdir /mounted/proc cp -pr /dev/* /mounted/dev rm -rf /mounted/var/db/* rm -rf /mounted/var/log/* rm -rf /mounted/usr/include/ rm -rf /mounted/usr/lib/pkgconfig/ cp -pr /mounted/usr/share/keymaps/i386/qwerty/ ./ rm -rf /mounted/usr/share/keymaps/* mkdir /mounted/usr/share/keymaps/i386/ mv ./qwerty/ /mounted/usr/share/keymaps/i386/ rm -rf /mounted/usr/share/nano/ rm -rf /mounted/var/lib/portage/ rm -rf /mounted/var/lib/gentoo/ rm -rf /mounted/var/lib/misc/ rm -rf /mounted/var/cache/* rm -rf /mounted/usr/share/terminfo/ rm -rf /mounted/usr/share/baselayout/ rm -rf /mounted/usr/share/consoletrans/ rm -rf /mounted/usr/share/openrc rm -rf /mounted/usr/sbin/fdformat rm -rf /mounted/var/lib/misc/ rm -rf /mounted/etc/logrotate.d/ rm -rf /mounted/etc/portage/ cp /sbin/fsck.ext4 /mounted/sbin/ cp /lib/libext2fs.so.2* /mounted/lib/ cp /lib/libcom_err.so.2* /mounted/lib/ cp /lib/libblkid.so.1* /mounted/lib/ cp /lib/libuuid.so.1* /mounted/lib/ cp /lib/libe2p.so.2* /mounted/lib/ cp /lib/libpthread* /mounted/lib/ cp /lib/ld-uClibc* /mounted/lib/ cp /mounted/usr/share/unimaps/iso01.uni ./ cp /mounted/usr/share/consolefonts/default8x16.psfu.gz ./ rm -rf /mounted/usr/share/unimaps/* rm -rf /mounted/usr/share/consolefonts/* mv ./default8x16.psfu.gz /mounted/usr/share/consolefonts/ mv ./iso01.uni /mounted/usr/share/unimaps/
(In reply to comment #3) > Sir, I think we should include only the bare minimum in order to get portage > working, minus the portage tree. That is what the current stages already > provide from what I understand, but we need a much smaller version. I think > other distros accomplish this via busybox and uclibc. There are a couple of > Gentoo attempts: > > http://judepereira.com/blog/going-embedded-with-mgentoo/ > http://en.gentoo-wiki.com/wiki/TinyGentoo > > Tiny Core is 8MB and the Angstrom rootfs I use is 29MB. Angstrom has a > really cool rootfs generator: > > http://narcissus.angstrom-distribution.org/ > > Here's Tiny Core's ARM rootfs: > > http://distro.ibiblio.org/tinycorelinux/4.x/armv7/rootfs.tgz > > Buildroot looks to have a nice system too: > > http://buildroot.uclibc.org/ I've tried to get portage inside, but failed, as having Python itself jumps the disk size to over 50MB, unless you have 128MB of RAM, then maybe its a possibility. Also, why would you have portage if the rootfs is in RAM? Are you intending to mount /usr or the like after boot? Having portage and doing any sort of install would not be persistent.
Thanks for lending your expertise Jude. If we have a stage and profile complete with python and portage and a compiler, would it be easy to remove them manually? Maybe a few 'rm -rf' commands? The packages could be injected to satisfy the profile. Maybe python and portage and the compiler could be emerged with FEATURES="buildpkg" before the stage is created so a user could chroot or boot the stage and: 1. emerge required packages 2. manually delete python, portage, and the compiler 3. back up and delete the python, portage, and compiler packages 4. use the tiny distro (in a ramdisk if necessary) 5. later extract the backed-up python, portage, and compiler packages 6. do more portage stuff 7. manually delete python, portage, and the compiler again 8. use the tiny distro again What do you think?
Actually I suppose the injection wouldn't even be necessary since the packages would have been deleted manually.
@above comments, You're taking stages that I've built and removing stuff in the hopes of slimming it down. Another approach is to replace as many mainline packages with a fully featured busybox. Take @world from say, my stage3-amd64-uclibc-hardened tarball, and find what packages can be replaced by busybox. I'll set up catalyst to build those stages. I'm too busy with other stuff to do that myself.
Jude, is that the approach you took with your stages?
Hi Grant, I'm not sure if it is entirely possible to do something like that. However, give me about an hour, I'll try to emerge portage alone in a chroot instance, and will update you on the size.
Hmmm, Grant, why don't you use the stage3 itself to build a chrooted install for your complete needs? I don't think we can get the current stage3's size significantly lower.
(In reply to comment #18) > Hi Grant, > I'm not sure if it is entirely possible to do something like that. However, > give me about an hour, I'll try to emerge portage alone in a chroot > instance, and will update you on the size. It is alway possible. Eg. tar will cause you problems, so you'll have to install app-arch/tar and configure busybox to no provide tar support. The question is if somene has or is willing to sort through "what we can get away with" in busybox. I now use catalyst with a saved config for uclibc to prevent configuration insanities. We can do the same for busybox.
I think I'm missing something. To me it seems like if we are able to come up with a 16M rootfs, we should be able to build a stage3 a lot smaller than 257M. I'm going to use Jude's recipe on stage3-amd64-uclibc-vanilla-20121031.tar.bz2 and poke around and see what I can learn. Jude, did you go through the sort of process Anthony is describing with busybox when you came up with your stage3, or did you not mess with busybox?
Jude, I have everything set up but this error prevents me from getting a shell after boot: /bin/sh: can't load library 'libgcc_s.so.1' Anyway, I think I have a better handle on all of this now. What's great about mGentoo is it's tiny. What's great about a stage3 is portage. I think what we're looking for is a way to have the best of both worlds. I can think of 3 approaches: 1. How about a script that archives a list of files equivalent to those installed by portage and its dependencies which are not otherwise depended upon, and then deletes them. If portage and its dependencies are responsible for most of the size difference between Jude's 16M mGentoo and his 257M stage3, this should yield a relatively tiny system. Another script could extract the same files from the archive back to the filesystem in order to use portage again. The system would then be restored back to a 100% sane, official Gentoo state for working with portage. As far as determining which files should be archived and deleted, maybe portage could be queried for --depclean, and then queried for --depclean again with portage removed from the profile, and then the files from the set of packages which were added to the --depclean list after portage was removed from the profile could be used. 2. A thorough busyboxing of a standard uclibc stage3 as Anthony suggested. It sounds like this would require some effort to put together as we aren't sure what can be successfully replaced with busybox and what can't. How much space could likely be saved this way? 3. Both 1 and 2.
To go along with #1, 2, or 3 above, maybe a new uclibc stage3 could be introduced with some of Jude's tricks like: -Os essential USE flags only INSTALL_MASK="HACKING.gz TODO.gz" Or maybe those would be good for the current uclibc stage3's?
(In reply to comment #22) > Jude, I have everything set up but this error prevents me from getting a > shell after boot: > > /bin/sh: can't load library 'libgcc_s.so.1' Add /usr/lib/gcc/x86_64-gentoo-linux-uclibc/4.6.3 to /etc/ld.so.conf (or wherever your libgcc_s.so.1 lives). Then do chroot /your/chroot /bin/ldconfig Unless you clobbered that directory, in which case your need to restore it.
Jude only installs the following for mGentoo: ROOT=/mounted/ emerge -auvND baselayout uclibc bash dropbear pam udev iptables coreutils nano util-linux shadow kbd net-tools grep procps gzip sed findutils mawk htop So I don't have a /usr/lib/gcc/ or libgcc_s.so.1 at all. I copied it from stage4-armv7a-softfloat-uclibc-hardened-20120929.tar.lzma to /lib/ and it works and I can boot all the way into mGentoo. Jude, how do you normally deal with libgcc_s.so.1? Is it normal to see a whole slew of errors during boot? Anthony, is the imaginary script I refer to above any good? Would it be worth pursuing if I had the skills?
Anthony, may I request more ARM uclibc stages? Maybe a vanilla stage3? I can submit a new bug if you'd like.
(In reply to comment #22) > Jude, I have everything set up but this error prevents me from getting a > shell after boot: > > /bin/sh: can't load library 'libgcc_s.so.1' > > Anyway, I think I have a better handle on all of this now. What's great > about mGentoo is it's tiny. What's great about a stage3 is portage. I > think what we're looking for is a way to have the best of both worlds. I > can think of 3 approaches: > > 1. How about a script that archives a list of files equivalent to those > installed by portage and its dependencies which are not otherwise depended > upon, and then deletes them. If portage and its dependencies are > responsible for most of the size difference between Jude's 16M mGentoo and > his 257M stage3, this should yield a relatively tiny system. Another script > could extract the same files from the archive back to the filesystem in > order to use portage again. The system would then be restored back to a > 100% sane, official Gentoo state for working with portage. As far as > determining which files should be archived and deleted, maybe portage could > be queried for --depclean, and then queried for --depclean again with > portage removed from the profile, and then the files from the set of > packages which were added to the --depclean list after portage was removed > from the profile could be used. > > 2. A thorough busyboxing of a standard uclibc stage3 as Anthony suggested. > It sounds like this would require some effort to put together as we aren't > sure what can be successfully replaced with busybox and what can't. How > much space could likely be saved this way? > > 3. Both 1 and 2. Hey Grant, I never did come up to that error, I faced other booting issues, but then I could fix all of them and I boot successfully without an error. And yes, I think the third option is decent, also, you could consider this option: On the first boot, all applications are used(you could give it always three or four boots), and every time that a certain script finds that any file is not accessed for over three or four days, then that file can be safely moved from the rootfs to a backup place, and whenever you'd want to emerge, you have an AUFS mount that mounts an "overlay" of portage deps on the real rootfs. So, what we have is: two filesystems - rootfs, and aufs_store. File not accessed more than three days, gets moved to aufs_store, and whenever an emerge is made, the aufs_store is mounted as an overlay ob the rootfs, thus combining the entire system for package management. This way, you have a very minimum rootfs, capable of booting completely into ram.
Re minimals and portage: what I have going even on my regular systems is squashfs-d portage. You can mostly get the whole tree squashed into ~50MB. On my buildhost I have two cron scripts, where ansimail generates nice html colored output scripts for e-mail: $ cat 09-eix-sync #!/bin/sh MAILTO="${MAILTO:-root}" PATH=$PATH:/usr/local/bin PORTDIR=/mnt/gentoo/portage eix-sync -C --quiet 2>&1 | ansimail -s "$0" "$MAILTO" $ cat 10-squash-portage #!/bin/sh MAILTO="${MAILTO:-root}" PATH=$PATH:/usr/local/bin TODAY=$(date +%F) # rsync personal overlay into squash work area (can skip) rsync -avhL --cvs-exclude --delete-excluded --delete /secure/home/leho/portage/ /var/lib/layman/.squash/overlays/leho 2>&1 | ansimail -s "$0" "$MAILTO" # prepare to switch weeks umount /mnt/gentoo/portage.sqfs # build main tree with -noappend mksquashfs /mnt/gentoo/portage /mnt/gentoo/portage-$TODAY.sqfs -noappend -no-progress 2>&1 | ansimail -s "$0" "$MAILTO" # append personal overlay (can skip) mksquashfs /var/lib/layman/.squash /mnt/gentoo/portage-$TODAY.sqfs -no-progress 2>&1 | ansimail -s "$0" root "$MAILTO" # append eix databases (can skip) mksquashfs /mnt/gentoo/eix /mnt/gentoo/portage-$TODAY.sqfs -no-progress -keep-as-directory 2>&1 | ansimail -s "$0" "$MAILTO" # switch mount symlinks ln -sfn /mnt/gentoo/portage-$TODAY.sqfs /mnt/gentoo/portage-current.sqfs chmod 755 /mnt/gentoo/portage-$TODAY.sqfs # rock'n'roll another week mount /mnt/gentoo/portage.sqfs
Oh yes, we can have reiser4 fs, as I myself use it. The portage tree is 60MB, and calculating dependencies is much faster.
Jude, that's great your install comes out so nicely. Maybe I'm having problems because I'm using a hardened stage4. There is no alternative for uclibc on arm. Do you have a libgcc_s.so.1? What is the contents of your /etc/ld.so.conf? BTW, I like your script idea a lot.
(In reply to comment #30) > Jude, that's great your install comes out so nicely. Maybe I'm having > problems because I'm using a hardened stage4. There is no alternative for > uclibc on arm. > > Do you have a libgcc_s.so.1? What is the contents of your /etc/ld.so.conf? > > BTW, I like your script idea a lot. ld.so.conf contains only: /usr/local/lib I do not have a libgcc in /lib. The system boots just fine without a single error. Do you want my minimal working qemu image to compare your results with?
I'm clearing up the errors and I think things will go more smoothly the next time I attempt this. I've been up against this for awhile when trying to start jackd: jackd: symbol 'clock_nanosleep': can't resolve symbol could not open driver .so '/usr/lib/jack/jack_dummy.so': (null) I don't know why this happens because the following files in both / and /mounted/ reference clock_nanosleep: lib/librt.so.0 usr/lib/librt.a I get: # ldconfig ldconfig: You should remove `/lib' from `/etc/ld.so.conf' ldconfig: You should remove `/usr/lib' from `/etc/ld.so.conf' # cat /etc/ld.so.conf # ld.so.conf autogenerated by env-update; make all changes to # contents of /etc/env.d directory /lib /usr/lib /usr/local/lib Do you know what I need to fix? Also, this could be a good addition to your make.conf: FEATURES="noman noinfo nodoc"
(In reply to comment #4) > (In reply to comment #3) > > Sir, I think we should include only the bare minimum in order to get portage > > working, minus the portage tree. That is what the current stages already > > provide from what I understand, but we need a much smaller version. > > I'd need a more specific package list. Maybe let's start from what we have > and slim down. Can you take a look at > > http://mirrors.rit.edu/gentoo/experimental/amd64/uclibc/ > > and > > http://mirrors.rit.edu/gentoo/experimental/x86/uclibc/ > > which are the stages I'm currently maintaining and let me know what you > would like removed? Those stages are coming in at ~100MB but a lot of that > is stuff under /usr/share which could be dumped. [from README] xfce is in there! is that true? i will be able to check after this friday, but that should be the case i think i would suggest to put a gui-free stage out there. so categories would be: toolchain emerge, and its deps of course (bash, python) minimum env (possibly with USE='-busybox' however[1], i will provide a list for this later) [1] this might be dealt with better in profiles since it is default for uclibc. i am not sure why?
(In reply to comment #33) > (In reply to comment #4) > > (In reply to comment #3) > > > Sir, I think we should include only the bare minimum in order to get portage > [from README] xfce is in there! is that true? i will be able to check after > this friday, but that should be the case i think i would suggest to put a > gui-free stage out there. There are gui free stages there. I'm push out a fresh stage3 every month. Regarding the xfce image, yes its in there, but be careful and don't update glib because there is an issue in the implementation of uclibc-0.9.33.2's eventfd which breaks >glib-2.30.3. It is a known issue and the next release of uclibc has the fix. I asked the uclibc people to push another maintenance release and when they do, I'll remaster that image, or I'll backport the fix. Besides this, the desktop environment is excellent. I love it, othewise I would not have poluted the /experimental with it. Sorry that was off topic.
I did poke around the uclibc stages and they were cool and all, but I did not worked out how the locales suppose to work there. uclibc support utf8 but I am unable to input unicode because of the locale variables, any idae how to handle it? Also the ldconfig messages like: ldconfig: You should remove `/lib' from `/etc/ld.so.conf' ldconfig: You should remove `/usr/lib' from `/etc/ld.so.conf' Are kind of weird.
(In reply to comment #35) > I did poke around the uclibc stages and they were cool and all, but I did > not worked out how the locales suppose to work there. uclibc support utf8 > but I am unable to input unicode because of the locale variables, any idae > how to handle it? > > Also the ldconfig messages like: > ldconfig: You should remove `/lib' from `/etc/ld.so.conf' > ldconfig: You should remove `/usr/lib' from `/etc/ld.so.conf' > Are kind of weird. I'm using the breakout library dev-libs/libiconv because uclibc's locale support sucks. I myself haven't explored how locales work on those images since I just stick with the default. Take a look at http://www.gnu.org/software/libiconv/ for more information.
