Creating a file on aufs mounted directory consumes 4096 bytes of disk space (tmpfs) which will not be returned after file deletion. Reproducible: Always Steps to Reproduce: Assume /var/lib is mounted using AUFS 1. df (to see free space) 2. touch /var/lib/test-file 3. rm /var/lib/test-file 4. df Actual Results: I have lost 4096 bytes even after file was deleted Expected Results: Disk space is being fully freed I do not have emerge command on the target machine. computer ~ # zcat /proc/config.gz | grep "AUFS" CONFIG_AUFS_FS=y CONFIG_AUFS_BRANCH_MAX_127=y # CONFIG_AUFS_BRANCH_MAX_511 is not set # CONFIG_AUFS_BRANCH_MAX_1023 is not set # CONFIG_AUFS_BRANCH_MAX_32767 is not set CONFIG_AUFS_SBILIST=y # CONFIG_AUFS_HNOTIFY is not set CONFIG_AUFS_RDU=y # CONFIG_AUFS_SP_IATTR is not set # CONFIG_AUFS_SHWH is not set CONFIG_AUFS_BR_RAMFS=y # CONFIG_AUFS_BR_FUSE is not set CONFIG_AUFS_BDEV_LOOP=y # CONFIG_AUFS_DEBUG is not set computer ~ # uname -a Linux computer 3.13.5-aufs #2 SMP Wed Mar 19 09:52:37 EDT 2014 x86_64 Intel(R) Atom(TM) CPU D2550 @ 1.86GHz GenuineIntel GNU/Linux computer ~ # df Filesystem 1K-blocks Used Available Use% Mounted on /dev/loop0 298240 298240 0 100% / etc-tmpfs 298240 298240 0 100% /aufs/etc var-tmpfs 298240 298240 0 100% /aufs/var/log varlib-tmpfs 298240 298240 0 100% /aufs/var/lib varspool-tmpfs 298240 298240 0 100% /aufs/var/spool home-tmpfs 298240 298240 0 100% /aufs/home/user etc-aufs 32768 128 32640 1% /etc var-aufs 32768 408 32360 2% /var/log varlib-aufs 16384 864 15520 6% /var/lib varspool-aufs 16384 1792 14592 11% /var/spool home-aufs 2560000 120868 2439132 5% /home/user tmpfs 404800 568 404232 1% /run udev 10240 0 10240 0% /dev shm 2023988 0 2023988 0% /dev/shm cgroup_root 10240 0 10240 0% /sys/fs/cgroup tmpfs 10240 4 10236 1% /var/lib/misc tmpfs 1048576 4 1048572 1% /tmp
Thanks for the report. I will test it and come back later. Perhaps you need to help me report this upstream.
Actually after I reported this bug here I did more research and found that upstream knows about this problem. This is maillist discussion about it (Nov 2012): http://sourceforge.net/p/aufs/mailman/aufs-users/thread/7069.1352289200%40jrobl/ This is same person posting it on ServerFault: http://serverfault.com/q/445445/145512 Apparently he discovered that issuing following command will release the space on immediate basis: # mount -o remount,itrunc_xino=0 /tmp/mnt And to prevent it from happening a mount option "trunc_xino" should be used. I am concerned on two facts: 1) This was known by upstream at least for two years and this is still an issue. 2) Is mounting with "trunc_xino" a real solution or just workaround? Or was it fixed on upstream and now we just need to fix our aufs-sources somehow?
(In reply to Nikolay from comment #2) > Actually after I reported this bug here I did more research and found that > upstream knows about this problem. Thanks for digging deeper into the problem. > > This is maillist discussion about it (Nov 2012): > http://sourceforge.net/p/aufs/mailman/aufs-users/thread/7069. > 1352289200%40jrobl/ It would be great if you can replay to this posts and add your problem there. Perhaps we will get some fixes from Junjiro. > > This is same person posting it on ServerFault: > http://serverfault.com/q/445445/145512 With this I can reproduce it. > > Apparently he discovered that issuing following command will release the > space on immediate basis: > # mount -o remount,itrunc_xino=0 /tmp/mnt > > And to prevent it from happening a mount option "trunc_xino" should be used. > > I am concerned on two facts: > > 1) This was known by upstream at least for two years and this is still an > issue. It is not clear to me if this is by design or really a bug. But perhaps a mail from you pointing to this report on the aufs-mailinglist could help clarifying this. > 2) Is mounting with "trunc_xino" a real solution or just workaround? > same as a above. It seems to me that it is an appropriate solution, but it is safer to ask on the mailing list again. > Or was it fixed on upstream and now we just need to fix our aufs-sources > somehow? No, our auf-sources always follow the latest upstream development.
New mail list conversation with aufs developer created. Anyone welcome to join and participate: http://sourceforge.net/p/aufs/mailman/message/32273921/
Justin, I have got confirmation from Mr. J. R. Okajima that usage of "trunc_xino" is official solution. Also he is willing to provide some patches. Would you mind to get those patches from him and see what you can do with them? http://sourceforge.net/p/aufs/mailman/message/32277452/ J. R. Okajima: > I'd suggest you to try specifying "trunc_xino" after reading aufs manual. > This is official solution. > If you don't care patching against tmpfs, I can write and send one to > improve the inode number assignment in tmpfs.
Created attachment 376036 [details] upstream patch Could you please try the patch and report if it works?
I tried, but I am getting malformed patch error. What should I patch it against? I tried to apply patch against aufs-sources and I got: computer linux # patch -p1 < a.patch patching file mm/shmem.c patch: **** malformed patch at line 25: @@ -182,6 + 183,7 @@ static struct backing_dev_info shmem_backing_dev_info __read_mostly = {
I did not manage to get patch working, so I manually patched aufs-sources line by line from patch we've got. With having original and modified sources I was able to make a patch file which emerge successfully applied to sources. Sources compiled well (I used aufs-sources-3.14.0). Now I am going to test the thing. I will post here once I will have results.
Created attachment 376082 [details, diff] tmpfs patch to enable "ino" mount option tmpfs patch provided by J. R. Okajima. This patch enables new tmpfs mount option "ino" which prevents disappearance of free tmpfs disk space while it is being used as aufs branch.
Provided patch works. Free disk space does not disappear anymore.
J. R. Okajima: > Glad to see the effect of "ino". > But I found a problem in the patch. The internal bitmap should be freed > when unmounting tmpfs but I forgot it. > Now I am considering how to treat this patch. To put it into aufs > release, or to ask LKML to put into mainline. > Until when it is decided, I'd suggest you to keep using aufs > "trunc_xino" option instead of tmpfs "ino" option. Well, I guess it is better to wait little bit until it is resolved...
We will wait for upstream
Aufs creator J. R. Okajima found proper solution for this bug. We have successfully tested new patch and now it is going to be added to the next release. He wrote in the aufs mail list: "Thank you very much. New tmpfs-idr.patch will be included in next release. J. R. Okajima"
(In reply to Nikolay from comment #13) > Aufs creator J. R. Okajima found proper solution for this bug. We have > successfully tested new patch and now it is going to be added to the next > release. > > He wrote in the aufs mail list: > > "Thank you very much. > New tmpfs-idr.patch will be included in next release. > > > J. R. Okajima" Thank you very much for all the work with upstream. I already included the patch in the gentoo release, but you need to apply them yourself.
(In reply to Justin Lecher from comment #14) > > Thank you very much for all the work with upstream. I already included the > patch in the gentoo release, but you need to apply them yourself. You are welcome! But will the necessity to apply the patch manually stay this way forever or at some point ebuild will apply this patch for me like it does with these three? aufs3-kbuild.patch aufs3-base.patch aufs3-mmap.patch I guess I will have to put it to /etc/portage/patches/sys-kernel/aufs-sources if ebuild will not take care of it.
I am not really sure whether it is good to apply it for everyone. Maybe with a USE but I need to think about it. This will give you the patch emerge aufs-sources cd /usr/src/linux bzcat /usr/share/doc/aufs-sources-3.15.0/tmpfs-idr.patch.bz | patch -p1
(In reply to Justin Lecher from comment #16) > I am not really sure whether it is good to apply it for everyone. Maybe with > a USE but I need to think about it. > > This will give you the patch > > emerge aufs-sources > cd /usr/src/linux > bzcat /usr/share/doc/aufs-sources-3.15.0/tmpfs-idr.patch.bz | patch -p1 Make sense. Manual patching is not that hard anyway, so it is fine. Thanks a lot.
Created attachment 379268 [details, diff] Final patch from J. R. Okajima Currently included patch tmpfs-idr.patch.bz is not the final one we used in testing and still causes the bug to occur (I tried aufs-sources 3.15.0-r1) This is the final patch which he sent to us in mail list while we were testing it: http://sourceforge.net/p/aufs/mailman/message/32472259/ It was intended for kernel 3.14. I confirm successful compilation on 3.15.0-r1 amd64 and x86 as well
