Summary: | sys-apps/sandbox: makes unpacking much slower | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Sam James <sam> |
Component: | Sandbox | Assignee: | Sandbox Maintainers <sandbox> |
Status: | CONFIRMED --- | ||
Severity: | critical | CC: | josef64, kocelfc, mattst88, pacho, zhuyifei1999 |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=910273 https://bugs.gentoo.org/show_bug.cgi?id=925032 https://bugs.gentoo.org/show_bug.cgi?id=447970 |
||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: | emerge --info |
Description
Sam James
2024-02-20 02:17:27 UTC
With /var/tmp/portage on zram (w/ zstd), I get: ``` $ hyperfine "sandbox 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -" 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -' hyperfine \ "sandbox 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -'" \ 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -' Benchmark 1: sandbox 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -' Time (mean ± σ): 36.998 s ± 23.093 s [User: 5.517 s, System: 33.107 s] Range (min … max): 27.402 s … 102.629 s 10 runs Warning: The first benchmarking run for this command was significantly slower than the rest (102.629 s). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run. Benchmark 2: xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof - Time (mean ± σ): 9.193 s ± 1.781 s [User: 3.641 s, System: 9.261 s] Range (min … max): 8.366 s … 14.213 s 10 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Summary xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof - ran 4.02 ± 2.63 times faster than sandbox 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -' ``` The first run was much slower, which is consistent with what I've seen in-the-wild, presumably because of caches (I could use the hyperfine options for it blah blah but I don't really think it would demonstrate anything here). While I appreciate that some slowdown is kind of unavoidable for what sandbox does, it feels like a huge waste and for the overhead to be ~2-4x (in reality, it's really 4, given nobody is unpacking and then unpacking repeatedly, and it seems to consistently be around that for me on cold runs) to be excessive. Created attachment 885479 [details]
emerge --info
I think I might be hitting two distinct issues here, actually. 1. sandbox adds some not-insignificant amount of time to tar extraction 2. ext4 on zram tmpfs has some sort of pathological inode allocation problem? When the profile is good, the hot paths are all in liblzma, as you'd expect. When it's bad, it's in tar in ext4's find_inode_bit/find_get_block/ext4_create/ext4_new_inode. If I change the zram tmpfs to xfs, things become way happier, the peak is way lower, but I also can't reproduce the serious spikes either. So, I think we might be able to put 2. down to a kernel bug? But for 1., it's still there, just less significant than before: ``` # hyperfine "sandbox 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -'" 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -' Benchmark 1: sandbox 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -' Time (mean ± σ): 12.101 s ± 1.613 s [User: 4.886 s, System: 9.578 s] Range (min … max): 7.578 s … 13.040 s 10 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Benchmark 2: xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof - Time (mean ± σ): 4.271 s ± 0.095 s [User: 3.430 s, System: 3.210 s] Range (min … max): 4.158 s … 4.514 s 10 runs Summary xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof - ran 2.83 ± 0.38 times faster than sandbox 'xz -T32 -d -c -- /var/cache/distfiles/gcc-14-20240218.tar.xz | tar xof -' ``` 2x is still a pretty big differnce, but there's not this huge range of several minutes anymore. Also, the profile w/ the xfs zram tmpfs is way more like you'd expect - some in the kernel, but a healthy mix of liblzma + tar + sandbox in tar. |