Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 887273 - dev-util/google-perftools has broken largepages* USE flags (dead code)
Summary: dev-util/google-perftools has broken largepages* USE flags (dead code)
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Bernard Cafarelli
URL:
Whiteboard:
Keywords: PATCH
Depends on:
Blocks:
 
Reported: 2022-12-20 00:05 UTC by Alec Ari
Modified: 2024-02-15 20:50 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
new ebuild: largepages and largepages64k fixed (google-perftools-2.10-r2.ebuild,2.95 KB, text/plain)
2022-12-20 00:43 UTC, Alec Ari
Details
diff of new ebuild vs upstream (largepages.patch,693 bytes, patch)
2022-12-20 00:45 UTC, Alec Ari
Details | Diff
Add 16/32/64 largepages support (full, correct patch) (0001-Add-google-perftools-with-working-largepages-16-32-6.patch,8.36 KB, patch)
2022-12-20 01:08 UTC, Alec Ari
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alec Ari 2022-12-20 00:05:41 UTC
I covered this problem on the gperftools Github page:

https://github.com/gperftools/gperftools/issues/1373

The ebuild currently uses:

use largepages && append-cppflags -DTCMALLOC_LARGE_PAGES
use largepages64k && append-cppflags -DTCMALLOC_LARGE_PAGES64K

however, TCMALLOC_LARGE_PAGES only exists now in ChangeLog.old

The way to properly set the page size is to use either the ./configure --with-tcmalloc-pagesize= flag, or in this case, as we use CMake, use TCMALLOC_PAGE_SIZE_SHIFT

Upon examining the INSTALL file in gperftools, I found:

*** TCMALLOC LARGE PAGES: TRADING TIME FOR SPACE

You can set a compiler directive that makes tcmalloc faster, at the
cost of using more space (due to internal fragmentation).

Internally, tcmalloc divides its memory into "pages."  The default
page size is chosen to minimize memory use by reducing fragmentation.
The cost is that keeping track of these pages can cost tcmalloc time.
We've added a new flag to tcmalloc that enables a larger page size.
In general, this will increase the memory needs of applications using
tcmalloc.  However, in many cases it will speed up the applications
as well, particularly if they allocate and free a lot of memory. We've
seen average speedups of 3-5% on Google applications.

To build libtcmalloc with large pages you need to use the
--with-tcmalloc-pagesize=ARG configure flag, e.g.:

   ./configure <other flags> --with-tcmalloc-pagesize=32

The ARG argument can be 4, 8, 16, 32, 64, 128 or 256 which sets the
internal page size to 4K, 8K, 16K, 32K, 64K, 128K and 256K respectively.
The default is 8K.

And in configure.ac:

AC_ARG_WITH([tcmalloc-pagesize],
            [AS_HELP_STRING([--with-tcmalloc-pagesize],
                            [Set the tcmalloc internal page size to 4K, 8K, 16K, 32K, 64K, 128K or 256K])],
            [],
            [with_tcmalloc_pagesize=$default_tcmalloc_pagesize])

[...]

case "$with_tcmalloc_pagesize" in
  4)
       AC_DEFINE(TCMALLOC_PAGE_SIZE_SHIFT, 12);;
  8)
       #Default tcmalloc page size.
       ;;
  16)
       AC_DEFINE(TCMALLOC_PAGE_SIZE_SHIFT, 14);;
  32)
       AC_DEFINE(TCMALLOC_PAGE_SIZE_SHIFT, 15);;
  64)
       AC_DEFINE(TCMALLOC_PAGE_SIZE_SHIFT, 16);;
  128)
       AC_DEFINE(TCMALLOC_PAGE_SIZE_SHIFT, 17);;
  256)
       AC_DEFINE(TCMALLOC_PAGE_SIZE_SHIFT, 18,
                 [Define internal page size for tcmalloc as number of left bitshift]);;
  *)
       AC_MSG_WARN([${with_tcmalloc_pagesize}K size not supported, using default tcmalloc page size.])
esac
Comment 1 Alec Ari 2022-12-20 00:43:24 UTC
Created attachment 843897 [details]
new ebuild: largepages and largepages64k fixed

I verified this change by hard-coding -DTCMALLOC_PAGE_SIZE_SHIFT=64 (absolutely ridiculous) to try and get a compiling error. It said something was too large or whatever and threw an error lol.
Comment 2 Alec Ari 2022-12-20 00:45:22 UTC
Created attachment 843899 [details, diff]
diff of new ebuild vs upstream

This is the patch for the ebuild itself
Comment 3 Alec Ari 2022-12-20 01:08:27 UTC
Created attachment 843901 [details, diff]
Add 16/32/64 largepages support (full, correct patch)

The USE flag for "largepages" (not largepages64k) in my previous patch was actually smaller than the default size, so let's just make it clear as day. Add the ability to specify 16, 32 and 64k largepages. metadata.xml is also patched

Everything all in one here, this is the correct patch
Comment 4 Alec Ari 2023-05-22 03:27:44 UTC
Just coming back to this.. Can we merge this fix for 16/32/64k pages?
Comment 5 Bernard Cafarelli gentoo-dev 2024-02-15 20:11:52 UTC
Sorry for the delay, I have been testing this change locally, and now that I took over the package, will merge it with version bump (changing to configure calls if possible)
Comment 6 Larry the Git Cow gentoo-dev 2024-02-15 20:50:23 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=50e5705ceb615fddd35e4ea300fe3418bea6d53a

commit 50e5705ceb615fddd35e4ea300fe3418bea6d53a
Author:     Bernard Cafarelli <voyageur@gentoo.org>
AuthorDate: 2024-02-15 20:49:55 +0000
Commit:     Bernard Cafarelli <voyageur@gentoo.org>
CommitDate: 2024-02-15 20:50:19 +0000

    dev-util/google-perftools: add 2.15
    
    Fix large page configure switch
    
    Closes: https://bugs.gentoo.org/887273
    Signed-off-by: Bernard Cafarelli <voyageur@gentoo.org>

 dev-util/google-perftools/Manifest                 |   1 +
 .../google-perftools/google-perftools-2.15.ebuild  | 118 +++++++++++++++++++++
 dev-util/google-perftools/metadata.xml             |  20 ++++
 3 files changed, 139 insertions(+)