Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 471102 (lfs-tracker) - [TRACKER] handling of Large File System (LFS) in packages (typically 32bit/glibc userland)
Summary: [TRACKER] handling of Large File System (LFS) in packages (typically 32bit/gl...
Alias: lfs-tracker
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Gentoo Toolchain Maintainers
Keywords: PullRequest, Tracker
Depends on: 543374 549092 550482 715558 884007 891411 892237 893656 896086 896340 898494 898960 899890 899954 902473 902475 902479 902901 904190 911863 912087 919173 922642 923492 923801 923803 924163 924711 925305 930512 45608 109237 146951 300307 389299 471024 548962 550438 550502 550508 550708 559572 576396 583282 595296 629704 681866 687548 760848 760857 761100 819762 835349 847442 847955 852812 856349 878475 880077 884717 885535 894564 896316 898506 899952 906101 911176 911261 911262 912680 914505 915316 916575 922508 923802 925207 928360 930513
  Show dependency tree
Reported: 2013-05-23 19:23 UTC by SpanKY
Modified: 2024-06-04 17:38 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description SpanKY gentoo-dev 2013-05-23 19:23:43 UTC
tracker for packages which still use 32bit filesystem interfaces (stat/read/lseek/etc... instead of stat64/read64/lseek64/etc...).  feel free to ask general questions here, but specific bug reports should be in dedicated bugs (and listing them as blockers to this).

this has always been a problem.  typically though people only notice when they try to do with large files (like working with files over 2GiB in size).  but the stat() issue is going to be affecting people even more as filesystems (like NFS and XFS) start to deploy 64bit inodes.  now if you stat() a file on disk that happens to have a 64bit inode, you'll get an error (EOVERFLOW).

for many apps, the fix is simple: build with -D_FILE_OFFSET_BITS=64.  this will redefine funcs/types/structs from the 32bit version to the 64bit version (e.g. off_t will automatically turn into off64_t, stat() will become stat64(), etc...).

if you're building a library that exports off_t as part of its ABI (in a struct or func return/arg), this change will most likely change things.  in that case, you should consider defining:

then you can internally change the code to call stat64() and use off64_t without changing the sizes of the 32bit types (e.g. off_t is still off_t).

this will require more work though to make things portable (like checking if stat64() even exists) as not all C libraries/operating systems have taken this approach.

some random background links:
Comment 1 Michael Weber (RETIRED) gentoo-dev 2014-01-08 15:04:11 UTC
I hit this with ~amd64 current sys-devel/gcc:4.7 and :4.8 (running glibc-2.17)
Comment 2 SpanKY gentoo-dev 2014-01-08 19:40:16 UTC
(In reply to Michael Weber from comment #1)

i have no idea what this means.  you really need to provide actual details.
Comment 3 SpanKY gentoo-dev 2014-03-13 22:35:04 UTC
it's come up again:

this time i've posted a patch.  i'll probably include it in glibc-2.20 when it comes out to see what goes boom ...
Comment 4 LW 2014-03-21 07:20:57 UTC
Is it possible that the glusterfs network file system is always returning
inode numbers larger than 2**32?

Trying to rebuild gcc where /var/tmp/portage is a glusterfs share
from an XFS file system on the file server:

  configure: updating cache ./config.cache
   * ../../sandbox-2.6/libsandbox/libsandbox.c:check_syscall():879: failure   (Value too large for defined data type):
   * ISE:
        abs_path: (null)
        res_path: /var/tmp/portage/sys-devel/gcc-4.7.3-r1/work/build/x86_64-pc-linux-gnu/32/libgomp/conftest.val
  configure: error: unsupported system, cannot find sizeof (omp_lock_t)
  make[2]: *** [configure-stage1-target-libgomp] Error 1

Seems to be similar to the forum discussion

This smells like a stat problem, since the server side logs
many of these:
    [server-rpc-fops.c:1739:server_stat_cbk] 0-porttmp-server: 2676750: STAT
    (null) (adbcef83-56c6-4caf-b974-7b14796bde8c) ==> 
        (No such file or directory)


Something that might also be related is that
    emerge --buildpkg y <some-package> fails, e.g. sys-libs/tmezone-data
fails with a glusterfs file system for /var/tmp/portage, but
    emerge --buildpkg n <some-package> does not.

    >>> Completed installing timezone-data-2013d into /var/tmp/portage/sys-libs

    strip: x86_64-pc-linux-gnu-strip --strip-unneeded -R .comment -R
       .GCC.command.line -R
    ecompressdir: bzip2 -9 /usr/share/man
    tar: ./usr/share/zoneinfo/posix: file changed as we read it
    tar: ./usr/share/zoneinfo/right/Asia: file changed as we read it
    tar: ./usr: file changed as we read it

This also generates STAT (null) errors on the server side
Comment 5 Ryan Hill (RETIRED) gentoo-dev 2014-04-06 22:07:00 UTC
The first bit looks similar to bug #502202.
Comment 6 SpanKY gentoo-dev 2015-05-26 06:24:05 UTC
if you're using autotools, you can use one of these macros in your

the latter is a super set of the former.  which is appropriate depends entirely on the package in question.  the manual has more details:
Comment 7 SpanKY gentoo-dev 2015-05-26 06:35:18 UTC
(In reply to SpanKY from comment #6)

to clarify, the latter will enable _GNU_SOURCE which implicitly enables LFS support.  however, it does not turn on transparent LFS, so packages still have to use alternative function names like fseeko.

if you use AC_SYS_LARGEFILE, then the build will automatically enable transparent LFS which means you don't have to do anything else.

if you wish to use both, make sure you call AC_SYS_LARGEFILE after AC_USE_SYSTEM_EXTENSIONS otherwise you'll get warnings when you run autotools.
Comment 8 Mario Klebsch 2021-04-20 18:46:14 UTC
Ok, it seems that in glibc, EOVERFLOW can be returned by stat(), although the lstat64() call was successful.

Unfortunately, I cannot see any way, to work around this problem.
Comment 9 SpanKY gentoo-dev 2021-04-20 21:09:07 UTC
(In reply to Mario Klebsch from comment #8)

the fix is to build the app in question with LFS flags.  it's not a bug in glibc that this happens since the struct the app is using is too small.
Comment 10 Larry the Git Cow gentoo-dev 2021-05-17 21:00:28 UTC Comment hidden (obsolete)