LARGEFILE support in Gentoo currently is rather haphazard. The 'core' packages such as fileutils and such compile with LARGEFILE support by default while other packages (such as many GNOME and KDE packages, etc) do not. The downsides of having LARGEFILE support are nonexistant compared to the downsides of not having it enabled. 2GB files are common these days and LARGEFILE should be enabled for any and all executables on platforms that fully support it. The easiest solution may be to automatically add "-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE" to CFLAGS and CXXFLAGS for all packages within portage itself. Comments?
*** Bug 14353 has been marked as a duplicate of this bug. ***
i just did `emerge -e system` on a 1.4 and 1.2 system and both compiled w/out errors ... i guess ill know if there are any runtimes bugs in the next few days ;) but this is a good sign i think
I also believe that Gentoo needs to take a look at supporting largefiles better. I've noticed that some ebuilds (such as apache) do switch on the appropriate compiler flags, while others (such as less and sysklogd) do not. It seems to me that Gentoo, one of the more cutting-edge linux distributions (IMHO, at least) should offer better support for them. I would certainly be willing to contribute patches, etc. There was actually a decent article concerning largefile support posted on freshmeat recently: http://freshmeat.net/articles/view/709/ I feel that an issue brought up in the above freshmeat article should be addressed before working on this issue: " As a result, however, it is highly dangerous to use off_t in header files in largefile-sensitive systems. Most software writers do not expect that an integral type like off_t can change its size; it is just used in the exported interface, as in making a new call "off_t my_lseek(int,off_t,int)". " How much of a concern is this for us?
this bug is a continuation of -core discussion that originated around that freshmeat article
on glibc-2.2.5 systems, distcc has a problem ... gcc -march=i686 -O3 -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -W -Wall -Wshadow -Wpointer-arith -Wcast-align -DHAVE_CONFIG_H -I./src -c -o src/exec.o src/exec.c In file included from src/io.c:66: /usr/include/sys/sendfile.h:26: #error "<sys/sendfile.h> cannot be used with _FILE_OFFSET_BITS=64" so i guess any package on a glibc-2.2.5 system will fail when they goto utilize this header ... glibc-2.3.x has updated this header file to support 64 bit fopen's
the new strace (4.4.93) fails to emerge with -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE in CFLAGS ... ill e-mail the author of strace about the bug ... or just file something upstream ... this happens on both gcc-2.x and gcc-3.x, glibc-2.2.x and glibc-2.3.x ... source='mem.c' object='mem.o' libtool=no \ depfile='.deps/mem.Po' tmpdepfile='.deps/mem.TPo' \ depmode=gcc /bin/sh ./depcomp \ gcc -DHAVE_CONFIG_H -I. -I. -I. -Ilinux/i386 -I./linux/i386 -Ilinux -I./linux -Wall -D_GNU_SOURCE -march=i686 -O3 -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -c `test -f 'mem.c' || echo './'`mem.c io.c: In function `sys_sendfile': io.c:279: warning: long unsigned int format, different type arg (arg 2) io.c: At top level: io.c:309: redefinition of `sys_pread' io.c:236: `sys_pread' previously defined here io.c:327: redefinition of `sys_pwrite' io.c:254: `sys_pwrite' previously defined here
latest xmms fails with latest glibc (2.3.2) ... the file /usr/include/fts.h cannot be used when enabling large file support In file included from util.c:26: /usr/include/fts.h:41:3: #error "<fts.h> cannot be used with -D_FILE_OFFSET_BITS==64" make[4]: *** [util.o] Error 1 make[4]: Leaving directory `/var/tmp/portage/xmms-1.2.7-r19/work/xmms-1.2.7/xmms'
i added filter-flags to xmms ebuild (all glibc/gcc vers) and the distcc ebuild (gcc-2.x ver) i patched strace to fix the prototyping ... turned out that the prototype was defined with 'off_t' while the func used 'long int' ...
Hmm... Any new developments on this front? Personally, I'm interested in having this option apply to Apache (via a "largefile" USE flag, perhaps), but obviously the problem is broader than just web servers... Thanks.
Here's a narrow case forum message detailing what I could gather was the current state of this issue (apologies if it is mischaracterized). http://forums.gentoo.org/viewtopic.php?t=186550 (I'm NightMonkey)
Any movement on this? Thanks!
Can this be re-classified out of "Unclassified (old)"? This is still a present problem.
OK, well, since there's no recent response from the development team on this bug, perhaps it should be closed as CLOSED:WONTFIX?
How do you want to solve it? Is it enough to add the D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE flag to the example make.conf files?
I believe that currently there is no coherent handling of this across ebuilds. Some ebuilds add the gcc flag, some do not. Some actively strip the LFS flags from CFLAGS - which, at least in >=apache-2.0.49 and >=mod_php-2.3.9, is no longer necessary. What I am not sure about is the stability impact of having the LFS flags added to CFLAGS in /etc/make.conf and used for the entire build history of a system (I have only used it for Apache and mod_php). So, in order to allow users to choose and make an informed decision, I think that a "lfs" or "largefile" USE flag would be key to making the choice in a "Gentoo" way. Then, each ebuild can be configured to add the LFS flags, or, in the case of known incompatibilities, issue warnings.
I think the user can make the descicion in /etc/make.conf, so no ebuild needs to be changed for this, just the defaults. We can filter the flag if it does not work.
No... he is saying we should add them cursory to what the user supplies in CFLAGS... I think the problem is that 1) we generally think that forcing anything is a bad idea 2) the wrong team has the bug 3) it really is a bug in the package which should be sent upstream
Also, I have built *all* of my systems with these flags in my CFLAGS and they work fine
So, the proper thing to do is to file individual bugs for every package that doesn't currently support largefile? Also, other than lack of testing, is there any reason NOT to have largefile support in every package?
Actually, as I said, it really should be enabled upstream, so definitely do not file individual bugs. I see no problem with enabling it by default. Perhaps we should reassign this to the portage team rather than base-system.
re Comment 18 i build all my system with these flags to make sure they're ok ... that means ive built amd64/arm/mips/hppa/ppc/sparc/x86 systems and they all work just fine for me re ebuilds pruning LFS flags that is unavoidable in some cases (glibc-2.2.x's sendfile.h is NOT LFS safe for example)
Removing release@ as this really has nothing to do with the releases themselves. It is really starting to seem instead like this should be implemented as a FEATURE perhaps? FEATURES="largefile" which auto-appends the LFS stuff to CLFAGS? That way ebuilds could still filter them, if they break the ebuild.
As a FEATURE or USE, it would be the same to me, from a user perspective. However, what I'm not sure about is how much awareness of FEATURES vis a vis USE is prevalent among Gentoo application packagers. My concern is centered around whether or not application packagers think as much about the impact of differening FEATURES settings vs. the impact of "USE" flags, and how much they might test the effects of USE flag changes vs. FEATURES changes. Am I making sense?
Over two years and no resolution on this... And the forum posts continue (just search on "largefile", or "file size exceeded"). I think Chris' idea of implementing this as a FEATURE looks great. But, whether the resolution is FEATURES or a new USE flag, something should be implemented! >2GB files are not that uncommon these days...
Sad that this has never gone anywhere, sounds easy to impement though. It seems to me a USE flag for it would make more sense from the users point of view as FEATURES are for how portage behaves, not what actually gets compiled into the package. Is there a dev watching this bug who knows portage who could submit a patch for this? If not I'll take a look once I have time, I just don't know anything about the portage code.
Is there any way for this to be re-classified out of "Unclassified (old)", please? And can we get a decision on this from the Uber-Devs, yea or nay? The fracturing among packages and their handling of LFS (and its associated flags) appears to be getting worse and is curently unmanaged and unpredictable. Thank you.
A decision to reverse and de-support existing LFS support in Apache is being executed now, due to problems not with Apache, but with other masked packages, apr and apr-utils. Were an LFS USE flag in place, this decsion would be made by the end-user, rather than having to ham-handedly remove functionality from a software package. How do we get greater attention to this glaring issue? Open and unresolved for over 2 years now?
Can someone PLEASE move this bug to the appropriate component group? Thanks.
Bug marked "NEW" for over 2 1/2 years? Some very cogent solutions have been proposed in this bug...
I've recently come to realize how annoying this is...didn't realize it has been going on for such an extended period with virtually no change.
At this point, I think that the FEATURES="largefile" option is not the best, as the flexibility of USE="lfs" has become more apparent. Especially in light of the new abilities (well, new since this bug was really actively discussed) of Portage and setting individual USE flag options in /etc/portage/package.use, a USE flag would be the best, in my opinion. In addition, if there are pacakges that are broken upstream for LFS, then the package maintainers could just not support the LFS USE flag at all in their package, and add functionality to new package versions when upstream has released a fixed version (AKA Apache). Of course, for me personally, Apache and PHP LFS support was a primary (though not the only) reason for my wanting this added, but the Apache team has removed LFS support for now from Apache proper due to an upstream problem with apr. But, if someone out there _does not_ want LFS support on a package that supports it, for whatever reason, well, that should be their choice. This is Gentoo, after all. :)
OK, I think I've come around to a different conclusion, after some very enlightening discussions and work with vericgar in #gentoo-apache. It appears that the handling of the implementation of LFS support is moving more and more into upstream development, rather than optional CFLAGS. For instance, he discovered that the developmental apr-1.x ./configure has these options now: <vericgar> from ./configure --help for apr: --disable-lfs Disable large file support on 32-bit platforms And that gives these lines in the configure script: <vericgar> Check for compiler flags... <vericgar> checking whether to enable -D_LARGEFILE64_SOURCE... yes <vericgar> adding "-D_LARGEFILE64_SOURCE" to CPPFLAGS <vericgar> that's from running the ./configure <vericgar> so it will be taken care of automaticly So, this development would make LFS USE flags and FEATURES="lfs" irrelevant, for the most part, unless there are a ton of other packages that need explicit CFLAG settings, which doesn't appear to be true. About 5-6 packages have hard-coded the -D_LARGEFILE_SOURCE and related CFLAGS into their pacakges, but, significantly, none of these are multimedia apps, which, if upstream weren't handling it, would already have filled bugzilla with complaints. So, I guess that this bug can probably be closed, at least from my point of view. Anyone else see flaws in this logic? I'll miss this bug... I'll be lonely without it. ;)
ecasound is another package that needs --enable-largefile for >2gb files.
I think this should be closed, can someone from base-system please do so?
I fail to see what's requested here... This can't be done "coherently", nor base-system can solve it, nor there's any reason for lfs use flag or whatever. Either is works and then it should be enabled by default, or it doesn't... E.g., <apache-2.2 will never work with LFS. File bugs upstream or to ebuild maintainers for individual packages if they are missing LFS support. Closing this one.
