| Summary: | sys-devel/llvm-roc-3.0.0 - ld: lib/Object/CMakeFiles/LLVMObject.dir/ArchiveWriter.cpp.o: .../work/llvm-project-roc-ocl-3.0.0/llvm/include/llvm/Support/raw_ostream.h:112: undefined reference to `llvm::raw_string_ostream::current_pos() const' | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Jan Ziak (atomsymbol) <0xe2.0x9a.0x9b> |
| Component: | Current packages | Assignee: | Gentoo Toolchain Maintainers <toolchain> |
| Status: | RESOLVED OBSOLETE | ||
| Severity: | normal | CC: | admnd, candrews, jarausch, llvm, ron, toolchain |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| URL: | https://gcc.gnu.org/PR93115 | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
sys-devel:llvm-roc-3.0.0:20191230-100531.log.xz
compile-ArchiveWriter.sh compile-ArchiveWriter-visibility.sh |
||
Hmmm, I'm using gcc 9.2.0 as well, and cannot reproduce this issue. Created attachment 601904 [details] compile-ArchiveWriter.sh (In reply to Craig Andrews from comment #1) > Hmmm, I'm using gcc 9.2.0 as well, and cannot reproduce this issue. Try one of these optimization options with gcc-9.2.0-r2: -O2 -fipa-cp-clone -O3 -fno-exceptions llvm-roc automatically adds -fno-exceptions, so only -O3 is necessary in make.conf for example. However, -O3 alone does not trigger the bug, -fno-exceptions has to be on gcc's command-line as well to trigger the bug. It should be possible to use the attached compile-ArchiveWriter.sh script to see that GCC 9.2 is adding the extra undefined symbol 'llvm::raw_string_ostream::current_pos() const' to the object file. Steps to reproduce: 1. Start emerging llvm-roc-3.0.0 2. Interrupt the build after it configures llvm-roc in /var/tmp/portage 3. Run the script compile-ArchiveWriter.sh I haven't tried gcc-9.2.0-r3, just gcc-9.2.0-r2. CC'ing toolchain@gentoo.org,llvm@gentoo.org: Any thoughts on this issue? It sounds like a bug in GCC? Perhaps sys-devel/llvm has already encountered it? (In reply to Jan Ziak (http://atom-symbol.net) from comment #2) > Created attachment 601904 [details] > compile-ArchiveWriter.sh > > (In reply to Craig Andrews from comment #1) > > Hmmm, I'm using gcc 9.2.0 as well, and cannot reproduce this issue. > > Try one of these optimization options with gcc-9.2.0-r2: > > -O2 -fipa-cp-clone > -O3 -fno-exceptions > > llvm-roc automatically adds -fno-exceptions, so only -O3 is necessary in > make.conf for example. However, -O3 alone does not trigger the bug, > -fno-exceptions has to be on gcc's command-line as well to trigger the bug. > > It should be possible to use the attached compile-ArchiveWriter.sh script to > see that GCC 9.2 is adding the extra undefined symbol > 'llvm::raw_string_ostream::current_pos() const' to the object file. > > Steps to reproduce: > 1. Start emerging llvm-roc-3.0.0 > 2. Interrupt the build after it configures llvm-roc in /var/tmp/portage > 3. Run the script compile-ArchiveWriter.sh > > I haven't tried gcc-9.2.0-r3, just gcc-9.2.0-r2. -r3 should not matter. Presence of undefined symbol alone should not be a manifestation of a problem as long as 'llvm::raw_string_ostream::current_pos() const' is emitted somewhere else. The common optimisation is to emit the out-of-line version of the symbol where virtual functions are defined. Let's see if I can reproduce it locally. (In reply to Sergei Trofimovich from comment #4) > Presence of undefined symbol alone should not be a manifestation of a > problem as long as 'llvm::raw_string_ostream::current_pos() const' is > emitted somewhere else. The common optimisation is to emit the out-of-line > version of the symbol where virtual functions are defined. 'llvm::raw_string_ostream::current_pos() const' is an inline method defined in the header file raw_ostream.h at line 512. I mentioned this, imprecisely though, in the initial post of this bug. (In reply to Jan Ziak (http://atom-symbol.net) from comment #5) > (In reply to Sergei Trofimovich from comment #4) > > Presence of undefined symbol alone should not be a manifestation of a > > problem as long as 'llvm::raw_string_ostream::current_pos() const' is > > emitted somewhere else. The common optimisation is to emit the out-of-line > > version of the symbol where virtual functions are defined. > > 'llvm::raw_string_ostream::current_pos() const' is an inline method defined > in the header file raw_ostream.h at line 512. I mentioned this, imprecisely > though, in the initial post of this bug. There are many definitions of "inline" in C++. Most of them don't require actual function to be always inlined. Especially when you use options like -fno-optimize-sibling-calls. Of good news I've reproduced failure locally with C{,XX}FLAGS=-O3 alone. (In reply to Sergei Trofimovich from comment #6) > There are many definitions of "inline" in C++. Most of them don't require > actual function to be always inlined. Especially when you use options like > -fno-optimize-sibling-calls. Yes. I meant the following kind of "inline": https://isocpp.org/wiki/faq/inline-functions#inline-member-fns-more I am using -fno-optimize-sibling-calls to get more accurate call-chain information for callgrind, gdb, perf. > Of good news I've reproduced failure locally with C{,XX}FLAGS=-O3 alone. Thanks. Tl;DR: I think it's an effect of -fvisibility-inlines-hidden that llvm-roc adds on it's own.
A few facts:
1. 'class llvm::raw_ostream' and 'class llvm::raw_string_ostream' from llvm-project-roc-ocl-3.0.0/llvm/include/llvm/Support/raw_ostream.h have a few virtual inlinable methods including current_pos.
2. some of virtual methods are not defined in header file (full class definition can't be inlined). For example 'virtual void llvm::raw_ostream::write_impl(const char *Ptr, size_t Size) = 0;'
3. some of virtual methods are only defined in raw_ostream.cpp file
4. raw_ostream.cpp is part of libLLVMSupport.so library
5. ArchiveWriter.cpp uses class raw_ostream and part of libLLVMObject.so
6. llvm-roc uses -fno-visibility-inlines-hidden to build libLLVMSupport.so (and other) objects
Why the failure happens mechanically:
When compiling ArchiveWriter.cpp in [5.] gcc notices that some of 'class raw_ostream' virtual definitions are missing in header file. This allows gcc not to emit out-of line definitions of inlinable methods and emit them only at definition site (when raw_ostream.cpp is compiled).
gcc relies on 'llvm::raw_string_ostream::current_pos()' definition to be provided by raw_ostream.cpp and libLLVMSupport.so to link it later into libLLVMObject.so
But -fvisibility-inlines-hidden hides all inlinable definitions from dynamic export table (to avoid PLT calls and apply more aggressive optimisations).
Thus we get a link failure.
-O2/-O3 behaviour mismatch is unfortunate but not unexpected. My guess is that gcc -O3 can more aggressively devirtualize 'llvm::raw_ostream::current_pos()' to 'llvm::raw_string_ostream::current_pos()' and get an out-of-line hidden reference in operator<< used locally.
I'd say it's not a gcc bug. I'll try to craft a small example and ask gcc upstream to verify. You might want to ask llvm-roc upstream as well.
Workarounds:
a) drop -fvisibility-inlines-hidden
b) add __attribute__((visibility("default")) to affected methods
Created attachment 602148 [details] compile-ArchiveWriter-visibility.sh (In reply to Sergei Trofimovich from comment #8) > Tl;DR: I think it's an effect of -fvisibility-inlines-hidden that llvm-roc > adds on its own. Thanks for looking into it. But it can't be caused by -fvisibility-inlines-hidden because the script compile-ArchiveWriter.sh isn't passing this option to the compiler. I updated the script and uploaded it as compile-ArchiveWriter-visibility.sh - the bottom half of the script is trying to find out whether non-default visibility is involved. It prints: Start visibility check 46333 __attribute__ ((visibility("hidden"))) 76558 MD_vcall_visibility = 29, End visibility check which should be sufficient indirect information to conclude that the bug isn't related to visibility, after examining the context of line 46333. -O2 and -O3 have no effect on visibility, which can be verified by running: $ gcc-9.2.0 -c --help=c++ -o /dev/null -Q -O0/O2/O3 | grep visibility -fvisibility-inlines-hidden -fvisibility-ms-compat [disabled] (In reply to Sergei Trofimovich from comment #8) > 6. llvm-roc uses -fno-visibility-inlines-hidden to build libLLVMSupport.so > (and other) objects I believe that point 6 is false. The output of command: $ zcat /var/log/portage/sys-devel:llvm-roc-3.0.0:20191231-234733.log.gz | grep no-visibility is empty. (In reply to Jan Ziak (http://atom-symbol.net) from comment #10) > (In reply to Sergei Trofimovich from comment #8) > > 6. llvm-roc uses -fno-visibility-inlines-hidden to build libLLVMSupport.so > > (and other) objects > > I believe that point 6 is false. The output of command: > > $ zcat /var/log/portage/sys-devel:llvm-roc-3.0.0:20191231-234733.log.gz | > grep no-visibility > > is empty. Don't know why I've added -fno-. It should have been: 6. llvm-roc uses -fvisibility-inlines-hidden to build libLLVMSupport.so (and other) objects. (In reply to Jan Ziak (http://atom-symbol.net) from comment #9) > Created attachment 602148 [details] > compile-ArchiveWriter-visibility.sh > > (In reply to Sergei Trofimovich from comment #8) > > Tl;DR: I think it's an effect of -fvisibility-inlines-hidden that llvm-roc > > adds on its own. > > Thanks for looking into it. > > But it can't be caused by -fvisibility-inlines-hidden because the script > compile-ArchiveWriter.sh isn't passing this option to the compiler. The script in itself does not manifest a bug. The real bug happened when llvm-project-roc-ocl-3.0.0/llvm/lib/Support/raw_ostream.cpp was compiled with -fvisibility-inlines-hidden. (In reply to Sergei Trofimovich from comment #12) > (In reply to Jan Ziak (http://atom-symbol.net) from comment #9) > > Created attachment 602148 [details] > > compile-ArchiveWriter-visibility.sh > > > > (In reply to Sergei Trofimovich from comment #8) > > > Tl;DR: I think it's an effect of -fvisibility-inlines-hidden that llvm-roc > > > adds on its own. > > > > Thanks for looking into it. > > > > But it can't be caused by -fvisibility-inlines-hidden because the script > > compile-ArchiveWriter.sh isn't passing this option to the compiler. > > The script in itself does not manifest a bug. The real bug happened when > llvm-project-roc-ocl-3.0.0/llvm/lib/Support/raw_ostream.cpp was compiled > with -fvisibility-inlines-hidden. My argument/reasoning in support of that it is a gcc bug is the following: 1. libLLVMSupport.so.10roc contains the implementation of 'llvm::raw_string_ostream::current_pos() const'. 2. Because of -fvisibility-inlines-hidden the symbol 'llvm::raw_string_ostream::current_pos() const' is local to the libLLVMSupport.so.10roc file (the symbol isn't exported). This is correct. 3. ArchiveWriter.cpp isn't part of libLLVMSupport.so.10roc. This is correct. 4. When compiling ArchiveWriter.cpp with -fvisibility-inlines-hidden and with the optimization option -fipa-cp-clone, the constant-propagation cloning pass enabled the compiler to determine that it is going to call 'llvm::raw_string_ostream::current_pos() const'. >>> The correctness of this call instruction is questionable, because the option -fvisibility-inlines-hidden is enabled and the compiler knows that the called function is a method that implicitly has the 'inline' keyword attached to it. <<< 5. Because of -fvisibility-inlines-hidden, if thanks to -fipa-cp-clone the compiler generates the instruction 'call llvm::raw_string_ostream::current_pos() const' it is bound to emit code for the called function and bound to define the local symbol 'llvm::raw_string_ostream::current_pos() const', since the worst possible scenario has to be assumed and the worst possible scenario is that the other compilation unit which created the instance of class llvm::raw_string_ostream, and which thus naturally contains the implementation of current_pos(), has also been compiled with -fvisibility-inlines-hidden. 6. The compiler didn't assume the worst case scenario. 7. The worst case scenario materialized when compiling llvm-roc. 8. Thus, the compiler should be patched to assume the worst case scenario. I agree it looks more like a gcc bug. I tried to craft a small example which would prevent gcc from emitting inline functions into every translation function and failed. Reducing complete example from llvm-roc now. Fun fact: gcc from master branch does not seem to exhibit the same behaviour (but it might be caused by unrelated changes). > Fun fact: gcc from master branch does not seem to exhibit the same behaviour > (but it might be caused by unrelated changes). gcc master changed it's behaviour in https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=commitdiff;h=40d8e1614ed0edfb9008f08b47571d76dbec4c7a """ commit 40d8e1614ed0edfb9008f08b47571d76dbec4c7a Author: jamborm <jamborm@138bc75d-0d04-0410-961f-82ee72b054a4> Date: Thu Sep 19 22:25:04 2019 +0000 New IPA-SRA """ Don't know if the fix was intended or the change is disruptive enough to upset the error condition. (In reply to Sergei Trofimovich from comment #15) > > Fun fact: gcc from master branch does not seem to exhibit the same behaviour > > (but it might be caused by unrelated changes). > > gcc master changed it's behaviour in > https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=commitdiff; > h=40d8e1614ed0edfb9008f08b47571d76dbec4c7a > > """ > commit 40d8e1614ed0edfb9008f08b47571d76dbec4c7a > Author: jamborm <jamborm@138bc75d-0d04-0410-961f-82ee72b054a4> > Date: Thu Sep 19 22:25:04 2019 +0000 > > New IPA-SRA > """ > > Don't know if the fix was intended or the change is disruptive enough to > upset the error condition. -fno-ipa-sra allowed me to reproduce the same failure on gcc-master. Filed upstream bug as https://gcc.gnu.org/PR93115. (In reply to Sergei Trofimovich from comment #16) > (In reply to Sergei Trofimovich from comment #15) > > > Fun fact: gcc from master branch does not seem to exhibit the same behaviour > > > (but it might be caused by unrelated changes). > > > > gcc master changed it's behaviour in > > https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;a=commitdiff; > > h=40d8e1614ed0edfb9008f08b47571d76dbec4c7a > > > > """ > > commit 40d8e1614ed0edfb9008f08b47571d76dbec4c7a > > Author: jamborm <jamborm@138bc75d-0d04-0410-961f-82ee72b054a4> > > Date: Thu Sep 19 22:25:04 2019 +0000 > > > > New IPA-SRA > > """ > > > > Don't know if the fix was intended or the change is disruptive enough to > > upset the error condition. > > -fno-ipa-sra allowed me to reproduce the same failure on gcc-master. Filed > upstream bug as https://gcc.gnu.org/PR93115. Thanks! Though I noticed that neither https://gcc.gnu.org/PR93115 nor the bug.cpp attachment there is mentioning -fno-ipa-sra, just -fipa-cp*. (In reply to Jan Ziak (http://atom-symbol.net) from comment #17) > Though I noticed that neither https://gcc.gnu.org/PR93115 nor the bug.cpp > attachment there is mentioning -fno-ipa-sra, just -fipa-cp*. -fipa-sra is an -O2 option. The https://gcc.gnu.org/PR93115 reproducer uses -O1 -fdevirtualize -fdevirtualize-speculatively -fipa-cp -fipa-cp-clone -fvisibility-inlines-hidden and thus avoid most of -O2 passes including -fipa-sra. *** Bug 711014 has been marked as a duplicate of this bug. *** Assigning to toolchain as this issue is in gcc Thanks! Exact same error in sys-devel/llvm-roc-3.7.0 (and 3.6.0 also).... Error gone on my side while building with gcc-10.2.0-r1 (previously tested with gcc 9.3.0-r1). Package is gone. |
Created attachment 601830 [details] sys-devel:llvm-roc-3.0.0:20191230-100531.log.xz .../ArchiveWriter.cpp.o: in function `llvm::raw_ostream::tell() const': .../raw_ostream.h:112: undefined reference to `llvm::raw_string_ostream::current_pos() const' This seems to be a gcc-9.2.0 bug. Build with gcc-8.3.0 succeeds. class raw_ostream { 112: uint64_t tell() const { return current_pos() + GetNumBytesInBuffer(); } virtual uint64_t current_pos() const = 0; }; class raw_string_ostream : public raw_ostream { uint64_t current_pos() const override { return OS.size(); } };