Opened 2 years ago

Last modified 19 months ago

#65246 new defect

clang-14: crashes occurring for multiple ports

Reported by: mascguy (Christopher Nielsen) Owned by:
Priority: Normal Milestone:
Component: ports Version: 2.7.2
Keywords: Cc: cjones051073 (Chris Jones), kencu (Ken), cooljeanius (Eric Gallager)
Port: clang-14 llvm-14 darktable libopenraw

Description (last modified by mascguy (Christopher Nielsen))

When compiling darktable 3.8.1, the following crash occurs, for source file src/common/iop_profile.c. Build log attached.

1.  <eof> parser at end of file
2.  Code generation
3.  Running pass 'Function Pass Manager' on module 'work/darktable-3.8.1/src/common/iop_profile.c'.
4.  Running pass 'Debug Variable Analysis' on function '@dt_ioppr_transform_image_colorspace'
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  libLLVM.dylib            0x000000010a0eea8b llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 43
1  libLLVM.dylib            0x000000010a0ed928 llvm::sys::RunSignalHandlers() + 248
2  libLLVM.dylib            0x000000010a0ee040 llvm::sys::CleanupOnSignal(unsigned long) + 208
3  libLLVM.dylib            0x000000010a00a06f CrashRecoverySignalHandler(int) + 191
4  libsystem_platform.dylib 0x00007fff95c38b3a _sigtramp + 26
5  libsystem_platform.dylib 0x00007fff58ebd930 _sigtramp + 18446744072688782864
6  libLLVM.dylib            0x000000010a3e9371 (anonymous namespace)::LDVImpl::runOnMachineFunction(llvm::MachineFunction&, bool) + 5457
7  libLLVM.dylib            0x000000010a4ac0b4 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 308
8  libLLVM.dylib            0x000000010a24ae2a llvm::FPPassManager::runOnFunction(llvm::Function&) + 922
9  libLLVM.dylib            0x000000010a251383 llvm::FPPassManager::runOnModule(llvm::Module&) + 67
10 libLLVM.dylib            0x000000010a24b449 llvm::legacy::PassManagerImpl::run(llvm::Module&) + 937
11 libclang-cpp.dylib       0x0000000107db85db clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, std::__1::unique_ptr<llvm::raw_pwrite_stream, std::__1::default_delete<llvm::raw_pwrite_stream> >) + 2395
12 libclang-cpp.dylib       0x00000001080ecb98 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) + 1928
13 libclang-cpp.dylib       0x0000000106ebfe63 clang::ParseAST(clang::Sema&, bool, bool) + 643
14 libclang-cpp.dylib       0x0000000108988dfa clang::FrontendAction::Execute() + 90
15 libclang-cpp.dylib       0x00000001088f1616 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 902
16 libclang-cpp.dylib       0x00000001089ff5f8 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 680
17 clang                    0x0000000106d42e53 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) + 2147
18 clang                    0x0000000106d41036 ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&) + 310
19 libclang-cpp.dylib       0x000000010853c707 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, bool*) const::$_1>(long) + 23
20 libLLVM.dylib            0x000000010a009dc2 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) + 226
21 libclang-cpp.dylib       0x000000010853c295 clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, bool*) const + 389
22 libclang-cpp.dylib       0x000000010850314f clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&) const + 415
23 libclang-cpp.dylib       0x000000010850364c clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::__1::pair<int, clang::driver::Command const*> >&) const + 124
24 libclang-cpp.dylib       0x000000010851f11c clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::__1::pair<int, clang::driver::Command const*> >&) + 396
25 clang                    0x0000000106d409b4 main + 10820
26 libdyld.dylib            0x00007fff95a29235 start + 1
27 libdyld.dylib            0x0000000000000088 start + 18446603338005704276
clang: error: clang frontend command failed with exit code 139 (use -v to see invocation)
clang version 14.0.3
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /opt/local/libexec/llvm-14/bin

Similarly, a link-time crash occurs with libopenraw. Build log attached.

  = note: 0  0x104026b51  __assert_rtn + 144
          1  0x104057063  ld::Fixup::Fixup(unsigned int, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 107
          2  0x10402f4e9  mach_o::relocatable::Parser<x86_64>::addFixup(mach_o::relocatable::Parser<x86_64>::SourceLocation const&, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 33
          3  0x10402a751  mach_o::relocatable::Section<x86_64>::addRelocFixup(mach_o::relocatable::Parser<x86_64>&, macho_relocation_info<Pointer64<LittleEndian> > const*) + 1471
          4  0x104052bb7  mach_o::relocatable::Section<x86_64>::makeFixups(mach_o::relocatable::Parser<x86_64>&, mach_o::relocatable::Parser<x86_64>::CFI_CU_InfoArrays const&) + 91
          5  0x10404e7c6  mach_o::relocatable::Parser<x86_64>::parse(mach_o::relocatable::ParserOptions const&) + 2034
          6  0x1040320fd  mach_o::relocatable::Parser<x86_64>::parse(unsigned char const*, unsigned long long, char const*, long, ld::File::Ordinal, mach_o::relocatable::ParserOptions const&) + 375
          7  0x1040629cc  archive::File<x86_64>::makeObjectFileForMember(archive::File<x86_64>::Entry const*) const + 758
          8  0x104062518  archive::File<x86_64>::justInTimeforEachAtom(char const*, ld::File::AtomHandler&) const + 122
          9  0x10407819b  ld::tool::InputFiles::searchLibraries(char const*, bool, bool, bool, ld::File::AtomHandler&) const + 215
          10  0x104080c90  ld::tool::Resolver::resolveUndefines() + 160
          11  0x104082f63  ld::tool::Resolver::resolve() + 79
          12  0x1040276c7  main + 689
          A linker snapshot was created at:
          /tmp/build_script_build-0fd30e212c66cd59-2022-05-30-185930.ld-snapshot
          ld: Assertion failed: (name != NULL), function Fixup, file /SourceCache/ld64/ld64-241.9/src/ld/ld.hpp, line 510.
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

Attachments (4)

darktable-devel-build-10.12-clang-14-crash.log.xz (40.6 KB) - added by mascguy (Christopher Nielsen) 2 years ago.
libopenraw-build-10.9-clang-14-crash.log (236.7 KB) - added by mascguy (Christopher Nielsen) 2 years ago.
libopenraw-build-10.9-clang-13-crash-link.log (236.9 KB) - added by mascguy (Christopher Nielsen) 2 years ago.
coreutils-clang-15-crash-10.6_x64.log.gz (39.5 KB) - added by mascguy (Christopher Nielsen) 19 months ago.

Download all attachments as: .zip

Change History (37)

Changed 2 years ago by mascguy (Christopher Nielsen)

comment:1 Changed 2 years ago by mascguy (Christopher Nielsen)

Of note, this occurs on multiple macOS releases, and doesn't appear to be affected by the macOS version.

comment:2 Changed 2 years ago by mascguy (Christopher Nielsen)

Also, compiling with Clang 13 works just fine.

comment:3 Changed 2 years ago by Christopher Nielsen <mascguy@…>

In 642a988efbf587db10145e7ed95133a513f9b78a/macports-ports (master):

darktable/darktable-devel: blacklist macports clang-14 for now, due to compilation crash
See: #65246

comment:4 Changed 2 years ago by mascguy (Christopher Nielsen)

Port: darktable darktable-devel added

comment:5 Changed 2 years ago by mascguy (Christopher Nielsen)

PR created, for llvm-14 update to 14.0.4:

https://github.com/macports/macports-ports/pull/14996

comment:6 Changed 2 years ago by mascguy (Christopher Nielsen)

Unfortunately this crash still occurs, even with LLVM 14.0.4.

Are there any additional upstream patches that we also need to apply...?

comment:7 Changed 2 years ago by kencu (Ken)

Usually what you would now is see if it is fixed in clang-devel.

If it is, then --> figure out why/which commit / which file (and ideally report it upstream for possible backports).

If it is not, then --> report it upstream with the files the error says to include with the report.

comment:8 Changed 2 years ago by kencu (Ken)

at least it is simpler now, as I believe it is all just github issues, and no longer the more complex bug tracker they've had all the previous years.

comment:9 Changed 2 years ago by Christopher Nielsen <mascguy@…>

In d9e7e218242f34605d9fc13f590e52d35c43e192/macports-ports (master):

llvm-14: update to 14.0.4
See: #65246

comment:10 Changed 2 years ago by mascguy (Christopher Nielsen)

Summary: clang-14 @14.0.3: crash during compilation of darktable 3.8.1clang-14: crash during compilation of darktable 3.8.1

comment:11 Changed 2 years ago by mascguy (Christopher Nielsen)

Description: modified (diff)
Summary: clang-14: crash during compilation of darktable 3.8.1clang-14: crashes occurring for multiple ports
Version: 2.7.2

comment:12 Changed 2 years ago by mascguy (Christopher Nielsen)

Port: libopenraw added; darktable-devel removed

Changed 2 years ago by mascguy (Christopher Nielsen)

comment:13 Changed 2 years ago by mascguy (Christopher Nielsen)

Description: modified (diff)

comment:14 Changed 2 years ago by Christopher Nielsen <mascguy@…>

In d8b7dea4e5ba84f2c2d941cecaa17b22bdb45519/macports-ports (master):

libopenraw: blacklist macports clang-14 for now, due to link crash
See: #65246

comment:15 Changed 2 years ago by mascguy (Christopher Nielsen)

Description: modified (diff)

comment:16 Changed 2 years ago by cjones051073 (Chris Jones)

LLVM 14 is proving to be a bit of a pain release. It has seen way more updates than LLVM devs normally put out, its up to 14.0.6, and if you read the release notes you can see the problems.

I think it might make sense to remove clang-14 as the default fallback, and go back to using clang-13 which seems more stable...

Last edited 2 years ago by cjones051073 (Chris Jones) (previous) (diff)

Changed 2 years ago by mascguy (Christopher Nielsen)

comment:18 in reply to:  16 Changed 2 years ago by mascguy (Christopher Nielsen)

Replying to cjones051073:

LLVM 14 is proving to be a bit of a pain release. It has seen way more updates than LLVM devs normally put out, its up to 14.0.6, and if you read the release notes you can see the problems.

I think it might make sense to remove clang-14 as the default fallback, and go back to using clang-13 which seems more stable...

Sounds good, though it looks like clang-13 may be missing one or more patches too. See attachment libopenraw-build-10.9-clang-13-crash-link.log​.

comment:19 Changed 2 years ago by cjones051073 (Chris Jones)

Can we have a different ticket for that, otherwise its gets messy...

comment:20 Changed 2 years ago by cjones051073 (Chris Jones)

b.t.w. its not obvious the issue is clang-13 there. The seg. faults there appear to come from rust, which internally builds against its own LLVM version...

comment:21 in reply to:  19 Changed 2 years ago by mascguy (Christopher Nielsen)

Replying to cjones051073:

Can we have a different ticket for that, otherwise its gets messy...

See issue:65434

comment:22 in reply to:  20 Changed 2 years ago by mascguy (Christopher Nielsen)

Replying to cjones051073:

b.t.w. its not obvious the issue is clang-13 there. The seg. faults there appear to come from rust, which internally builds against its own LLVM version...

Ah, the plot thickens! Suggestions as to how we proceed?

comment:23 Changed 2 years ago by cjones051073 (Chris Jones)

debugging rust internals is a rabbit hole I am not going down... I suggest reporting to upstream devs for the port in question and see what they say...

comment:24 Changed 2 years ago by catap (Kirill A. Korinsky)

Just for keep things in one place. Building of llvm-devel on 10.6 by clang-13 and clang-14 fails with the same reason like:

Assertion failed: (name != NULL), function Fixup, file src/ld/ld.hpp, line 394.
0  0x100001fa0  __assert_rtn + 79
1  0x10007564e  ld::Fixup::Fixup(unsigned int, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 70
2  0x10009015c  mach_o::relocatable::Parser<x86_64>::FixupInAtom::FixupInAtom(mach_o::relocatable::Parser<x86_64>::SourceLocation const&, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 30
3  0x10009baff  mach_o::relocatable::Parser<x86_64>::addFixup(mach_o::relocatable::Parser<x86_64>::SourceLocation const&, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 31
4  0x10008882d  mach_o::relocatable::Section<x86_64>::addRelocFixup(mach_o::relocatable::Parser<x86_64>&, macho_relocation_info<Pointer64<LittleEndian> > const*) + 1977
5  0x1000a4b2d  mach_o::relocatable::Section<x86_64>::makeFixups(mach_o::relocatable::Parser<x86_64>&, mach_o::relocatable::Parser<x86_64>::CFI_CU_InfoArrays const&) + 75
6  0x1000b8a0f  mach_o::relocatable::Parser<x86_64>::parse(mach_o::relocatable::ParserOptions const&) + 1251
7  0x1000b8e5f  mach_o::relocatable::Parser<x86_64>::parse(unsigned char const*, unsigned long long, char const*, long, unsigned int, mach_o::relocatable::ParserOptions const&) + 66
8  0x100089643  mach_o::relocatable::parse(unsigned char const*, unsigned long long, char const*, long, unsigned int, mach_o::relocatable::ParserOptions const&) + 319
9  0x10006ff07  archive::File<x86_64>::makeObjectFileForMember(archive::File<x86_64>::Entry const*) const + 407
10  0x1000709de  archive::File<x86_64>::justInTimeforEachAtom(char const*, ld::File::AtomHandler&) const + 86
11  0x10000de13  ld::tool::InputFiles::searchLibraries(char const*, bool, bool, bool, ld::File::AtomHandler&) const + 251
12  0x100064785  ld::tool::Resolver::resolveUndefines() + 171
13  0x100066579  ld::tool::Resolver::resolve() + 113
14  0x10000b896  main + 220
15  0x1000039b4  start + 52

comment:25 Changed 2 years ago by cooljeanius (Eric Gallager)

Cc: cooljeanius added

comment:26 in reply to:  24 Changed 2 years ago by mascguy (Christopher Nielsen)

Replying to catap:

Just for keep things in one place. Building of llvm-devel on 10.6 by clang-13 and clang-14 fails with the same reason

So the $100,000 question I'd like to throw out to the group, is this: Why are we introducing another upstream LLVM release - 15 - when LLVM 13 and 14 aren't rock-solid?

That makes zero sense to me...

comment:27 Changed 2 years ago by cjones051073 (Chris Jones)

Who knows, Maybe its better...

Just introducing the new port does not mean it will be used anywhere yet. It will only be used as a fallback once it is added to the _resources, and this hasn't happened yet.

comment:28 Changed 2 years ago by kencu (Ken)

The above linker issues might also be related to the absolutely ancient linker being used. Certainly upstream never tests something this old.

I use ld64-450 on 10.6.8, and I've never seen errors like that with clang-13 or clang-14, but it could just be coincidence.

Perhaps the next line of effort might most effectively be to get that bootstrapping sorted out to have all the Intel systems 10.6+ use at least ld64-450 as a linker (or newer if possible).

comment:29 Changed 2 years ago by kencu (Ken)

Actually - no. clang-14 errors out building darktable on the most up-to-date Monterey, with:

% ld -v
@(#)PROGRAM:ld  PROJECT:ld64-819.6
BUILD 14:58:37 Aug  5 2022
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 14.0.0, (clang-1400.0.29.102) (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 14.0.0 (tapi-1400.0.11)

comment:30 Changed 19 months ago by mascguy (Christopher Nielsen)

Just witnessed another crash on our buildbots, using clang-15, for port coreutils:

/opt/local/bin/clang-mp-15   -pipe -Os -arch x86_64  -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 -o src/df src/df.o src/find-mount-point.o src/libver.a lib/libcoreutils.a -lintl -Wl,-framework -Wl,CoreFoundation  lib/libcoreutils.a

0  0x100001fa0  __assert_rtn + 79
1  0x10007564e  ld::Fixup::Fixup(unsigned int, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 70
2  0x10009015c  mach_o::relocatable::Parser<x86_64>::FixupInAtom::FixupInAtom(mach_o::relocatable::Parser<x86_64>::SourceLocation const&, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 30
3  0x10009baff  mach_o::relocatable::Parser<x86_64>::addFixup(mach_o::relocatable::Parser<x86_64>::SourceLocation const&, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 31
4  0x10008882d  mach_o::relocatable::Section<x86_64>::addRelocFixup(mach_o::relocatable::Parser<x86_64>&, macho_relocation_info<Pointer64<LittleEndian> > const*) + 1977
5  0x1000a4b2d  mach_o::relocatable::Section<x86_64>::makeFixups(mach_o::relocatable::Parser<x86_64>&, mach_o::relocatable::Parser<x86_64>::CFI_CU_InfoArrays const&) + 75
6  0x1000b8a0f  mach_o::relocatable::Parser<x86_64>::parse(mach_o::relocatable::ParserOptions const&) + 1251
7  0x1000b8e5f  mach_o::relocatable::Parser<x86_64>::parse(unsigned char const*, unsigned long long, char const*, long, unsigned int, mach_o::relocatable::ParserOptions const&) + 66
8  0x100089643  mach_o::relocatable::parse(unsigned char const*, unsigned long long, char const*, long, unsigned int, mach_o::relocatable::ParserOptions const&) + 319
9  0x10006ff07  archive::File<x86_64>::makeObjectFileForMember(archive::File<x86_64>::Entry const*) const + 407
10  0x1000709de  archive::File<x86_64>::justInTimeforEachAtom(char const*, ld::File::AtomHandler&) const + 86
11  0x10000de13  ld::tool::InputFiles::searchLibraries(char const*, bool, bool, bool, ld::File::AtomHandler&) const + 251
12  0x100064785  ld::tool::Resolver::resolveUndefines() + 171
13  0x100066579  ld::tool::Resolver::resolve() + 113
14  0x10000b896  main + 220
make[2]: *** [src/df] Error 1

Logfile attached; filename: coreutils-clang-15-crash-10.6_x64.log.gz

Changed 19 months ago by mascguy (Christopher Nielsen)

comment:31 in reply to:  30 Changed 19 months ago by mascguy (Christopher Nielsen)

Replying to mascguy:

Just witnessed another crash on our buildbots, using clang-15, for port coreutils

Logfile attached; filename: coreutils-clang-15-crash-10.6_x64.log.gz

Queued another build attempt for 10.6_x64, hopefully it will succeed this time. Stay Tuned...

comment:32 Changed 19 months ago by kencu (Ken)

Oh, interesting. I suppose it could be that we'll have to force 10.6.8 to always use a newer linker when it's using a new clang.

Well, that could prove awkward, but needs to happen anyway. That is what clang11-bootstrap was supposed to allow.

comment:33 Changed 19 months ago by kencu (Ken)

I had to revert the new defaulting to newer clangs. I believe the issue is related to the old ld64-127 that is the buildbot default for SL.

We have been meaning to sort out how to default the linker on SL to something newer; I guess it's time we did.

Chris once-upon-a-time suggested the ${prefix}/bin/ld could be a shell script that looks for the newest to oldest ld64 versions we offer. That would work, but is a bit non-deterministic as you never really nail which linker you're getting ahead of time... but that could be worked around to a degree.

Alternatively we could sort out how to make clang-11-bootstrap build the newest ld64. The issue there is we need an LLVM for ld64 to link against, and libtapi too, so there are several parts to be sorted out.

SIgh.

Note: See TracTickets for help on using tickets.