clang-devel @devel-r365158_1+analyzer+assertions+libstdcxx re-build failure

[cave-canem]

sudo port -v upgrade -n --force clang-devel

Error: Failed to build clang-devel: command execution failed

= All logs in the attached crash.zip file. =

[cave-canem]

Four files in this zip archive.

[mf2k (Frank Schima)]

The Milestone field is for use by Macports team members only, please do not set it.

[cave-canem]

For Frank Schima: thanks for editing (newcomers are not aware of the subtleties).

[kencu (Ken)]

Can I ask what your toolchain details are? The results of this information should be sufficient:

port -v installed | grep llvm
port -v installed | grep clang
port select --list clang
port select --list llvm
port -v installed | grep ld64
port -v installed cctools
which ld
ld -v

[cave-canem]

Thanks for your reply.

I forgot to write that I tried re-build clang-devel using clang-devel:

sudo port -v upgrade -n --force clang-devel

And these are the answers to your questions:

port -v installed | grep llvm
  cctools @927.0.2_2+llvmdev (active) platform='darwin 13' archs='x86_64' date='2020-03-18T11:49:36+0300'
  ld64-274 @274.2_0+llvm37 (active) platform='darwin 13' archs='x86_64' date='2020-02-23T18:46:48+0300'
  llvm-3.7 @3.7.1_4 (active) platform='darwin 13' archs='x86_64' date='2020-02-23T18:43:21+0300'
  llvm-5.0 @5.0.2_1 (active) platform='darwin 13' archs='x86_64' date='2020-02-25T17:15:15+0300'
  llvm-9.0 @9.0.1_0 (active) platform='darwin 13' archs='x86_64' date='2020-04-02T00:32:03+0300'
  llvm-devel @devel-r365158_1+assertions (active) platform='darwin 13' archs='x86_64' date='2020-03-31T02:02:15+0300'
  llvm_select @2_0 (active) platform='darwin 13' archs='noarch' date='2020-02-23T18:04:22+0300'

port -v installed | grep clang
  clang-3.7 @3.7.1_6+analyzer (active) platform='darwin 13' archs='x86_64' date='2020-02-23T20:36:58+0300'
  clang-5.0 @5.0.2_4+analyzer+defaultlibcxx+libstdcxx (active) platform='darwin 13' archs='x86_64' date='2020-02-25T19:45:21+0300'
  clang-9.0 @9.0.1_0+analyzer+libstdcxx (active) platform='darwin 13' archs='x86_64' date='2020-04-02T05:41:26+0300'
  clang-devel @devel-r365158_1+analyzer+assertions+libstdcxx (active) platform='darwin 13' archs='x86_64' date='2020-02-25T11:54:10+0300'
  clang_select @2.2_0 (active) platform='darwin 13' archs='noarch' date='2020-02-23T18:46:04+0300'

(llvm-9.0 and clang-9.0 are installed after trying to build clang-devel, see the dates).

port select --list clang
Available versions for clang:
	mp-clang-3.7
	mp-clang-5.0
	mp-clang-9.0 (active)
	mp-clang-devel
	none

port select --list llvm
Available versions for llvm:
	mp-llvm-3.7
	mp-llvm-5.0
	mp-llvm-9.0 (active)
	mp-llvm-devel
	none

port -v installed | grep ld64
  ld64 @3_2+ld64_274 (active) platform='darwin 13' archs='x86_64' date='2020-02-23T18:46:49+0300'
  ld64-274 @274.2_0+llvm37 (active) platform='darwin 13' archs='x86_64' date='2020-02-23T18:46:48+0300'

port -v installed cctools
The following ports are currently installed:
  cctools @927.0.2_2+llvmdev (active) platform='darwin 13' archs='x86_64' date='2020-03-18T11:49:36+0300'

which -a ld
/opt/MacPorts/bin/ld
/usr/bin/ld

ld -v
@(#)PROGRAM:ld  PROJECT:ld64-274.2
configured to support archs: i386 x86_64 x86_64h armv6 armv7 armv7s armv7m armv7k arm64 (tvOS)
LTO support using: LLVM version 3.7.1

/usr/bin/ld -v
@(#)PROGRAM:ld  PROJECT:ld64-241.9
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7m armv7em
LTO support using: LLVM version 3.5svn

[cave-canem]

Hi Ken!

I do not know which of the three factors affected, but after installing the port z3 @ 4.8.7, upgrading to cmake @ 3.17.0_1 and using clang-9.0 @ 9.0.1 to compile clang-devel, the clang-devel @devel-r365158_1 build was successful.

port -v installed | egrep "(z3|cmake|clang)"
  clang-3.7 @3.7.1_6+analyzer (active) platform='darwin 13' archs='x86_64' date='2020-02-23T20:36:58+0300'
  clang-5.0 @5.0.2_4+analyzer+defaultlibcxx+libstdcxx (active) platform='darwin 13' archs='x86_64' date='2020-02-25T19:45:21+0300'
  clang-9.0 @9.0.1_0+analyzer+libstdcxx (active) platform='darwin 13' archs='x86_64' date='2020-04-02T05:41:26+0300'
  clang-devel @devel-r365158_1+analyzer+assertions+libstdcxx (active) platform='darwin 13' archs='x86_64' date='2020-04-03T04:02:37+0300'
  clang_select @2.2_0 (active) platform='darwin 13' archs='noarch' date='2020-02-23T18:46:04+0300'
  cmake @3.17.0_0 platform='darwin 13' archs='x86_64' date='2020-03-25T04:41:17+0300'
  cmake @3.17.0_1 (active) platform='darwin 13' archs='x86_64' date='2020-04-02T23:18:11+0300'
  z3 @4.8.7_0+gmp+lto+threads (active) platform='darwin 13' archs='x86_64' date='2020-04-02T07:32:13+0300'

Unfortunately, I forgot to use the -k option when building the port, so the log is lost.

[kencu (Ken)]

I might guess that using clang-9.0 to build it was the key factor, but can't be sure. Building something with a very new version of clang generating the bitcode, but then trying to LTO it with something as old as llvm-3.7, can't be too healthy either.

Glad to see you have it done. I'm going to close this as a 'works for me' as I don't think there is anything here that we can fix. Sooner or later clang-devel will get updated, depending on when the module issue with ncurses gets fixed and what kind of fix for that issue is brought forward (ie how resilient it is).

[cave-canem]

Sorry Ken, I don’t know what the abbreviation "LTO" means, , but the last (unsuccessful) time I re-builting clang-devel using clang-devel.

[kencu (Ken)]

"Link Time Optimization". Basically clang compiles all the source code into LLVM bitcode instead of into actual machine language object files. Then, at the end, in the link stage, the linker (ld64) will then pass all that bitcode from all the different objects and libraries through optimization passes in llvm, and then finally the optimized llvm bitcode final product with be sent to the assembly writer modules to be finaly made into a real executable file.

But when clang-9.0 takes a bunch of source code and compiles it into llvm bitcode, then it gets sent to an older ld64 such as you are using that is linked against an even older llvm (3.7) to do the final optimization stages -- well frankly I'm surprised it actually ever works.

I usually upgrade my ld64 to the latest possible version (ld64-latest runs on 10.6.8 and up) and I build it against the latest llvm that we have available (currently llvm-9.0) and I also use cctools that is likewise linked against the latest llvm we have, all with the idea that it will minimize this sort of thing.

Probably we should all do that -- the +llvm37 variants of things should all be "+bootstrap" variants that have to be upgraded before actually being used.

[cave-canem]

Ken, thanks for the answer.

I tried to use ld64-latest a couple of years ago, but that didn’t work for me. Now everything is working successfully (maybe I got smarter :))?

One more question, if you allow me: how can I change the clang optimization flag to compile a port?

I am very grateful for your help,

Kind regards, cave-canem.

[kencu (Ken)]

Glad it works. MacPorts is very powerful, and helps these older systems a lot.

To change the optflags, you need to edit the port's Portfile, like this:

bbedit `port file MYPORT`

if there are already optflags written into the portfile, you can change them.

configure.optflags -03   
#or whatever

otherwise add the line yourself.

It doesn't always work, depending on the build system, but it will work with a large majority of ports.

To see what ports do when that doesn't work, go to the top of the port tree, for example like this:

cd `port dir llvm-3.7`/../../

and take a look around with your searcher of choice. I like the_silver_searcher so I would do

ag configure.optflags .

and you can see what others have done in various ports to monkey with optflags.

[cave-canem]

Thanks Ken!

P.S.

"ag" seems like a good tool, did not know about it before.

P.P.S.

I forgot to apologize for bad English — my native language is Russian. = Спасибо, Кен! =