Opened 2 years ago

Last modified 2 years ago

#65236 new defect

OpenBLAS @0.3.20_0+gcc11+lapack+native build failure on intel mac 11.6.5: gcc is putting out x86-pad-for-align=false but assembler is not accepting it

Reported by: dsavransky (Dmitry Savransky) Owned by:
Priority: Normal Milestone:
Component: ports Version: 2.7.2
Keywords: Cc: NicosPavlov, michaelld (Michael Dickens), cooljeanius (Eric Gallager), cjones051073 (Chris Jones)
Port: OpenBLAS, gcc11

Description

Here's a 'fun' one: went to upgrade openblas today and got a build failure (log attached). Basic error appears to be bad llvm argument ('-x86-pad-for-align=false') but I can't find much info on that anywhere. Most likely some llvm/clang clash. However, if I do: sudo port upgrade --enforce-variants openblas +lapack +native +gcc10 -gcc11 it builds just fine.

I'm on macos 11.6.5, with xcode 13.2.1 (and 13.2 command line tools installed). I previously had OpenBLAS @0.3.19_0+gcc11+lapack+native installed with no apparent issues.

Thanks.

Attachments (2)

main.log (4.7 MB) - added by dsavransky (Dmitry Savransky) 2 years ago.
OpenBLAS @0.3.20_0+gcc11+lapack+native failing build log
main.2.log (4.7 MB) - added by dsavransky (Dmitry Savransky) 2 years ago.
OpenBLAS @0.3.20_1+gcc12+lapack+native failing build log

Change History (32)

Changed 2 years ago by dsavransky (Dmitry Savransky)

Attachment: main.log added

OpenBLAS @0.3.20_0+gcc11+lapack+native failing build log

comment:1 Changed 2 years ago by cooljeanius (Eric Gallager)

Cc: cooljeanius added

comment:2 Changed 2 years ago by cooljeanius (Eric Gallager)

This seems to be blocking a lot of my upgrades; I didn't realize so many ports depended on OpenBLAS...

comment:3 in reply to:  2 ; Changed 2 years ago by dsavransky (Dmitry Savransky)

Replying to cooljeanius:

This seems to be blocking a lot of my upgrades; I didn't realize so many ports depended on OpenBLAS...

Have you tried building with gcc10 instead of 11? That previously worked and allowed me to update everything else.

comment:4 Changed 2 years ago by kencu (Ken)

that argument x86-pad-for-align=false was added to gcc to get around a bug:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100340

but here it appears that the Xcode clang assembler on the failing system doesn't know what to do with it, and errors...

This is a strange turn of events; the option is meant to be added only when needed I believe, so exactly why it's now unaccepted is a bit confusing. There are a lot of variables at play here now.

comment:5 Changed 2 years ago by kencu (Ken)

Cc: cjones051073 added
Port: gcc11 added
Summary: OpenBLAS @0.3.20_0+gcc11+lapack+native build failure on intel mac 11.6.5OpenBLAS @0.3.20_0+gcc11+lapack+native build failure on intel mac 11.6.5: gcc is putting out x86-pad-for-align=false but assembler is not accepting it

comment:6 Changed 2 years ago by kencu (Ken)

probably more gccs involved than just gcc11 I assume, but gcc11 is what we know (knew) then

comment:7 Changed 2 years ago by cjones051073 (Chris Jones)

For those blocked by this, you can work around the issues by removing the native variant which is forcing you to build from source.

Also note the default gcc used now is gcc12, so if you move to using that as well you should just oick up the binary tarball, rather than building from source yourself.

comment:8 in reply to:  3 ; Changed 2 years ago by cooljeanius (Eric Gallager)

Replying to dsavransky:

Replying to cooljeanius:

This seems to be blocking a lot of my upgrades; I didn't realize so many ports depended on OpenBLAS...

Have you tried building with gcc10 instead of 11? That previously worked and allowed me to update everything else.

Oh yeah I was actually already trying with gcc10 in the first place; I didn't notice that this bug was originally reported against gcc11... anyways, using the gcc12 variant lets me get a binary from the server; let me try actually building with it next...

comment:9 Changed 2 years ago by cjones051073 (Chris Jones)

i cannot reproduce. OpenBLAS +native built fine for me on Intel macOS12.

comment:10 in reply to:  8 Changed 2 years ago by cooljeanius (Eric Gallager)

Replying to cooljeanius:

Replying to dsavransky:

Replying to cooljeanius:

This seems to be blocking a lot of my upgrades; I didn't realize so many ports depended on OpenBLAS...

Have you tried building with gcc10 instead of 11? That previously worked and allowed me to update everything else.

Oh yeah I was actually already trying with gcc10 in the first place; I didn't notice that this bug was originally reported against gcc11... anyways, using the gcc12 variant lets me get a binary from the server; let me try actually building with it next...

OK so building with +gcc12+lapack+native fails too; I dunno how the buildbot managed it...

comment:11 Changed 2 years ago by cjones051073 (Chris Jones)

Buildbot does not use native variant, as its not enabled by default.

However, I used the variant and was also unable to reproduce.

comment:12 Changed 2 years ago by cjones051073 (Chris Jones)

Please ensure your ports are up to date before building, just to be sure..

sudo port -d sync
sudo port upgrade outdated

Then try again.

Changed 2 years ago by dsavransky (Dmitry Savransky)

Attachment: main.2.log added

OpenBLAS @0.3.20_1+gcc12+lapack+native failing build log

comment:13 Changed 2 years ago by dsavransky (Dmitry Savransky)

I just did a full selfupdate and then attempted to build with gcc12. Same exact error as before. I think this is very much a macOS 11.6/Xcode 12.4 -specific issue. new log attached.

comment:14 Changed 2 years ago by cjones051073 (Chris Jones)

self update does not necessarily update outdated ports. Can you please confirm running the commands i sent above does nothing ?

comment:15 Changed 2 years ago by cjones051073 (Chris Jones)

Buildbot macOS 11 build used Xcode 13

https://build.macports.org/builders/ports-11_x86_64-builder/builds/79119/steps/install-port/logs/stdio

Could you try updating your Xcode and try again ?

comment:16 Changed 2 years ago by dsavransky (Dmitry Savransky)

sorry, typo on the xcode version. I have Xcode 13.2.1 on that system (as per my orig bug report). port -d sync reports:

Total number of ports parsed: 0 Ports successfully parsed: 0 Ports failed: 0 Up-to-date ports skipped: 29607

port upgrade outdated fails on OpenBLAS build, as before.

comment:17 Changed 2 years ago by cjones051073 (Chris Jones)

Please attempt a build from source without the native variant, using only the defaults

sudo port uninstall OpenBLAS
sudo port -s install OpenBLAS

comment:18 Changed 2 years ago by dsavransky (Dmitry Savransky)

Same error building without native.

comment:19 Changed 2 years ago by cjones051073 (Chris Jones)

Then I am afraid you are going to have to play spot-the-difference with the buildbot log I posted above, as that uses the same variants and Xcode 13, so on the face of it is the same and worked just fine. Its looking like something specific to your system i am afraid….

Last edited 2 years ago by cjones051073 (Chris Jones) (previous) (diff)

comment:20 Changed 2 years ago by cjones051073 (Chris Jones)

… or just use the binary install and forget about the problem..

comment:21 Changed 2 years ago by kencu (Ken)

Eric, exactly what cctools do you have installed, and do you have any macports-clang-* compilers installed?

I have a suspicion different assemblers may be at work here...

comment:22 Changed 2 years ago by iains

as noted in the GCC PR, I cannot reproduce the failure - the option seems to be accepted by clang -cc1as for all the versions I've tried from CLT 12.x onwards (on darwin20 and 21). I'd go with @kencu's suspicion about a bogus assembler in the mix.

comment:23 Changed 2 years ago by cjones051073 (Chris Jones)

ken’s suggestion of the cctools version you are using is a good shout. Fwiw I have the version using the xcode variant installed , which basically just installs wrapper scripts around whatever xcrun provides, and is the default on the new OS versions.

cctools @949.0.1_2+xcode (active)

Oberon ~ > cat /opt/local/bin/as
#!/bin/bash
if [[ -x /usr/bin/xcrun ]] ; then
  exec /usr/bin/xcrun as "${@}"
elif [[ -x /usr/bin/as ]] ; then
  exec /usr/bin/as "${@}"
else
  exec as "${@}"
fi
Oberon ~ > 
Last edited 2 years ago by cjones051073 (Chris Jones) (previous) (diff)

comment:24 Changed 2 years ago by dsavransky (Dmitry Savransky)

oh wow. I feel pretty silly now. I had an active clang-11 (not quite sure why that was getting used in this build). I got rid of that (and everything other than clang-14) and am now able to build OpenBLAS from source, with and without -native. Thank you all for your help and patience.

comment:25 Changed 2 years ago by cjones051073 (Chris Jones)

Just having macports clang 11 installed would not on its own affect the build, there must be some-other piece to the puzzle.

To repeat, please confirm what cctools version you have installed ?

Had you also by chance port selected a specific version clang to be the default ?

comment:26 Changed 2 years ago by dsavransky (Dmitry Savransky)

you're right. my active cctools was @949.0.1_2+llvm10. i had not selected a specific clang.

comment:27 Changed 2 years ago by kencu (Ken)

Aha -- any non-Xcode variant of cctools will spin off assembly to a relatively new macports-clang-N if it is found. So that is likely to be part of it.

If the build failed when clang-11 was installed but succeeded when it was deactivated, then possibly clang-11 doesn't accept that argument.

But clang-14 (if installed, and it appears it was) should have been selected *first* anyway, unless the order these clangs are chosen is wrong (backwards).

comment:28 Changed 2 years ago by kencu (Ken)

lots of moving parts, indeed.

comment:29 in reply to:  21 Changed 2 years ago by cooljeanius (Eric Gallager)

Replying to kencu:

Eric, exactly what cctools do you have installed, and do you have any macports-clang-* compilers installed?

I have a suspicion different assemblers may be at work here...

$ port installed cctools
The following ports are currently installed:
  cctools @949.0.1_2+llvm10 (active)
$ port installed clang*
The following ports are currently installed:
  clang-9.0 @9.0.1_6+analyzer+emulated_tls+libstdcxx (active)
  clang-10 @10.0.1_7+analyzer+emulated_tls+libstdcxx (active)
  clang-11 @11.1.0_6+analyzer+debug+emulated_tls+libstdcxx (active)
  clang-12 @12.0.1_0+analyzer+debug+libstdcxx+tests (active)
  clang-13 @13.0.1_2+analyzer+libstdcxx (active)
  clang-14 @14.0.6_0+analyzer+debug+libstdcxx+tests (active)
  clang_select @2.2_0 (active)
$ port select clang
Available versions for clang:
	mp-clang-10
	mp-clang-11
	mp-clang-12
	mp-clang-13
	mp-clang-14
	mp-clang-9.0 (active)
	none

comment:30 Changed 2 years ago by kencu (Ken)

So Eric I'm not 100% sure what OS version you have going there, but anything fairly recent should be running cctools +xcode to pick up the tools (including the assembler) that Xcode supplies. (Also the ld64 xcode variant, for the same reason). Here is what I have, on Monterey:

% port -v installed | grep cctools
  cctools @949.0.1_2+xcode (active) requested_variants='' platform='darwin 21' archs='noarch' date='2022-07-28T06:58:15-0700'

% port -v installed | grep ld64
  ld64 @3_4+ld64_xcode (active) requested_variants='' platform='darwin 21' archs='x86_64' date='2022-07-28T06:58:16-0700'
  ld64-xcode @2_4 (active) requested_variants='' platform='darwin 21' archs='x86_64' date='2022-07-28T06:58:16-0700'

With what you have installed there, your cctools "as" will look at the macports-clangs that you have installed, and choose one of them to be used as the assembler instead.

Now that *should* be the newest one (clang-14) which should accept that assembler command that is failing -- but because you have "selected" clang-9.0 as your active clang, maybe that one is somehow being chosen above clang-14. Or maybe clang-14 doesn't like that directive, I would have to check to see exactly what likes what when and where.

for sure, clang-9.0 is likely to be too old to understand the x86-pad-for-align=false directive, I would think.

Note: See TracTickets for help on using tickets.