Opened 9 years ago

Closed 9 years ago

#48507 closed defect (fixed)

julia @0.3.9 +gcc48: destroot fails building OpenBLAS with native clang

Reported by: anowacki (Andy Nowacki) Owned by: seanfarley (Sean Farley)
Priority: Normal Milestone:
Component: ports Version: 2.3.99
Keywords: Cc: petrrr, NicosPavlov
Port: julia

Description

When trying to upgrade julia from 0.3.8 to 0.3.9, or install julia @0.3.9 +gcc48, destroot fails when trying to build OpenBLAS (see following from main.log):

[Lots of other undeclared BLAS identifiers in getarch_2nd.c]
[...]
:info:destroot getarch_2nd.c:70:50: error: use of undeclared identifier 'ZGEMM_DEFAULT_Q'
:info:destroot     printf("#define ZLOCAL_BUFFER_SIZE\t%ld\n", (ZGEMM_DEFAULT_Q * ZGEMM_DEFAULT_UNROLL_N * 2 * 2 *  sizeof(double)));
:info:destroot                                                  ^
:info:destroot fatal error: too many errors emitted, stopping now [-ferror-limit=]
:info:destroot 20 errors generated.
:info:destroot make[4]: *** [getarch_2nd] Error 1
:info:destroot make[4]: Entering directory `/opt/local/var/macports/build/_Users_user_macports_trunk_dports_lang_julia/julia/work/julia/deps/openblas-v0.2.13'
:info:destroot Makefile:131: *** OpenBLAS: Detecting CPU failed. Please set TARGET explicitly, e.g. make TARGET=your_cpu_target. Please read README for the detail..  Stop.
:info:destroot make[4]: Leaving directory `/opt/local/var/macports/build/_Users_user_macports_trunk_dports_lang_julia/julia/work/julia/deps/openblas-v0.2.13'
:info:destroot *** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0 if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. ***
:info:destroot make[3]: *** [openblas-v0.2.13/libopenblas.dylib] Error 1
:info:destroot make[3]: Leaving directory `/opt/local/var/macports/build/_Users_user_macports_trunk_dports_lang_julia/julia/work/julia/deps'
:info:destroot make[2]: *** [julia-release] Error 2
:info:destroot make[2]: Leaving directory `/opt/local/var/macports/build/_Users_user_macports_trunk_dports_lang_julia/julia/work/julia'
:info:destroot make[1]: *** [release] Error 2
:info:destroot make[1]: Leaving directory `/opt/local/var/macports/build/_Users_user_macports_trunk_dports_lang_julia/julia/work/julia'
:info:destroot make: *** [install] Error 2
:info:destroot make: Leaving directory `/opt/local/var/macports/build/_Users_user_macports_trunk_dports_lang_julia/julia/work/julia'
:info:destroot Command failed:  cd "/opt/local/var/macports/build/_Users_user_macports_trunk_dports_lang_julia/julia/work/julia" && /usr/bin/make -w install CC=/usr/bin/clang CXX=/usr/bin/clang++ FC=/opt/local/bin/gfortran-mp-4.8 USE_SYSTEM_LLVM=1 LLVM_CONFIG=llvm-config-mp-3.5 USE_SYSTEM_LIBUNWIND=1 USE_SYSTEM_LIBM=1 USE_SYSTEM_GMP=1 USE_SYSTEM_MPFR=1 USE_SYSTEM_ZLIB=1 USE_SYSTEM_READLINE=1 USE_SYSTEM_PCRE=1 USE_SYSTEM_FFTW=1 USE_SYSTEM_ARPACK=1 USE_SYSTEM_SUITESPARSE=1 DESTDIR=/opt/local/var/macports/build/_Users_user_macports_trunk_dports_lang_julia/julia/work/destroot 

This occurs whether upgrading the existing port, or cleaning then installing julia +gcc48. I haven't tried installing gcc49 and using the julia +gcc49 (default) variant on this machine, but another machine (2013 Mac Pro) with gcc49 upgrades without issue, so I suspect this may be a gcc version problem.

I am on MacPorts trunk r139164, OS X 10.10.4.

Attachments (1)

main.log (79.9 KB) - added by anowacki (Andy Nowacki) 9 years ago.

Download all attachments as: .zip

Change History (19)

Changed 9 years ago by anowacki (Andy Nowacki)

Attachment: main.log added

comment:1 Changed 9 years ago by mf2k (Frank Schima)

Cc: sean@… removed
Owner: changed from macports-tickets@… to sean@…

comment:2 Changed 9 years ago by anowacki (Andy Nowacki)

I note ticket #48381 has the same issue but for the OpenBLAS port. Apologies for not spotting it earlier.

[Previous edit had incorrect ticket in mind; post was: Apologies if I'm wrong, but might this possibly have the same cause as previous problems with the OpenBLAS port (#46684)?]

Last edited 9 years ago by anowacki (Andy Nowacki) (previous) (diff)

comment:3 Changed 9 years ago by anowacki (Andy Nowacki)

The issue is not with the compiler version, but with detecting the processor type in the OpenBLAS build; this explains the difference between the Mac Pro and MacBook Pro (because the latter has a Broadwell CPU). See: https://github.com/xianyi/OpenBLAS/issues/529

This has been fixed upstream in OpenBLAS, but has not been merged into master or a release yet (see #48381). The linked GitHub page contains suggestions for workarounds in the meantime.

comment:4 Changed 9 years ago by petrrr

Resolution: duplicate
Status: newclosed

I am closing this as duplicate, to limit discussion on this at #48381.

comment:5 Changed 9 years ago by petrrr

Cc: petr@… added

Cc Me!

comment:6 Changed 9 years ago by petrrr

Resolution: duplicate
Status: closedreopened

Sorry for have misinterpreted the situation. This is above OpenBLAS build within julia. Reopening!

comment:7 Changed 9 years ago by petrrr

Cc: nicos@… added

I am somewhat wondering if it is really necessary to build OpenBLAS as part of julia, or if ti might be possible to rely on the OpenBLAS{-devel} port?

I allow myself to CC the maintainer of OpenBLAS here.

comment:8 Changed 9 years ago by petrrr

Okay, the answer to my question above is in the Portfile, which a have looked at only now. I report it here for simplicity:

# julia can't use Apple's Accelerate framework so the choices are to build
# lapack (32-bit interface) or build OpenBLAS (64-bit interface).
# Alternatively, we could try to use MacPorts' own OpenBLAS port but that would
# need to be updated to build the 64-bit interface which is inocmpatible with
# the 32-bit interface. Since that could break other ports dependent on
# OpenBLAS, we'll just stick with having julia download and build its own
# internal OpenBLAS.

comment:9 in reply to:  8 Changed 9 years ago by seanfarley (Sean Farley)

Replying to petr@…:

Okay, the answer to my question above is in the Portfile, which a have looked at only now. I report it here for simplicity:

# julia can't use Apple's Accelerate framework so the choices are to build
# lapack (32-bit interface) or build OpenBLAS (64-bit interface).
# Alternatively, we could try to use MacPorts' own OpenBLAS port but that would
# need to be updated to build the 64-bit interface which is inocmpatible with
# the 32-bit interface. Since that could break other ports dependent on
# OpenBLAS, we'll just stick with having julia download and build its own
# internal OpenBLAS.

Yep, this is indeed the reason. One possible solution would be a blas/lapack port group that could unify all these different options. As for this ticket, though, I haven't had time to look at it.

comment:10 Changed 9 years ago by NicosPavlov

This is only partly related to the issue of the ticket, but I do not really understand the note above. To the best of my knowledge, the OpenBLAS port already builds 64 binaries on relevant architectures, and also supports universal builds.

comment:11 in reply to:  10 Changed 9 years ago by seanfarley (Sean Farley)

Replying to nicos@…:

This is only partly related to the issue of the ticket, but I do not really understand the note above. To the best of my knowledge, the OpenBLAS port already builds 64 binaries on relevant architectures, and also supports universal builds.

64-bit binaries != 64-bit API, namely we need to set INTERFACE64=1. That being said, we could try to reuse the MacPorts either by building both interfaces (and perhaps renaming the 64bit ones) or just always setting the interface and seeing what breaks (seems there is only one dependency on openblas ... but people could be using it outside of MacPorts).

comment:12 Changed 9 years ago by NicosPavlov

OK, thanks for the clarification, got it.
In case julia would depend on the existing port, the most ideal would seem to me to be a subport, which could separately build the 64-bit API to depend on it. I could make the changes if they are to be used. It could also be possible to use a binary dependency, so that both OpenBLAS or OpenBLAS-devel could satisfy it.

Most related to the ticket, a patch applied to the OpenBLAS port in #48381 enabled version 0.2.14 to build with newer CPUs, although it does not detect them yet. Furthermore, OpenBLAS-devel, from a recent commit of the repository, builds while handling Broadwell chips.

comment:13 Changed 9 years ago by seanfarley (Sean Farley)

I agree but have been too busy to really do that kind of work.

comment:14 Changed 9 years ago by seanfarley (Sean Farley)

So, is there any resolution on what to do here?

comment:15 Changed 9 years ago by NicosPavlov

As mentioned before, there is a patch which solved the issue for the OpenBLAS portfile, refer to r139234, r140759 and r140768.

Most related to the ticket, a patch applied to the OpenBLAS port in #48381 enabled version 0.2.14 to build with newer CPUs, although it does not detect them yet. Furthermore, OpenBLAS-devel, from a recent commit of the repository, builds while handling Broadwell chips.

Last edited 9 years ago by NicosPavlov (previous) (diff)

comment:16 Changed 9 years ago by seanfarley (Sean Farley)

Ok, I'll take a look at those patches.

comment:17 Changed 9 years ago by NicosPavlov

Just in case, OpenBLAS just released a version 0.2.15, in which the support for newer processors and clang is much better, and could possibly directly solve the issues described in this ticket.

comment:18 Changed 9 years ago by seanfarley (Sean Farley)

Resolution: fixed
Status: reopenedclosed

I think this is fixed with the release of julia 0.4.0. Please reopen with a new log if it's still not fixed.

Note: See TracTickets for help on using tickets.