qrupdate: +atlas variant fails to build: undefined symbols

[moe.nicholas@…]

Hello,

I am trying to install octave on my PowerBook G4. I have run

$ sudo port selfupdate

and

$ sudo port upgrade outdated

The last dependency to install, qrupdate, fails to build and gives the following output:

$ sudo port install octave
--->  Computing dependencies for octave
--->  Dependencies to be installed: qrupdate
--->  Fetching qrupdate
--->  Attempting to fetch qrupdate-1.1.1.tar.gz from http://superb-east.dl.sourceforge.net/qrupdate
--->  Attempting to fetch qrupdate-1.1.1.tar.gz from http://downloads.sourceforge.net/qrupdate
--->  Verifying checksum(s) for qrupdate
--->  Extracting qrupdate
--->  Applying patches to qrupdate
--->  Configuring qrupdate
--->  Building qrupdate
Error: Target org.macports.build returned: shell command failed
Log for qrupdate is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_qrupdate/main.log
Error: The following dependencies failed to build: qrupdate
Error: Status 1 encountered during processing.
To report a bug, see <http://guide.macports.org/#project.tickets>
$

I am attaching the log file cited above.

[jmroot (Joshua Root)]

The symbols that are undefined here are present in /opt/local/lib/liblapack.dylib on my system (10.6/x86_64).

[penyuan@…]

The same thing happens on my MacBook Pro running Mac OS X 10.6.4 when trying to install Octave. Is there a workaround?

[penyuan@…]

Cc Me!

[stefan@…]

Same on my iBook G4 with 10.5.8. liblapack.dylib is present but not recognized.

[stefan@…]

Cc Me!

[michaelld (Michael Dickens)]

On my 10.6.4 x86_64 box, the undefined symbols are actually in -lptf77blas (either .a or .dylib) -- which is interesting because that library isn't directly referenced during the link stage (nor is the linking told to ignore undefined symbols), but the linking still works. I'm attached a patch that might work; please try it out.

[michaelld (Michael Dickens)]

The Octave issue is on another ticket already. For -devel, that's #26640. IDK which one it is for octave alone, but I'm sure it's there.

[michaelld (Michael Dickens)]

@stefan: Can you attach a debug-log of your install, to see if it's the same issue as for this ticket?

[michaelld (Michael Dickens)]

"svn diff" of 'qrupdate' Portfile, to add in missing atlas symbols

[michaelld (Michael Dickens)]

Cc Me!

[stefan@…]

main.log file of my failed qrupdate build. iBook G4 OS X 10.5.8

[mfuhrer@…]

Ran into this link problem this evening on my Powerbook G4 with Mac OS X 10.5.8. Tried Michael's patch with no success, as I don't have -lptf77blas in /opt/local/lib. Then I found -lf77blas which does define the missing symbols. Modified Michael's patch and was able to link and install qrupdate successfully.

[mfuhrer@…]

Modified patch with -lf77blas

[mfuhrer@…]

Cc Me!

[AlonzoQuixote@…]

Cc Me!

[jsnodgra@…]

Cc Me!

[ejtttje@…]

I'm seeing this now following an update to atlas (#34295) which builds the blas libraries in question. I also had to add -latlas.

The -ptf77blas and -ltatlas give multi-threaded usage, whereas -lf77blas and -latlas give single-threaded libraries. I'm not sure which is more appropriate here.

I still wound up running into issues with __gfortran_transfer_character_write and __gfortran_transfer_integer_write being undefined, which is supposedly fixed by setting the linker/loader to use gfortran, but I couldn't figure that out. I wound up adding -undefined dynamic_lookup to get around this, which probably isn't the right solution.

[ejtttje@…]

Cc Me!

[Veence (Vincent)]

The dylibs automatically built by atlas 3.9.75 differ from those that were (manually) built by 3.9.47. They are named libtatlas.dylib and libsatlas.dylib. You must look after the undefined symbols using nm then maybe patch qrupdate accordingly.

[nabble@…]

I had a similar problem trying to install octave, link failing on qrupdate, but I had to tweak my Portfile differently to get it to build - attached as qrupdate_Portfile3.diff

[ryandesign (Ryan Carsten Schmidt)]

Vincent, is the patch attached by makuch.org correct—add "-latlas -lsatlas -ltatlas -lf77blas -lptf77blas"?

I've rearranged things a bit in my update to 1.1.2 so you may want to keep that in mind when preparing a fix for this issue as well.

[nabble@…]

I seriously doubt it's correct, but it worked for me :-) so I just thought I'd share it with you guys knowing you would fix it correctly.

[Veence (Vincent)]

-latlas -lf77blas -lptf77blas is redundant with -libsatlas and -libtatlas. The first three are static archives, and the last two dynamic libraries. Then you can choose between satlas or tatlas whether you need threaded Atlas or not. So, to wrap up, only one of libsatlas or libtatlas is needed.

[mauricio_ville@…]

Cc Me!

[MacPorts@…]

Cc Me!

[m1cha3ld0m1n0@…]

Cc Me!

[enderw88@…]

I was able to get qrupdate to build and install by

1. cd'ing into the working directory /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/math/qrupdate/work/qrupdate-1.1.1
2. executing sudo make solib.
3. executing sudo port install qrupdate.

[m1cha3ld0m1n0@…]

Using enderw88's steps, was able to build on 10.5.8. Thanks!

[ryandesign (Ryan Carsten Schmidt)]

enderw88, that's not a fix. When you do that, you're bypassing the PREFIX, BLAS and LAPACK arguments MacPorts would have set for you, so at that point you're not actually using atlas, plus the library will get the wrong install_name. So you may as well install qrupdate without atlas support at that point.

sudo port selfupdate
sudo port clean qrupdate
sudo port install qrupdate -atlas

[enderw88@…]

Replying to ryandesign@…:

enderw88, that's not a fix. When you do that, you're bypassing the PREFIX, BLAS and LAPACK arguments MacPorts would have set for you, so at that point you're not actually using atlas, plus the library will get the wrong install_name. So you may as well install qrupdate without atlas support at that point.

sudo port selfupdate
sudo port clean qrupdate
sudo port install qrupdate -atlas

Thanks, at this point I just really want to get Octave built. But I can see why this isn't a real solution.

[sewebster@…]

Cc Me!

[jerome.lelong@…]

Add blas dependency for +atlas variant

[jerome.lelong@…]

qrupdate does not build with variant +atlas because of missing symbols at link actually provided by Blas. It seems that the patch qrupdate_Portfile_4.diff solves the issue at least if atlas was installed with gcc45. If atlas was installed with variant +gcc46, then you should consider adding the corresponding variant to qrupdate to achieve the build.

[ben.pts@…]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #34654.

[weasel@…]

It appears a diff four fixes the problem I noted in #34654 with qrupdate, but forces atlas +gcc46 be installed, which oddly enough completes without gcc46 being installed. There is still no /opt/local/lib/libsatlas.dylib, maybe that's intentional. I haven't yet tested if octave ultimately works correctly.

[mojca (Mojca Miklavec)]

Cc Me!

[jovo@…]

Cc Me!

[beunjb@…]

Octave works with the diff patch

cd $(port dir qrupdate)
sudo patch -p0 < ~/Downloads/qrupdate_Portfile_4.diff

Thanks for the good work.

[rjfergus@…]

Replying to beunjb@…:

Octave works with the diff patch

cd $(port dir qrupdate)
sudo patch -p0 < ~/Downloads/qrupdate_Portfile_4.diff

Thanks for the good work.

Hi,

I have the patch file in my $(port dir qrupdate) directory but when I run the patch command above, I get "can't find file to patch at input line 3" and "Perhaps you used the wrong -p or --strip option?". I notice in the diff file that there are some strange tabs but I don't know the syntax ...

[lawrence.ong@…]

Cc Me!

[lawrence.ong@…]

Using the forth patch, qrupdate and octave both install without error. But when launching octave, I get the following error message:

dyld: Library not loaded: /opt/local/lib/libptcblas.dylib
  Referenced from: /opt/local/lib/libarpack.2.dylib
  Reason: image not found
Trace/BPT trap: 5

[Veence (Vincent)]

The patch should read: BLAS="-L${prefix}/lib -ltatlas" and nothing else behind.

Arpack needs rebuilding too. The Atlas dynamic libraries have changed into libsatlas (single threaded) and libtalas (threaded).

[Veence (Vincent)]

I have committed r93765 that should fix this bug. GCC46 is now default, like atlas, to avoid messing up gfortran libraries. Please test and report.

[ryandesign (Ryan Carsten Schmidt)]

We want all ports having gcc variants to default to the same version. gcc45 is the version we have chosen at this time. If you want to change that default for all ports, you should start a discussion about it on the mailing list.

[Veence (Vincent)]

I wasn’t aware of this. Then you can make the default gcc depend on whether the +atlas variant is set or not. gcc46 in the former case, gcc45 in the latter.

[ryandesign (Ryan Carsten Schmidt)]

Why shouldn't atlas default to gcc45 as well?

[Veence (Vincent)]

Because atlas is all about performance and a gcc46 compiled atlas performs 10 to 15 % better than a gcc45 one. On the other hand, gcc47 improvements over gcc46 (at this time) are moot.

[anddam (Andrea D'Amore)]

Replying to vince@…:

gcc46 compiled atlas performs 10 to 15 % better than a gcc45 one.

That's not a really big enhancement, I vote for keeping the default at gcc45 like other fortran-needing ports.

If an user really needs the speed bonus he/she can enable +gcc46, speaking in a broad sense I'd keep dependencies more streamlined.

[anddam (Andrea D'Amore)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

The reason we default everything to the same gcc version is that subtle hard-to-track-down problems can occur if programs linked with different versions of the C library (i.e. from different versions of the gcc ports) are linked together. I don't know if that affects atlas or not.

[jerome.lelong@…]

If atlas was compiled with gcc46, then qrupdate must alo be compiled with gcc46 otherwise linking fails with missing Fortran symbols. Hence, a gcc46 variant should be added to qrupdate Portfile even if gcc45 remains the default.

[Veence (Vincent)]

The update I have committed have removed obsolete gcc43/44 variants in favor of gcc46/47.

[Veence (Vincent)]

Replying to and.damore@…:

Replying to vince@…:

gcc46 compiled atlas performs 10 to 15 % better than a gcc45 one.

That's not a really big enhancement

That’s your opinion. In the field of HPC, 10% is a lot, especially when all you have to do is change compiler. It’s about half an hour win every four hours. If a user is not concerned with Atlas, he can just opt out by selecting the no_atlas variants and get standard Accelerate framework performance. If he cares about Atlas, then it’s probably to squeeze the most out of his hardware.