atlas @3.10.1_3: threaded libraries missing on PPC dual processor build

[jdgleeson]

Vince, I finally got lucky and captured a log file for you.

[Veence (Vincent)]

John, the culprit is here:

:info:build make[3]: Entering directory `/opt47/local/var/macports/build/_opt47_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.10.1/build/tune/threads'
:info:build make[3]: *** No rule to make target `IThreadTune'.  Stop.

That’s why you get no threaded lib at the end. Now, you should also get a valid altivec vectorizer flag during configure, that obviously you don’t. Please could you selfupdate and try again? Also, before building, could you please send me the contents of the aforementionned Makefile (i.e. in …/atlas-3.10.1/build/tune/threads). Thanks!

[jdgleeson]

I'm afraid I already removed that work directory. It crossed my mind to save a copy of it, and now I regret that I didn't. I expect to have a Makefile for you in about 12 hrs. (and I'll selfupdate)

build/tune/threads is empty in my most recent build, but that may be because it failed while tuning blas/ger -- the same failure I saw when I first tried to build atlas@3.10.1_3. GER tuning is apparently very unstable when other applications are running on my machine, and I had inadvertently left Activity Monitor running while atlas was tuning..

[Veence (Vincent)]

No luck. I bet your dual G5 has never seen such a high compiler activity ;) As you say, Atlas tuning is very sensitive; I guess the process of trying such a high number of kernels with various configuration of prefetching and cache stresses somewhat the memory – if anything interferes at the moment that Atlas is probing for cache parameters, for example, then it might well crash or report wrong figures… I think the best way would be to run Atlas in single user mode ;) Cheers and thanks for both your patience and help.

[cooljeanius (Eric Gallager)]

Cc Me!

[jdgleeson]

Replying to vince@…:

No luck. I bet your dual G5 has never seen such a high compiler activity ;)

Indeed. The fans have been howling.

[jdgleeson]

This build completed normally, and there is no tune/threads/Makefile. I did some searching of the work directory and noticed that Make.topbak has a command for creating tune/threads/Makefile, but Make.top does not. I've attached a file findMakefile.txt that lists the Makefiles generated and the lines that generated them in Make.top. My eyes are getting blurry, so I didn't try to track down how Make.top is generated; besides, you are surely much more familiar with this than I am.

Once again, the main.log file was clobbered just before the activate phase. But I do have all the "port -d" output if you need it. The "-V 2" option shows up now.

[Veence (Vincent)]

Please try again. Ryan pointed out some major flaws in the Portfile, the new version committed in r105259 should work better.

[jdgleeson]

Replying to vince@…:

It can't find liblapack.a. I don't see any sign that it even tries to build the lapack library. Log main-nolapacklib.log.bz2 attached.

[Veence (Vincent)]

I’ll have a look tomorrow. Thanks for reporting.

[Veence (Vincent)]

This was probably caused by the lack of the famous ‘-force_cpusubtype_ALL’ flag, lack that causes some compiler failures. I fixed that in r105312, try again, and thanks for your patience!

[jdgleeson]

That helped! The build terminated normally, and it took only 35 minutes (instead of 12 hours).

The threaded libraries are still not built, though. Presuming I am able to correctly anticipate your next request, the subdirectory build/tune/threads is empty (no Makefile).

The log file has only the activation phase, so I've attached the port -d terminal output instead; see termout.txt.bz2.

[Veence (Vincent)]

Good! At least we know we are striding the right path. I’ll see the log ASAP (tonight), but you have an alternative: instead of ‘port install’, you could you use ‘port build’ which would stop at the end of the build phase (and then you should get the build log…) Thanks for testing on and on!

[Veence (Vincent)]

Got it. There was a line left in the Portfile that was wiping out all Makefiles related to threading for ppc and ppc64 arch. I have moved it elsewhere so that it is executed only on mono-processor systems. Update to r105337 and try again!

[jdgleeson]

Success!! I have the following libraries:

$ ls -l /opt47/local/lib/*atlas* /opt47/local/lib/*blas* /opt47/local/lib/*lapack*
-rw-r--r--  1 root  admin  7302584 Apr 18 13:12 /opt47/local/lib/libatlas.a
-rw-r--r--  1 root  admin   265088 Apr 18 13:12 /opt47/local/lib/libcblas.a
-rw-r--r--  1 root  admin   247832 Apr 18 13:12 /opt47/local/lib/libf77blas.a
<snip gsl cblas libraries>
-rw-r--r--  1 root  admin  7034320 Apr 18 13:12 /opt47/local/lib/liblapack.a
-rw-r--r--  1 root  admin   265128 Apr 18 13:12 /opt47/local/lib/libptcblas.a
-rw-r--r--  1 root  admin   248016 Apr 18 13:12 /opt47/local/lib/libptf77blas.a
-rwxr-xr-x  1 root  admin  9918680 Apr 18 13:12 /opt47/local/lib/libsatlas.dylib
-rwxr-xr-x  1 root  admin  9995004 Apr 18 13:12 /opt47/local/lib/libtatlas.dylib

Thanks for mentioning port build. I should have known, because I used it recently to test a bug fix in matplotlib. Anyway, I followed your suggestion and first did port build. You nearly got another bug report -- I actually puzzled over the output, wondering why it stopped after the build phase when there seemed to be no errors :$

[Veence (Vincent)]

Noooooo :) Atlas is already enough tricky do debug, please no fancy tickets :) Glad it works now. As we say in French, one thorn less in the foot! Enjoy!