atlas takes a long time to build

[ryantkoehler@…]

Hi

I am trying to install bluefish and am getting stuck at the "Building atlas" stage of the process. What I mean by stuck is that the system just sort of hangs there (let it go for nearly an hour more than once, cleaning in between with "port clean atlas").

I've seen some messages (e.g. "needs lapack") that seem to be related, but don't really have a clue what to do about this? I'm new to macports (but not unix / linux) so maybe there are some "obvious" things to try on the mac?

Here's the non-verbose output (verbose attached):

Ryans-MacBook-Pro [~/play] 143> sudo port install atlas
--->  Computing dependencies for atlas
--->  Fetching atlas
--->  Verifying checksum(s) for atlas
--->  Extracting atlas
--->  Applying patches to atlas
--->  Configuring atlas
--->  Building atlas

[jmroot (Joshua Root)]

Please remember to preview and use WikiFormatting, fill in the Port field, and cc the maintainer.

[jmroot (Joshua Root)]

There's no error in the attached log, you just ctrl-C'd the build process while it was still in progress.

[ryantkoehler@…]

Replying to jmr@…:

There's no error in the attached log, you just ctrl-C'd the build process while it was still in progress.

Yes, the attached log ends with a ^C but this was after ~20 minutes just sitting at that stage. Previously I let the install process sit for more than an hour several times but it never progressed past the "Building atlas" stage. Thus, it appears to me that something isn't right?

[kadorken@…]

I have observed the same problem trying to build GIMP (sudo port install gimp) on MACOS 10.6 (new machine). At least two processors have been busy for 3 hours on an otherwise idle machine..

Will ctrl-d it for now as I have other things to do.

[ryandesign (Ryan Carsten Schmidt)]

Atlas is a large piece of software. It wouldn't surprise me if it needs several hours to build, depending on the machine. Just let it run.

[tiago@…]

I think this ticket can be closed. I've just done the installation (Mac OX 10.6.6 intel i7 8Gb) and it took approximately 1hr and 30 minutes to finish the build and the installation was successful. So, yes, accordingly with Ryan, it might take more than 2 hours depending on the machine.

[tonior@…]

Cc Me!

[jmroot (Joshua Root)]

What was done to fix this? If nothing, the resolution should be worksforme, invalid, or wontfix; not fixed.

[Veence (Vincent)]

You're right (nothing was done, it just happens that 3.8.3 is outdated and is replaced by a newer version). I just mistakenly chose the wrong option and didn't realize it until it was committed…

[rakatan@…]

Cc Me!

[rakatan@…]

does not work. for me, the building atlas process is stuck for 2 or more hours, I have i5 dual core, and there is no process, which would use that cpu (checked in top).

[rakatan@…]

retried several times, waited for it to do seemingly nothing :) after some retry I got compile error.. then cleared working dir (sudo port clean atlas) and retried again - and this time it started building, and it was high cpu usage for both compiling and tuning stages

[rakatan@…]

(forgot to mention, that the last time was actually successfull)

[marco-m (Marco Molteni)]

On my MacBookPro, MacOSX 10.6.8, ports tree refreshed as of today 2011-07-26, the "hang" is actually a crash of the build process ("ld" crashes!), that seems not to be detected by the macports driver, so that it seems to hang. Quite severe IMO, the macports driver must be modified to detect this kind of crash and a bug must be filed with Apple to fix their ld, which crashes. Here is the end of the build log:

======================= :info:build BEGIN STAGE 5-1-0: FINAL STATIC LIBRARY UPDATE at 08:47 :info:build make -f Makefile IBuildLibs IBuildPtlibs0 2>&1 | ./xatlas_tee INSTALL_LOG/LIBUPDATE.LOG :info:build DONE STAGE 5-1-0 at 08:47 :info:build :info:build :info:build :info:build :info:build ATLAS install complete. Examine :info:build ATLAS/bin/<arch>/INSTALL_LOG/SUMMARY.LOG for details. :info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build' :info:build /usr/bin/make clean :info:build make[1]: Entering directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build' :info:build rm -rf *.dSYM :info:build rm -rf *.o x* config?.out *core* :info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build' :info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build' :debug:build Executing proc-post-org.macports.build-build-0 :info:build sh: line 1: 92765 Segmentation fault /usr/bin/ld -arch x86_64 -dynamic -dylib -single_module -dead_strip -x -all_load -L. -L/opt/local/lib/gcc44/x86_64 -L/opt/local/lib/gcc44 -ldylib1.o -dylib_install_name /opt/local/lib/libatlas.dylib libatlas.a -o libatlas.dylib -lSystem

:info:build shell command "cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build/lib && ( test ! -e libatlas.a

/usr/bin/ld -arch x86_64 -dynamic -dylib -single_module -dead_strip -x -all_load -L. -L/opt/local/lib/gcc44/x86_64 -L/opt/local/lib/gcc44 -ldylib1.o -dylib_install_name /opt/local/lib/libatlas.dylib libatlas.a -o libatlas.dylib -lSystem )" returned error 139

:error:build Target org.macports.build returned: shell command failed (see log for details) :debug:build Backtrace: shell command failed (see log for details)

while executing

"$post $targetname" :info:build Warning: the following items did not execute (for atlas): org.macports.install org.macports.build org.macports.destroot ======================

Here is how to reproduce by hand by reissuing the command from the log, /usr/bin/ld crashes:

$ cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build/lib

$ /usr/bin/ld -arch x86_64 -dynamic -dylib -single_module -dead_strip -x -all_load -L. -L/opt/local/lib/gcc44/x86_64 -L/opt/local/lib/gcc44 -ldylib1.o -dylib_install_name /opt/local/lib/libatlas.dylib libatlas.a -o libatlas.dylib -lSystem Segmentation fault

[marco-m (Marco Molteni)]

Cc Me!

[jmroot (Joshua Root)]

Later comments in this ticket have nothing to do with the description given by the reporter. Yes, atlas can take a very long time to build, and it doesn't look like there's really anything that can be done about that in the port.

[bpabbott@…]

Atlas is taking a long time on my Intel i7 and is consuming less than 20% of my available CPU(s). Any reason the build process isn't using more cores?

[Veence (Vincent)]

Atlas has its own parallel build process. Since all cores share L2 cache, it cannot safely run time measurements on two or more cores simultaneously, because this could cause spurious cache coherency and introduce a bias. That’s why it does compile several pieces of code in parallel, but tests each one serially.

[Veence (Vincent)]

Besides, at the end, you will get almost a 2x faster Atlas thanks to AVX-based kernels that are not compilable with MacOS GCCs. So it’s worth the patience!