atlas: build failure on 10.6.8

[atif.azad@…]

Since I am behind a proxy, I ran the following commands (I am running OS X 10.6.8, and Xcode 3.2.6, macport 2.1.3:

sudo port -d sync

sudo port upgrade outdated

--->  Computing dependencies for atlas
--->  Building atlas
Error: org.macports.build for port atlas returned: command execution failed
Please see the log file for port atlas for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_math_atlas/atlas/main.log
Error: Unable to upgrade port: 1
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets

[larryv (Lawrence Velázquez)]

Please Cc the port maintainer when you open tickets.

:debug:main Skipping completed org.macports.archivefetch (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.fetch (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.checksum (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.extract (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.patch (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.configure (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:build build phase started at Fri May 10 20:45:11 IST 2013

You didn’t clean before building. Please run sudo port clean atlas and try upgrading again. Attach the new main.log if you see the failure.

[larryv (Lawrence Velázquez)]

P.S. Please use WikiFormatting and the Trac Preview functionality when writing tickets.

[larryv (Lawrence Velázquez)]

This looks very similar to #39030. I can’t really judge whether they’re duplicates, though.

[cecilyen@…]

My error log

[cecilyen@…]

I can confirm this error with my main.log uploaded.

[atif.azad@…]

Replying to larryv@…:

Please Cc the port maintainer when you open tickets.

:debug:main Skipping completed org.macports.archivefetch (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.fetch (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.checksum (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.extract (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.patch (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:main Skipping completed org.macports.configure (atlas)
:debug:main Privilege de-escalation not attempted as not running as root.
:debug:build build phase started at Fri May 10 20:45:11 IST 2013

You didn’t clean before building. Please run sudo port clean atlas and try upgrading again. Attach the new main.log if you see the failure.

I have run sudo port clean atlas tens of times but this error persists. I just forgot to mention that in my original post because I didn't run this command just before submitting this ticket.

[larryv (Lawrence Velázquez)]

Replying to atif.azad@…:

I have run sudo port clean atlas tens of times but this error persists.

The point of having you clean before submitting your log is only partly to see if it clears the problem up. The main reason is to give the maintainer a complete log to look at, containing output from every port stage.

Again, please clean and try again and attach the log. A complete log will look like the one cecilyen attached and will not have the “Skipping completed” lines I quoted.

[Veence (Vincent)]

Ok, I have enough info to figure out what’s wrong. Stay tuned for a fix.

[cooljeanius (Eric Gallager)]

Cc Me!

[Veence (Vincent)]

The culprit is this: the AVX detection routine works only on 10.7 or higher but not on 10.6 (because it uses a kernel sysctl that did not exist before 10.7): I wasn’t expecting someone could run SnowLeopard on such a new hardware. I’ll update in a few moments when I figure out a workaround.

[Veence (Vincent)]

Can you tell me please what does:

sysctl hw.cpufamily

outputs on your machine? Thanks.

[atif.azad@…]

Unfortunately I can't provide that output until Monday as the problem was on ,my office machine. But I appreciate you quickly looking into it. Thanks very much

[Veence (Vincent)]

Replying to atif.azad@…:

Unfortunately I can't provide that output until Monday as the problem was on ,my office machine. But I appreciate you quickly looking into it. Thanks very much

You’re most welcome. I hope I’ll be able to find an easy workaround using the cpufamily ID. Otherwise, I’ll have no choice but disabling AVX build on 10.6, which means you would get an halved performance compared to what clang delivers. As an immediate workaround, you can rebuild atlas with the +clang33 option, which should work fine.

[atif.azad@…]

Main.log achieved after running 'port clean atlas' and failing to upgrade atlas

[atif.azad@…]

Hi, here is the output of running 'sysctl'

sysctl hw.cpufamily

hw.cpufamily: 1418770316

Then I ran the following commands:

sudo port clean atlas

sudo port upgrade outdated

Above, the main.log is attached.

[atif.azad@…]

Replying to vince@…:

Replying to atif.azad@…:

Unfortunately I can't provide that output until Monday as the problem was on ,my office machine. But I appreciate you quickly looking into it. Thanks very much

You’re most welcome. I hope I’ll be able to find an easy workaround using the cpufamily ID. Otherwise, I’ll have no choice but disabling AVX build on 10.6, which means you would get an halved performance compared to what clang delivers. As an immediate workaround, you can rebuild atlas with the +clang33 option, which should work fine.

can you please advise on how to use +clang33 option. I don't want to mess it up given my little background in changing macport files. Thanks

[Veence (Vincent)]

I have modified the Portfile to allow for detection of AVX hardware on 10.6 in r106013. Please update and try again.

[atif.azad@…]

Thanks. I have updated the port tree.

now running

sudo port upgrade outdated
.
.
Selected C compiler: /opt/local/bin/clang-mp-3.3
Selected F77 compiler: gfortran47
--->  Building atlas

It's taking forever though. Is this slow down in build process is expected?

[Veence (Vincent)]

You now get the correct compiler set. Good. Atlas was meant to be compiled with gccXX for which it has an extended list of precomputed figures. With clang, it must go through an extensive process of finding those figures that optimize computation time. But it pays off, since you’re rewarded with AVX assembly code that doubles the efficiency wrt to gccXX which, on OS X, is crippled to SSE4.2. Please report when the build is done, for me to close the bug. Thanks!

[atif.azad@…]

Thanks. I am still waiting. It is still building Atlast after 2 hours. No further messages printed on the console yet.

[atif.azad@…]

Hi Vince,

Atlas has now built successfully. Feel free to close the bug. Many thanks for your help.

Kind Regards

[Veence (Vincent)]

Fine! Have fun.

[atif.azad@…]

main.log-Mon 11 Aug 2014 15:12:50 IST

[atif.azad@…]

This issue has come up again. Atlast won't build. Please see the main.4.log for details. It was produced after running sudo port clean atlas

--->  Configuring atlas
Selected C compiler: /usr/bin/clang
Selected F77 compiler: gfortran47
--->  Building atlas
Error: org.macports.build for port atlas returned: command execution failed
Please see the log file for port atlas for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_math_atlas/atlas/main.log
Error: Unable to upgrade port: 1

Running OS X 10.9.4 and Macports 2.3.1

$ sysctl hw.cpufamily
hw.cpufamily: 1418770316

[Veence (Vincent)]

This is caused by an error in your gcc4.7 implementation. Either try reinstalling gcc47 or upgrade to a later version, and let me know.

:info:configure dyld: Library not loaded: /opt/local/lib/libisl.10.dylib
444	:info:configure   Referenced from: /opt/local/libexec/gcc/x86_64-apple-darwin10/4.7.3/f951
445	:info:configure   Reason: image not found
446	:info:configure gfortran-mp-4.7: internal compiler error: Trace/BPT trap: 5 (program f951)
447	:info:configure Please submit a full bug report,
448	:info:configure with preprocessed source if appropriate.
449	:info:configure See <https://trac.macports.org/newticket> for instructions.
450	:info:configure make[2]: *** [IRunF77Comp] Error 4

[atif.azad@…]

I tried the following command:

sudo port upgrade gcc47

however, it returned further errors:

Error: llvm-3.1 is not supported on Mavericks or later.
Error: org.macports.fetch for port llvm-3.1 returned: unsupported platform
Please see the log file for port llvm-3.1 for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.1/llvm-3.1/main.log
Error: Unable to upgrade port: 1

[Veence (Vincent)]

You have to install llvm34, which is the most modern version of the LLVM compiler infrastructure first. You must upgrade ld64 and cctools to use llvm34 too.

[atif.azad@…]

I have installed llvm-3.4 but it triggers installing other software, which in turn seems to require llvm-3.1

--->  Found 25 broken file(s), matching files to ports   
--->  Found 2 broken port(s), determining rebuild order
--->  Rebuilding in order
     gcc47 @4.7.3 
     texlive-bin @2013 +x11
--->  Computing dependencies for llvm-3.1
--->  Fetching distfiles for llvm-3.1
Error: llvm-3.1 is not supported on Mavericks or later.
Error: org.macports.fetch for port llvm-3.1 returned: unsupported platform

I wonder how I can set it so that llvm-3.1 is never used. Couldn't this have been disabled by default?

[Veence (Vincent)]

Can you upload the whole rundown of your installed ports? (ie the output of ‘port installed’). Thanks!

[atif.azad@…]

run down of "post installed"

[atif.azad@…]

Sorry. Meant "port installed' up there.

[Veence (Vincent)]

First of all, you should uninstall all ports you don't use anymore! Your cctools and ld64 ports are still built over llvm31. Please do:

port install ld64 +llvm34
port install cctools +llvm34
port uninstall inactive

And then can you upload the rundown again? Thanks!

[atif.azad@…]

New list of installed ports.

[atif.azad@…]

Upon running {{{ port uninstall inactive }}}

I got following error:

--->  Unable to uninstall libstdcxx @4.8-20130411_0, the following ports depend on it:
--->  	gcc44 @4.4.7_6
--->  	gcc45 @4.5.4_7
Error: org.macports.uninstall for port libstdcxx returned: Please uninstall the ports that depend on libstdcxx first.

[Veence (Vincent)]

Uninstall manually gcc44 an gcc45 (port uninstall gcc44 then port uninstall gcc45) and try again.

[atif.azad@…]

Well I had upgraded gcc44 and gcc45 in the mean time:

So now I have following

  gcc44 @4.4.7_8 (active)
  gcc45 @4.5.4_9 (active)
  gcc47 @4.7.3_5 (active)
  gcc_select @0.1_8 (active)

Uninstalling the inactive ports went fine after that.

However, building atlas after that seems do nothing in that it seems to stop at:

sudo port upgrade outdated
--->  Building atlas

Terminal window shows a process "xatlast_tee" running. But when I run

ps aux | grep xatlas
macports        62791   0.0  0.0  2440952    512 s001  S+    4:11pm   0:00.00 ./xatlas_tee INSTALL_LOG/dMMSEARCH.LOG
macports        62789   0.0  0.0  2444632    972 s001  S+    4:11pm   0:00.00 sh -c make -f Makefile IRunMMDef pre=d 2>&1 | ./xatlas_tee INSTALL_LOG/dMMSEARCH.LOG^J
macports        60657   0.0  0.0  2451244    612 s001  S+    4:09pm   0:00.00 ./xatlas_build -1 0 -a 1 -l 1
macports        60656   0.0  0.0  2434392    968 s001  S+    4:09pm   0:00.00 /bin/sh -c cd bin/ ; ./xatlas_build -1 0 -a 1 -l 1

it appears the processes are idle.

[Veence (Vincent)]

Normal. Xatlas_tee is the process that writes the log, and it is niced at a very low priority. The actual processes bear different names according to the kernel they compute.

[atif.azad@…]

Thanks for all the help thus far. It still seems to be stuck at that. Perhaps it is also normal to take very long? A few hours?

[Veence (Vincent)]

Atlas takes anywhere from three to five hours to build, depending on your processor. It optimizes core by core, so having many cores does not help. Just be patient!

[larryv (Lawrence Velázquez)]

Replying to atif.azad@…:

It still seems to be stuck at that. Perhaps it is also normal to take very long? A few hours?

More than a few. atlas @3.10.2_1+clang+universal took about 6 hours to build on my MacBookPro11,3 and has been building for at least 12 hours on my MacBook5,1.

If you want some visual feedback, you could terminate the current upgrade and run it again as port -v upgrade, which will print the usual build output. Or, like Vincent said, you could just wait it out.

[atif.azad@…]

Thanks for the feedback. I guess I should wait until tomorrow morning.

[Veence (Vincent)]

I guess, yes. Then tell me.

[atif.azad@…]

atlas installed but llvm-3.1 continues to pester.

--->  Attempting to fetch lcms-1.19_5.darwin_13.x86_64.tbz2 from http://packages.macports.org/lcms
--->  Attempting to fetch lcms-1.19_5.darwin_13.x86_64.tbz2.rmd160 from http://packages.macports.org/lcms
--->  Computing dependencies for lcms
--->  Unable to uninstall lcms @1.19_5, the following ports depend on it:
--->  	libmng @1.0.10_3
Warning: Uninstall forced.  Proceeding despite dependencies.
--->  Deactivating lcms @1.19_5
--->  Unable to deactivate lcms @1.19_5, the following ports depend on it:
--->  	libmng @1.0.10_3
Warning: Deactivate forced.  Proceeding despite dependencies.
--->  Cleaning lcms
--->  Uninstalling lcms @1.19_5
--->  Cleaning lcms
--->  Computing dependencies for lcms
--->  Installing lcms @1.19_5
--->  Activating lcms @1.19_5
--->  Cleaning lcms
--->  Fetching archive for libmng
--->  Attempting to fetch libmng-1.0.10_3.darwin_13.x86_64.tbz2 from http://packages.macports.org/libmng
--->  Attempting to fetch libmng-1.0.10_3.darwin_13.x86_64.tbz2.rmd160 from http://packages.macports.org/libmng
--->  Computing dependencies for libmng
--->  Unable to uninstall libmng @1.0.10_3, the following ports depend on it:
--->  	qt4-mac @4.8.6_0
Warning: Uninstall forced.  Proceeding despite dependencies.
--->  Deactivating libmng @1.0.10_3
--->  Unable to deactivate libmng @1.0.10_3, the following ports depend on it:
--->  	qt4-mac @4.8.6_0
Warning: Deactivate forced.  Proceeding despite dependencies.
--->  Cleaning libmng
--->  Uninstalling libmng @1.0.10_3
--->  Cleaning libmng
--->  Computing dependencies for libmng
--->  Installing libmng @1.0.10_3
--->  Activating libmng @1.0.10_3
--->  Cleaning libmng
--->  Computing dependencies for llvm-3.1
--->  Fetching distfiles for llvm-3.1
Error: llvm-3.1 is not supported on Mavericks or later.
Error: org.macports.fetch for port llvm-3.1 returned: unsupported platform
Please see the log file for port llvm-3.1 for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.1/llvm-3.1/main.log

[atif.azad@…]

Ignore the last message: got rid of dependencies of llvm-3.1 and uninstalled llvm-3.1.

Now going ahead with upgrading the outdated ports.

[Veence (Vincent)]

Okay, great. At least for Atlas, I consider this ticket closed.