#30280 closed defect (fixed)
Atlas @3.9.37 +GCC44 : STAGE 2-3-2: CacheEdge DETECTION FAILED
| Reported by: | pierre@… | Owned by: | macports-tickets@… |
|---|---|---|---|
| Priority: | Normal | Milestone: | |
| Component: | ports | Version: | 2.0.99 |
| Keywords: | lion | Cc: | jameskyle@…, faisal.moledina@…, marc.schlaich@…, wojtekwn@…, rabbielvis@…, randompeach@…, ecbrown (Eric Brown), talezshin@…, jacobu@…, st.hennig@…, rex4539 (Dimitris Apostolou), ith140@…, MasakiOita, stromnov (Andrey Stromnov), cod3monk (Julian), axeljaeger@…, rsachdev@…, dmirkitanov@…, captainproton1971 (Captain Proton), nicholas.north@…, m.haller@…, kwasir@…, cdeil (Christoph Deil), mb86@…, melanochaitus@…, michelle.lynn.gill@…, gjznituv@…, sperka@…, patrice.kouame@…, sander@…, danmichaelo+macports@…, muchatel@…, fyu@…, jeff.wilson@…, geoffgrimwood, maurice@…, steve@…, lawrence.ong@…, jowens@…, yamada.manabu.1207@…, tzonghao@…, adolfo.benedetti@…, bonoba@…, a.y.harano@…, smcavoy@…, christoph.jacob@…, tonior@…, fyu+macports@…, thomas@…, axel.kirch@…, fleason (Fred Leason), derekathomas@…, lukas.reichlin@… |
| Port: | atlas |
Description
Fresh SVN install, Xcode 4.2 on Mac OS 10.7 Lion on MPB 13" Mid 2010. Atlas does not build (see log attached).
DONE STAGE 2-3-1 at 17:39
BEGIN STAGE 2-3-2: CacheEdge DETECTION at 17:39
make -f Makefile INSTALL_LOG/atlas_zdNKB.h pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMCACHEEDGE.LOG
make[1]: *** [build] Error 255
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_math_atlas/atlas/work/atlas-3.9.37/build'
make: *** [build] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_math_atlas/atlas/work/atlas-3.9.37/build'
shell command " cd "/opt/local/var/macports/build/_opt_mports_trunk_dports_math_atlas/atlas/work/atlas-3.9.37/build" && /usr/bin/make -w build " returned error 2
Error: Target org.macports.build returned: shell command failed (see log for details)
DEBUG: Backtrace: shell command failed (see log for details)
while executing
"command_exec build"
(procedure "portbuild::build_main" line 8)
invoked from within
"$procedure $targetname"
Warning: the following items did not execute (for atlas): org.macports.activate org.macports.build org.macports.destroot org.macports.install
Log for atlas is at: /opt/local/var/macports/logs/_opt_mports_trunk_dports_math_atlas/atlas/main.log
Error: Status 1 encountered during processing.
To report a bug, see <http://guide.macports.org/#project.tickets>
Attachments (14)
Change History (163)
Changed 13 years ago by pierre@…
Changed 13 years ago by faisal.moledina@…
| Attachment: | zMMCACHEEDGE.LOG added |
|---|
The relevant zMMCACHEEDGE.LOG
comment:1 Changed 13 years ago by faisal.moledina@…
In /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/bin/INSTALL_LOG. The file zMMCACHEEDGE.LOG ends with the following error.
/usr/bin/ranlib: file: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/lib/libatlas.a(ATL_dtrmv.o) has no symbols /usr/bin/ranlib: file: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/lib/libatlas.a(ATL_strmv.o) has no symbols ar: fatal error in /usr/bin/ranlib make[6]: *** [dlib.grd] Error 1 make[6]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/src/blas/gemm' make[5]: *** [dlib] Error 2 make[5]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/src/blas/gemm' make[4]: *** [dmmlib] Error 2 make[4]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/tune/blas/gemm' make[3]: *** [res/atlas_zdNKB.h] Error 2 make[3]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/tune/blas/gemm' make[2]: *** [/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/tune/blas/gemm/res/atlas_zdNKB.h] Error 2 make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/bin'
comment:12 Changed 13 years ago by ryandesign (Ryan Carsten Schmidt)
| Cc: | eric.c.brown@… shengcer@… talezshin@… added |
|---|
Has duplicate #30296.
comment:26 Changed 13 years ago by captainproton1971 (Captain Proton)
| Cc: | captainproton1971@… added |
|---|
Cc Me!
comment:34 Changed 13 years ago by raimue (Rainer Müller)
| Keywords: | lion added |
|---|
comment:38 Changed 13 years ago by patrice.kouame@…
I have some relevant attachments from unsuccessful build/install @
comment:39 follow-up: 41 Changed 13 years ago by Veence (Vincent)
I am working on the newest version of Atlas (3.9.46) with lapack 3.3.1 and a mix from clang/gfortran. I'll keep you posted.
comment:41 Changed 13 years ago by patrice.kouame@…
Replying to vince@…:
I am working on the newest version of Atlas (3.9.46) with lapack 3.3.1 and a mix from clang/gfortran. I'll keep you posted.
FYI... I just tried building (standalone from website) 3.9.45 on my system with Xcode 4.2 (latest beta4) installed and it also fails...Same CacheEdge Detection issue.
my system gcc is currently:
gcc -v Using built-in specs. Target: i686-apple-darwin11 Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1 Thread model: posix gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)
NOTE:
I configure with --nof77
I have zipped logs still available if it helps...
comment:49 Changed 13 years ago by Veence (Vincent)
I am making significant progress, but since compiling Atlas on my old MacBook (late 2008) takes more than 5 hours, it is painful. I have no cache edge detection problems. The biggest hurdle I've encountered so far is a bug in ar on Lion: "ar r" crashes while calling libtool. I've written a little sh script to use libtool directly instead, as a workaround, but I expect the final archive to be unusable (libtool does not replace .o in the archive like ar r does. It just simply adds them warning of a duplicate; maybe I should call ranlib just after). I have filed a radar about this.
There are some other issues like some non-void functions returning no value that causes error with clang, assembler instructions lacking size specification, etc. I'm going to forwards them upstream to Clint.
comment:56 Changed 13 years ago by Veence (Vincent)
Ok. The situation is thus: Atlas *compiles* all right. But, on 10.7, ar is buggy, so it does not link (fatal error in ar). I have written a shell script that simulates 'ar r' by calling libtool directly, but the shell is yet too primitive and the dylib generation at the end fails because of duplicate symbols. I am going to try to modify the script so that it emulates perfectly 'ar r' (replacing already stored .o instead of storing everything blindly). I'll keep you posted.
comment:57 Changed 13 years ago by Veence (Vincent)
Good. I have a 10.7 clean build with the latest version 3.9.46/lapack 3.3.1 and gcc45. I'll post the updated Portfile and patch files later today.
Changed 13 years ago by Veence (Vincent)
| Attachment: | atlas_10.7_gcc45.tgz added |
|---|
Portfile and patch files for 3.9.46/3.3.1
comment:59 follow-ups: 61 84 96 Changed 13 years ago by Veence (Vincent)
So, I append a .tgz file containing the Portfile and all the patch files. Since this is highly experimental, it is given as is (not in the form of a diff), without any guarantee, and I strongly advise you to unpack it in a private section of your ports tree (see Macports manual to figure out how to setup one). If you feel adventurous, you can extract it directly in the atlas repertory of the main tree: You can always revert to previous state by wiping out everything and syncing again.
It should work on all MacOS versions with gcc45 in 64-bit mode. All other combinations are not yet tested. I've an old MacMini CoreDuo/10.6 machine on which I should test building it tonight, and a G5 server too.
Support for clang is almost here too, but since clang is stricter than gcc, there are some extra patch files to be generated. Besides, building with clang takes hours, whereas building with gcc is a matter of minutes. I don't know why.
Good luck, and please tell me if you succeed or not.
comment:60 Changed 13 years ago by Veence (Vincent)
Some additional elements I forgot:
- When I say "wiping out everything", I meant in the atlas directory, of course;
- It is normal, on Lion, to get ar warnings about not found object files. This is part of the workaround.
comment:61 Changed 13 years ago by carbncl@…
Replying to vince@…:
Good luck, and please tell me if you succeed or not.
Strangely I did not need atlas anymore, it might have been removed of dependencies of <no idea> package. Anyway, just tried to build it with your attached files, works perfectly, good job! :)
comment:62 follow-up: 63 Changed 13 years ago by Veence (Vincent)
Thanks! ;) I am currently building it the CoreDuo/10.6 macmini and the G5/10.5 server. We'll see in a few hours if it also succeed on these configurations.
comment:63 Changed 13 years ago by carbncl@…
Replying to vince@…:
Thanks! ;) I am currently building it the CoreDuo/10.6 macmini and the G5/10.5 server. We'll see in a few hours if it also succeed on these configurations.
You're welcome :) FYI, I built it on a Core 2 duo MBP / 10.7.
comment:64 follow-ups: 70 91 Changed 13 years ago by Veence (Vincent)
Great. That's actually almost the same configuration as mine (MB Core 2 duo late 2008). I wonder what will come out the newer machines (various Core i3/5/7). This version of Atlas should be able to use the latest AVX instructions set, too.
comment:65 Changed 13 years ago by Veence (Vincent)
The same ar bug appears to be present on the 32-bit SL build. I'll to modify the port file accordingly.
comment:66 Changed 13 years ago by faisal.moledina@…
@vince: builds perfectly for me. Same configuration, MB C2D late 2008. On my way to building NumPy now. Thanks.
comment:67 Changed 13 years ago by Veence (Vincent)
@faisal: cool. Glad it works. gcc45 seems not to be usable on G4/G5: internal compiler error on some file. I'll have to disable the option for those processors.
comment:68 Changed 13 years ago by jowens@…
@vince: Also works fine for me on a newer MBP (Core i7). Woo! Thanks Vince, I know that was a lot of work.
Took under an hour to compile, for the record.
comment:69 Changed 13 years ago by Veence (Vincent)
Nice! Since you have a powerful machine, maybe I'll ask your help to build the future clang/dragonegg option, if you don't mind. Meanwhile, enjoy, and thanks again for your message.
comment:70 Changed 13 years ago by michelle.lynn.gill@…
Replying to vince@…:
Great. That's actually almost the same configuration as mine (MB Core 2 duo late 2008). I wonder what will come out the newer machines (various Core i3/5/7). This version of Atlas should be able to use the latest AVX instructions set, too.
@vince: I just built your version of atlas using gcc44 in Lion on my MBP mid2010, which has an i5 processor.
Atlas built in around 10 minutes for me. I assume this was so fast because the tests atlas performs during normal installation have been disabled?
Also, I've been able to build py26-numpy, py26-scipy, and py26-cvxopt against your version of atlas. Fantastic! Thanks!
comment:71 Changed 13 years ago by Veence (Vincent)
Great to know it works with gcc44 too on a "modern" processor.
I have no real answer to why it compiles so quickly. I will ask upstream. Maybe some configurations are pre-recorded somewhere, recognized, and all the lengthy test phase is skipped? Right now, I have no better hypothesis.
Please run the scipy/numpy tests, tell me if it works properly.
Changed 13 years ago by faisal.moledina@…
| Attachment: | py27-numpy+scipy-tests.txt added |
|---|
numpy.test() and scipy.test() for py27 ports + vince' atlas fix
comment:72 Changed 13 years ago by faisal.moledina@…
Just attached the output of numpy.test() and scipy.test() after building py27-numpy and py27-scipy agains the fixed atlas port. Some failures; not sure if they're atlas related or if it's just a combination of new NumPy version + 10.7 issues.
comment:73 follow-up: 74 Changed 13 years ago by Veence (Vincent)
Hmmm… The scipy errors look suspicious. I'll try myself tomorrow. Thanks for your time.
Changed 13 years ago by michelle.lynn.gill@…
| Attachment: | py26-numpy-test-lion.txt added |
|---|
Changed 13 years ago by michelle.lynn.gill@…
| Attachment: | py26-scipy-test-lion.txt.txt added |
|---|
comment:74 Changed 13 years ago by michelle.lynn.gill@…
Replying to vince@…:
Hmmm… The scipy errors look suspicious. I'll try myself tomorrow. Thanks for your time.
Just attached the results of my tests.
I had some failures as well using py26-numpy and py26-scipy compiled on Lion against the experimental atlas. I have my old MacPorts library (compiled on Snow Leopard) at home and I can repeat the tests the same hardware using the old MacPorts install with Lion and then with the old MacPorts install and a clone of my old (Snow Leopard) system.
comment:75 Changed 13 years ago by Veence (Vincent)
I don't think the failures you have are really caused by atlas, so I am not too much worried. Numpy failures seem to be related to borderline computations involving infinite and NaN, these are not caused by atlas and anyhow not worrisome. The scipy failure does not appear to be caused by atlas either, but, what is a bit strange, is that you do not get the same errors as faisal… I'll try on my own, see how it turns out, and keep you posted. V.
comment:76 Changed 13 years ago by talezshin@…
Mac Mini 2010 (not server) which has almost identical configuration with MBP 2008 late confirmed successful on atlas. Now working on octave!
Changed 13 years ago by michelle.lynn.gill@…
| Attachment: | py26-scipy-test-lion-mp1.9.2.txt added |
|---|
Changed 13 years ago by michelle.lynn.gill@…
| Attachment: | py26-scipy-test-snowleo-mp1.9.2 added |
|---|
comment:82 Changed 13 years ago by michelle.lynn.gill@…
Above I have attached two more results from scipy.test() on my MBP X,2. The first was performed using Mac OS X 10.7 with scipy/atlas from the most recent version of my MacPorts library that was compiled under Mac OS X 10.6.8. (This version of the library utilized MacPorts 1.9.2 since I waited to upgrade to MacPorts 2.0 until I installed Lion.) The second version utilizes the old version of the library under Snow Leopard (Mac OS X 10.6.8).
Based on all three scipy tests (the third being the MacPorts library I compiled under Lion using Vince's version of atlas), the same test fails every time--"test_morestats.TestAnderson". I believe this suggests that, at least in my case, any issues encountered are not related to either Lion or Vince's atlas compilation.
Identical hardware was used for the tests and for the compilation of both versions of the MacPorts library.
If you'd like me to perform any other tests on this system, let me know.
comment:83 Changed 13 years ago by michelle.lynn.gill@…
Oops, that should say "MBP 6,2" not "MBP X,2"
comment:84 Changed 13 years ago by patrice.kouame@…
Replying to vince@…:
So, I append a .tgz file containing the Portfile and all the patch files. Since this is highly experimental, it is given as is (not in the form of a diff), without any guarantee, and I strongly advise you to unpack it in a private section of your ports tree (see Macports manual to figure out how to setup one). If you feel adventurous, you can extract it directly in the atlas repertory of the main tree: You can always revert to previous state by wiping out everything and syncing again.
It should work on all MacOS versions with gcc45 in 64-bit mode. All other combinations are not yet tested. I've an old MacMini CoreDuo/10.6 machine on which I should test building it tonight, and a G5 server too.
Support for clang is almost here too, but since clang is stricter than gcc, there are some extra patch files to be generated. Besides, building with clang takes hours, whereas building with gcc is a matter of minutes. I don't know why.
Good luck, and please tell me if you succeed or not.
V! Thank you so much for this prompt fix!
I built against gcc44 and gcc45 on Lion with the latest Xcode 4.2 (from the bleeding edge iOS 5 beta 4) with great success. All in a local ports repository as you suggested. I tested meld with all the python dependencies and it all works fine.
By the way the builds on my box take about 15 minutes. I have a Mac Pro dual quad-core Xeon. I know I'm lucky. If you need help with your long compiles/build just ask...you have my email...
Also decided no to push my luck with gcc46 and clang or llvm 3.0. Will wait for your official distros.
I think your ar workaround may give me clues to other stuff that currently breaks on Lion. You filed a radar bug, may I have the number?
Thanks again, Patrice
comment:85 follow-ups: 89 90 Changed 13 years ago by Veence (Vincent)
You're welcome. Thanks for testing with gcc44 and 45, and for the proposal. I will probably add a dragonegg port soon, so that we can enjoy LLVM superior performance both for C and for Fortran. I'll tell you as soon as it is ready, so maybe you can do some performance comparisons. For the 'ar r' bug, that is also present on SL (but has obviously gone unnoticed), the radar # is 9830754.
Cheers, have fun! Vincent
comment:89 Changed 13 years ago by michelle.lynn.gill@…
Replying to vince@…:
For the 'ar r' bug, that is also present on SL (but has obviously gone unnoticed), the radar # is 9830754.
@Vince
If the 'ar r' bug existed under Snow Leopard, do you know why it (apparently) didn't cause an issue with atlas builds on that system? I'm asking this out of nothing more than curiosity, so no worries if you're busy with other things.
Thanks.
comment:90 Changed 13 years ago by patrice.kouame@…
Replying to vince@…:
You're welcome. Thanks for testing with gcc44 and 45, and for the proposal. I will probably add a dragonegg port soon, so that we can enjoy LLVM superior performance both for C and for Fortran. I'll tell you as soon as it is ready, so maybe you can do some performance comparisons. For the 'ar r' bug, that is also present on SL (but has obviously gone unnoticed), the radar # is 9830754.
Cheers, have fun! Vincent
Yes to Dragonegg. I personally am impressed with all the LLVM goodness wrapped in the latest Xcode builds (3.0 is impressive). Dragonegg seems pretty solid for C. The authors claim there are a few quirks with C++ and boost which worry me though (and I use boost a lot). Arguably a good direction for you since Fortran support is critical. Pretty bleeding edge, but then you know what you're doing (smile)...Good luck.
Any interest in my build logs? The patch blew them away but I can re-generate.
Patrice
comment:91 Changed 13 years ago by wojtekwn@…
Replying to vince@…:
Great. That's actually almost the same configuration as mine (MB Core 2 duo late 2008). I wonder what will come out the newer machines (various Core i3/5/7). This version of Atlas should be able to use the latest AVX instructions set, too.
Works great on my 2009 MB Pro (Core 2 Duo), but not on my 2011 iMac (Core i7).
comment:92 follow-up: 101 Changed 13 years ago by Veence (Vincent)
The log suggests that gcc45 might be outdated for the Sandy bridge line of processors: the gas45 assembler seems not to be aware of the brand new AVX instructions atlas already uses. Try to build gcc46 and use the +gcc46 option instead.
comment:93 follow-up: 95 Changed 13 years ago by adolfo.benedetti@…
I just tried with gcc45 on Lion;
$ sudo port -v install atlas +gcc45
and after 4 hours of build process, I got;
make -f Makefile INSTALL_LOG/zMMRES.sum pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG
The best matmul kernel was SSEGENOUTDIR/zgenmm_sse.c, NB=60, written by Zalkin & Whaley
Performance: 8979.05MFLOPS (374.13 percent of of detected clock rate)
(Gen case got 4546.46MFLOPS)
make -f Makefile INSTALL_LOG/zNCNB pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOGmake -f Makefile INSTALL_LOG/zbestNN_40x40x40 pre=z nb=40 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG NCgemmNN : muladd=1, lat=2, pf=1, nb=40, mu=4, nu=2 ku=40,
ForceFetch=1, ifetch=4 nfetch=2
Performance = 4273.02 (47.59 of copy matmul, 178.04 of clock)
make -f Makefile INSTALL_LOG/zbestNT_40x40x40 pre=z nb=40 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG NCgemmNT : muladd=1, lat=4, pf=1, nb=40, mu=4, nu=2 ku=40,
ForceFetch=1, ifetch=4 nfetch=2
Performance = 3959.74 (44.10 of copy matmul, 164.99 of clock)
make -f Makefile INSTALL_LOG/zbestTN_40x40x40 pre=z nb=40 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG NCgemmTN : muladd=1, lat=2, pf=1, nb=40, mu=4, nu=2 ku=40,
ForceFetch=1, ifetch=4 nfetch=2
Performance = 4321.79 (48.13 of copy matmul, 180.07 of clock)
make -f Makefile INSTALL_LOG/zbestTT_40x40x40 pre=z nb=40 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG NCgemmTT : muladd=1, lat=5, pf=1, nb=40, mu=4, nu=2 ku=40,
ForceFetch=1, ifetch=4 nfetch=2
Performance = 4064.43 (45.27 of copy matmul, 169.35 of clock)
make -f Makefile MMinstall pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG
DONE STAGE 2-3-1 at 13:13
BEGIN STAGE 2-3-2: CacheEdge DETECTION at 13:13
make -f Makefile INSTALL_LOG/atlas_zdNKB.h pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMCACHEEDGE.LOG
make[1]: *** [build] Error 255
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build'
make: *** [build] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build'
shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build" && /usr/bin/make -w build " returned error 2
Error: Target org.macports.build returned: shell command failed (see log for details)
Warning: the following items did not execute (for atlas): org.macports.activate org.macports.build org.macports.destroot org.macports.install
Log for atlas is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/main.log
Error: Status 1 encountered during processing.
To report a bug, see <http://guide.macports.org/#project.tickets>
Changed 13 years ago by adolfo.benedetti@…
| Attachment: | main_adolfo.benedetti_31.07.11_gcc45_lion.log added |
|---|
comment:95 Changed 13 years ago by Veence (Vincent)
Ciao Adolfo,
Replying to adolfo.benedetti@…:
I just tried with gcc45 on Lion;
$ sudo port -v install atlas +gcc45
Sure, you tried with the old version (.37). Pick up the new one that is append in this page as atlas_10.7_gcc45.tgz and try again. Buona fortuna, mi dica se tutto è andato bene.
comment:96 Changed 13 years ago by ke.wu@…
Replying to vince@…: That works for me, MBP 2009 13', Lion and Xcode 4.1.
So, I append a .tgz file containing the Portfile and all the patch files. Since this is highly experimental, it is given as is (not in the form of a diff), without any guarantee, and I strongly advise you to unpack it in a private section of your ports tree (see Macports manual to figure out how to setup one). If you feel adventurous, you can extract it directly in the atlas repertory of the main tree: You can always revert to previous state by wiping out everything and syncing again.
It should work on all MacOS versions with gcc45 in 64-bit mode. All other combinations are not yet tested. I've an old MacMini CoreDuo/10.6 machine on which I should test building it tonight, and a G5 server too.
Support for clang is almost here too, but since clang is stricter than gcc, there are some extra patch files to be generated. Besides, building with clang takes hours, whereas building with gcc is a matter of minutes. I don't know why.
Good luck, and please tell me if you succeed or not.
comment:97 Changed 13 years ago by Veence (Vincent)
Thanks.
I upload a new version of the Portfile and patch files. It should now build on x86_64, i386 and ppc alike. I apologize because at the same time I altered the Portfile to clean up spaces and indentation: The original was a mess. I know I should have done that separately, but the changes are so extensive anyhow that I took on me to do both at the same time, so that we have now a firm ground on which to start afresh.
comment:101 Changed 13 years ago by christoph.jacob@…
Replying to vince@…:
The log suggests that gcc45 might be outdated for the Sandy bridge line of processors: the gas45 assembler seems not to be aware of the brand new AVX instructions atlas already uses. Try to build gcc46 and use the +gcc46 option instead.
I got the same problem as wojtekwn on my 2011 MBP (Core i7).
I also tried +gcc46, which fails with a different problem. The log is attached. If there is anything I can test, please let me know.
Changed 13 years ago by christoph.jacob@…
| Attachment: | main.3.log added |
|---|
main.log for atlas-3.9.46 +gcc46 on Core i7
comment:102 follow-up: 104 Changed 13 years ago by Veence (Vincent)
Sorry, there was a typo in the Portfile I gave. Try this corrected one and tell me.
comment:104 Changed 13 years ago by christoph.jacob@…
Replying to vince@…:
Sorry, there was a typo in the Portfile I gave. Try this corrected one and tell me.
I tried, but I still get the same error in the log file.
comment:105 follow-up: 106 Changed 13 years ago by Veence (Vincent)
Did you upgrade to macports 2.0 ?
comment:106 Changed 13 years ago by christoph.jacob@…
Replying to vince@…:
Did you upgrade to macports 2.0 ?
Yes, I run MacPorts 2.0.99 from SVN.
The configure problem also only shows up with gcc46, with gcc45 the configure step finishes without problems.
comment:107 Changed 13 years ago by Veence (Vincent)
Hmmm… I have not tested that myself because Macports gcc46 is outdated (current version is 4.6.1).
| " and tell me. |
comment:110 follow-up: 117 Changed 13 years ago by Veence (Vincent)
Okay: break! :) I am going to install gcc 4.6.1 from a private modified version of the gcc46 Portfile and try by myself. As soon as I've figured out what's wrong, I'll post a fix.
comment:112 Changed 13 years ago by a.y.harano@…
I use Xcode 4.1 build 4B110; MacPorts 4.0.1 and when I try to build atlas @3.9.37+gcc45-universal, it fails, even after port selfupdate.
comment:114 Changed 13 years ago by Veence (Vincent)
3.9.37 is dead: Forget it. Use the atlas_3_9_46.tgz package I have uploaded instead.
comment:117 Changed 13 years ago by christoph.jacob@…
Ok, I found the problem, and sorry, it was my mistake: gcc46 does not build gfortran by default. So, of course atlas didnt find the fortran compiler. Building it with +gfortran doesnt work for me, but that is another problem.
comment:118 Changed 13 years ago by a.y.harano@…
Thanks, it worked really well. Just for curiosity, is there any estimated date to replace the official atlas's Portfile with yours, since .37 is dead?
comment:119 follow-up: 120 Changed 13 years ago by Veence (Vincent)
@a.y.harano: I let the official maintainer decide. Well, if nothing is done by the end of the week, I will commit it anyway.
@christoph: bad news :( Neither gcc 4.6.0 nor 4.6.1 will compile because of a ld (1) bug I filed a radar about. Gcc and g++ do compile, but gfortran does not. Worse, gfortran is not enabled nor tested on the beta version currently available in Macports.
To wrap up: you’re stuck.
At this point, the only workaround I see is to somehow downgrade the processor to make use of SSE instructions only. It’ll half the performance, but, at least, it should work…
comment:120 Changed 13 years ago by christoph.jacob@…
At this point, the only workaround I see is to somehow downgrade the processor to make use of SSE instructions only. It’ll half the performance, but, at least, it should work…
Thanks a lot anyway. What I did get to work is the "old" 3.9.37 atlas by just including your ar2 workaround into the old Portfile. So I have at least something ...
comment:121 Changed 13 years ago by Veence (Vincent)
@christoph: Could you please tell me what is the output of
sysctl -a | grep hw
Thanks!
Changed 13 years ago by christoph.jacob@…
| Attachment: | sysctl.out added |
|---|
Output of sysctl -a | grep hw on my 2011 MBP
comment:127 Changed 13 years ago by michelbuck@…
Hi Vince,
I was just referred to this page due to duplication. Can I ask how to install the tgz attached to this page? I'm pretty new to all of this. Thanks,
m
comment:128 follow-up: 130 Changed 13 years ago by Veence (Vincent)
@michelbuck: Get the tgz file, unpack it in a local directory of your choice, then follow the instruction of section 4.6 of the manual (http://guide.macports.org/#development.local-repositories) Vincent
comment:129 Changed 13 years ago by Veence (Vincent)
I append a new version of the Portfile that should force atlas to downgrade corei2 CPU to corei1, thereby enabling gcc45 compilation. Please test and report any error.
Changed 13 years ago by Veence (Vincent)
Patched portfile, should work with newest Sandy bridge processors and gcc45
comment:130 Changed 13 years ago by michelbuck@…
Replying to vince@…:
@michelbuck: Get the tgz file, unpack it in a local directory of your choice, then follow the instruction of section 4.6 of the manual (http://guide.macports.org/#development.local-repositories) Vincent
Sorry to bother again, I followed the instructions - I've decompressed the tar file to ~/ports/atlas but when I do cd ~/ports portindex
I get
Total number of ports parsed: 0 Ports successfully parsed: 0 Ports failed: 0 Up-to-date ports skipped: 0
Am I doing something wrong? If there's a beginner's forum I should rather post these questions to please let me know. m
comment:132 follow-up: 135 Changed 13 years ago by fleason (Fred Leason)
@michelbuck
Here is a simpler list of instructions than what I saw on section 4.6. of the manual.
$ locate atlas/files /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-ATL_AVgcc-fix.diff /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-SpewMakeInc.c.diff /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-archinfo_freebsd.c.diff /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-archinfo_freebsd_c.diff /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-build-Make.top.diff /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-emit_mm_c.diff
Will show you the directory where the files need to go. I have a plain vanilla implementation, so in my case /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/
After you open and expand the tgz you will have a directory called files. It will have the ar2 program and a bunch of patch*.diff files. In your case I'm guessing
$ cd ~/ports/files $ sudo cp * /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files
This will copy the work-around into your normal MacPorts workspace.
Then cd up a level and copy the portfile.
$ cd .. $ cp Portfile /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/
Then you can go ahead and try an upgrade
$ sudo port upgrade atlas
If you make a mistake, you can always start from scratch as follows:
$ sudo port clean --all atlas $ sudo port install atlas
Let it blow up and then repeat the patching process above.
comment:133 Changed 13 years ago by christoph.jacob@…
Hi Vince, if I change hw.optional_avx1_0 to hw.optional.avx1_0, then it works.
comment:134 Changed 13 years ago by Veence (Vincent)
Darn! I know I had to do a typo! :) Thanks so much for testing. I’ll correct the typo and repost an updated (& hopefully final) tar file.
Changed 13 years ago by Veence (Vincent)
| Attachment: | atlas_3_9_46.tgz added |
|---|
Updated atlas subdirectory containing Portfile and patches. Should work on any machine with gcc44 or 45.
comment:135 Changed 13 years ago by michelbuck@…
@fleason
Hi, Thanks for the help. So I installed atlas, getting he standard error message
Error: Target org.macports.build returned: shell command failed (see log for details) Log for atlas is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/main.log Error: Status 1 encountered during processing. To report a bug, see <http://guide.macports.org/#project.tickets>
and I then copied the files into the relevant folders. However, upon updating I get
0 Sat Aug 06 20:03:29 michelbuck ~/atlas/atlas$ sudo port upgrade atlas Error: atlas is not installed To report a bug, see <http://guide.macports.org/#project.tickets>
Is there something obvious going wrong? m
comment:136 Changed 13 years ago by fleason (Fred Leason)
@michaelbuck
Try
$ sudo port install atlas
comment:137 Changed 13 years ago by matthew.g.russell@…
I have the same problem.
I followed the instructions from michelbuck using the files from atlas_3_9_46.tgz, and I never get it into a state where I can upgrade it.
So, if I clean, install, replace the Portfile and files/*, then try to upgrade ==> I can't. So I run the install .. the install will see that the Portfile has been replaced and completely re-run and bail a couple hours later with the same error.
If I instead only replace the files in files/* at that point, the install will jump to the error right away.
Just in case, I ran a selfupdate and selfupgrade today.. Didn't change anything.
Am I doing something wrong?
comment:138 follow-up: 139 Changed 13 years ago by fleason (Fred Leason)
I wish I saved the steps that worked for me. I cannot remember the sequence of clean, clean --all, install, upgrade that made it work for me. I don't understand the MacPorts environment to help. But it did work.
The only other thing I did different is I used the patched Portfile that is downloaded separate from the tgz. And I went ahead and did the gcc46 install with gfortran. Although my script indicated that it used gcc44. I will attach my log.
comment:139 Changed 13 years ago by fleason (Fred Leason)
comment:140 Changed 13 years ago by Veence (Vincent)
I am going to commit my version of atlas. It should avoid all this hocus-pocus.
comment:141 follow-up: 144 Changed 13 years ago by Veence (Vincent)
Committed in r82161. Please update and try again.
comment:142 Changed 13 years ago by cdeil (Christoph Deil)
When I update the atlas directory using svn I get the following error:
svn: In directory '/opt/local/var/macports/sources/svn.macports.org/trunk/dports/math/atlas/files' svn: Can't open file '/opt/local/var/macports/sources/svn.macports.org/trunk/dports/math/atlas/files/.svn/tmp/text-base/patch-Make_ttune.diff.svn-base': No such file or directory shell command "/opt/local/bin/svn update --non-interactive /opt/local/var/macports/sources/svn.macports.org/trunk/dports" returned error 1
I also tried removing the atlas directory and checking out a fresh version. Same error.
Any advice?
comment:144 follow-ups: 146 147 Changed 13 years ago by geoffgrimwood
Replying to vince@…:
Committed in r82161. Please update and try again.
Successfully installed in less than 10 minutes on 2011 MBP running Lion and XCode 4.1 (latest i7 processor).
Great!
Thank you very much Vince for your efforts over the last couple of weeks.
Fingers now crossed that py27-matplotlib will build and install.
comment:146 follow-up: 148 Changed 13 years ago by Veence (Vincent)
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Replying to geoff@…:
Replying to vince@…:
Committed in r82161. Please update and try again.
Successfully installed in less than 10 minutes on 2011 MBP running Lion and XCode 4.1 (latest i7 processor).
Great!
Thank you very much Vince for your efforts over the last couple of weeks.
My pleasure. Glad to know it works on the latest Core i7 with gcc45. But please note that due to the inability to compile gcc46, you get only half the performance you could draw out of your brand new chip.
I'll now report all the patches upstream, so that we may have a cleaner future version.
Cheers.
PS: I close the ticket, since it seems to work for everyone.
comment:147 Changed 13 years ago by michelle.lynn.gill@…
Replying to geoff@…:
Thank you very much Vince for your efforts over the last couple of weeks.
Yes, thank you, Vincent.
Fingers now crossed that py27-matplotlib will build and install.
Worked fine for me on Lion.
comment:148 Changed 13 years ago by fleason (Fred Leason)
Replying to vince@…:
Replying to geoff@…:
My pleasure. Glad to know it works on the latest Core i7 with gcc45. But please note that due to the inability to compile gcc46, you get only half the performance you could draw out of your brand new chip.
I got gcc46 to build on a first generation i7. Is it only an issue with the second generation?
comment:149 Changed 13 years ago by Veence (Vincent)
It is an issue with the ld (1) utility (the linker) on 10.7, it does not depend on the CPU. GCC45 does not support the additional AVX instruction set introduced with Sandy bridge, therefore Atlas assembler routines that use it fails. As a workaround, I hardwired the detection routine to consider any CoreiX processor to be first generation. That way, Atlas makes use of instructions known to gcc45. But it is a waste of power. If you have succeeded in building gcc46, you can easily patch the Portfile to make use of it.
log