py-scipy @0.13.3_1: builds agains ATLAS (variant +atlas) Seg. fault during testing.

[petrrr]

I built py-scipy against ATLAS. This version of scipy crashes during testing with a Segmentation fault. The details below are for py27-scipy +atlas, but it was observed before (version @0.13.3) with other python subports as well.

There seem to be some real failures before the crash as well.

In [2]: import scipy

In [3]: scipy.test()
Running unit tests for scipy
NumPy version 1.8.0
NumPy is installed in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy
SciPy version 0.13.3
SciPy is installed in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy
Python version 2.7.6 (default, Nov 22 2013, 13:39:24) [GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))]
nose version 1.3.1
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/utils.py:134: DeprecationWarning: `scipy.lib.blas` is deprecated, use `scipy.linalg.blas` instead!
  warnings.warn(depdoc, DeprecationWarning)
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/utils.py:134: DeprecationWarning: `scipy.lib.lapack` is deprecated, use `scipy.linalg.lapack` instead!
  warnings.warn(depdoc, DeprecationWarning)
..............................................................................................................................................................................................................................K..................................................................................................................K.......................................................................................K..K...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F..FF......................................................................................................................................SSSSSS......SSSSSS......SSSS..............................FFF.....................................................0-th dimension must be fixed to 3 but got 15
..0-th dimension must be fixed to 3 but got 5
...F.Segmentation fault

ATLAS is installed as:

atlas @3.10.1_5+mpclang33 (active)

I also attach a crash report in the hope that it might help.

Not sure if this could be due to compiler mismatch. I CC Vince as atlas maintainer as well.

[petrrr]

crash report

[seanfarley (Sean Farley)]

Thanks for the report. I see the failure for me is from test_nonlin.TestJacobianDotSolve.test_broyden1 (ran with scipy.test(verbose=10)). Searching around on the internet makes it seem that this is a known problem with ATLAS:

​http://mail.scipy.org/pipermail/scipy-dev/2011-September/016582.html

and that the workaround is just not to use ATLAS.

[seanfarley (Sean Farley)]

In other words, I am stuck.

[Veence (Vincent)]

If you deem it useful to you, I can ask Atlas developer (Clint) to know if this is fixed in the latest unstable version.

[seanfarley (Sean Farley)]

Replying to vince@…:

If you deem it useful to you, I can ask Atlas developer (Clint) to know if this is fixed in the latest unstable version.

Sure, but I suspect this is something in the fortran compiler (maybe misaligning the arrays?). The bug report from scipy is from 2011 and they basically just say, "If this is a problem, then use another lapack package." I'll try to point them to this bug report and see if that gains any traction.

[seanfarley (Sean Farley)]

Replying to sean@…:

Replying to vince@…:

If you deem it useful to you, I can ask Atlas developer (Clint) to know if this is fixed in the latest unstable version.

Sure, but I suspect this is something in the fortran compiler (maybe misaligning the arrays?). The bug report from scipy is from 2011 and they basically just say, "If this is a problem, then use another lapack package." I'll try to point them to this bug report and see if that gains any traction.

I found the issue on their github tracker here:

​https://github.com/scipy/scipy/issues/3193

[seanfarley (Sean Farley)]

Looks like this issue gained no traction on the scipy front. I still get the error but don't have the resources to dig into the scipy code. Marking this as 'wontfix' since it seems to be a bug in scipy.

[petrrr]

I guess it makes perfectly sense not to try to fix bug in upstream software.

However, I wonder if we shouldn't disable variants if these are broken? Or do we judge that this is one too specific case we do not want to bother about?

[seanfarley (Sean Farley)]

Replying to petr@…:

I guess it makes perfectly sense not to try to fix bug in upstream software.

However, I wonder if we shouldn't disable variants if these are broken? Or do we judge that this is one too specific case we do not want to bother about?

That is a great question. I am debating the same thing with the cgal +demo variant (which is still broken for me).

[seanfarley (Sean Farley)]

Actually, looking into the compiler situation with scipy and numpy, I think I do have a fix for this.

[seanfarley (Sean Farley)]

It seemed that numpy and scipy made incorrect (or outdated) assumptions about atlas (before the clang changes). I've fixed those in r130955. Some of the tests don't pass because numpy being too new.

[Veence (Vincent)]

Sean, thanks for fixing this. I seem to have written a lot of things on my MacPorts backlog lately. I’ll try to catch up.