Trouble with MacPorts installation of graph tool

[t.m.isele@…]

Hi, I've installed graph-tool via Mac Ports. 2 days ago, I ran

$ sudo port selfupdate
$ sudo port upgrade outdated

Graph-tool was compiled anew (which took around 30 hours(!)) Now, when I want to include graph-tool in python I get the following error:

Python 2.7.6 (default, Dec 23 2013, 22:03:28) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import graph_tool
dyld: lazy symbol binding failed: Symbol not found: __ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFNSt3__14pairIPvS2_EES5_E
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/graph_tool/libgraph_tool_core.so
  Expected in: flat namespace

dyld: Symbol not found: __ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFNSt3__14pairIPvS2_EES5_E
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/graph_tool/libgraph_tool_core.so
  Expected in: flat namespace

Trace/BPT trap: 5

I tested the version and variant of the boost package with "port installed | grep boost": Result:

  boost @1.49.0_0+python27
  boost @1.53.0_1+no_single+no_static+python27
  boost @1.53.0_2+no_single+no_static+python27
  boost @1.54.0_0+no_single+no_static+python27
  boost @1.55.0_1+no_single+no_static+python27
  boost @1.55.0_2+no_single+no_static+python27 (active)

For graph-tool itself, the result of "port installed | grep graph-tool" is:

  py27-graph-tool @2.2.25_0
  py27-graph-tool @2.2.29_0
  py27-graph-tool @2.2.29.1_0 (active)

Does anybody have an idea?

Thanks,

Thomas

[mamoll (Mark Moll)]

Can you paste the output of this command?:

otool -L /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/graph_tool/libgraph_tool_core.so

[t.m.isele@…]

Sure, it reads

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/graph_tool/libgraph_tool_core.so:
	/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.0)
	/opt/local/lib/libboost_iostreams-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libboost_regex-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libCGAL.10.dylib (compatibility version 10.0.0, current version 10.0.2)
	/opt/local/lib/libexpat.1.dylib (compatibility version 8.0.0, current version 8.0.0)
	/opt/local/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.6)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 56.0.0)

[mamoll (Mark Moll)]

I am guessing you are not using Mavericks (10.9), right? I suspect that perhaps py-graph-tool needs some C++11 features that are available in libc++, but not libstdc++ (see ​http://stackoverflow.com/questions/18389799/boost-python-undefined-symbols-error-even-though-the-lib-files-are-linked).

[t.m.isele@…]

Hi, I'm not using Mavericks (yet). My System version is 10.8.5.

Are you suggesting, that a

port install libcxx

and then rebuilding boost and graph tool could fix the issue?

I'm a little hesitant, because last time I rebuilt graph-tool, it took about 30 hours on my MacBookPro. (Is that normal?)

Thanks, Thomas

[mamoll (Mark Moll)]

No, I am not suggesting that. I am just speculating why it is broken and why it works for me (on Mavericks). You can't mix and match c++ standard libraries easily (Boost would still be linked against libstdc++, for instance). But I am not 100% convinced this is the problem. Is everything else "standard" on your MacPorts install (no default variants, no override of default compiler, etc.)?

[t.m.isele@…]

As far as I know, everything should be standard (what do you mean exactly by this?)

This is the current compiler

$ ll `which gcc`
lrwxr-xr-x  1 root  admin  25 24 Jun  2012 /opt/local/bin/gcc -> /opt/local/bin/gcc-mp-4.5

How would I check the rest?

[mamoll (Mark Moll)]

Wait, what? If type "sudo port -dv install <portname>" you should see clang as the compiler, not gcc. What I meant is that you haven't made changes in the files under /opt/local/etc/macports.

[t.m.isele@…]

I have not fiddled with those files ever. Here's the result of the command ( dry run )..

$ sudo port -dvy install boost

DEBUG: Copying /Users/Thomas/Library/Preferences/com.apple.dt.Xcode.plist to /opt/local/var/macports/home/Library/Preferences
DEBUG: Changing to port directory: /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/devel/boost
DEBUG: OS darwin/12.5.0 (Mac OS X 10.8) arch i386
DEBUG: compiler clang 425.0.28 not blacklisted because it doesn't match {clang < 421}
DEBUG: Reading variant descriptions from /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/_resources/port1.0/variant_descriptions.conf
DEBUG: universal variant already exists, so not adding the default one
DEBUG: Executing variant python27 provides python27
DEBUG: Executing variant no_static provides no_static
DEBUG: Executing variant no_single provides no_single
DEBUG: Running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Finished running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Finished running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: changing euid/egid - current euid: 0 - current egid: 0
DEBUG: egid changed to: 501
DEBUG: euid changed to: 503
DEBUG: Starting logging for boost
--->  Computing dependencies for boost
DEBUG: boost has no conflicts
DEBUG: Searching for dependency: zlib
DEBUG: Found Dependency: receipt exists for zlib
DEBUG: Searching for dependency: expat
DEBUG: Found Dependency: receipt exists for expat
DEBUG: Searching for dependency: bzip2
DEBUG: Found Dependency: receipt exists for bzip2
DEBUG: Searching for dependency: libiconv
DEBUG: Found Dependency: receipt exists for libiconv
DEBUG: Searching for dependency: icu
DEBUG: Found Dependency: receipt exists for icu
DEBUG: Searching for dependency: python27
DEBUG: Found Dependency: receipt exists for python27
DEBUG: Skipping activate (boost @1.55.0_2+no_single+no_static+python27) since this port is already active
For boost: skipping org.macports.main (dry run)
    .. and skipping org.macports.clean

[mamoll (Mark Moll)]

This doesn't actually show the compiler being invoked.

[t.m.isele@…]

It is clang. I just found out by installing some port. I remember that graph-tool was also using clang.

[absima@…]

I also have the same problem which I reported a few days ago and have the issue redirected to the current ticket. My system is 10.7.5. and I could not not upgrade it any further; so nor could I install Mavericks.

Does it mean that there is no way of getting my graph-tool work again since it has presumably used dependence on mavericks? If so, is it possible to install the older versions of graph-tool via port (preferred) or some other way (from repos)? Thanks for any opinion in advance

[mamoll (Mark Moll)]

I don't know what the last version is that worked with 10.7. See ​https://trac.macports.org/wiki/howto/InstallingOlderPort for instructions on installing an older version of a port.

[t.m.isele@…]

I got a little further on this issue by now. I met Tiago, the developer of graph_tool on a conference and asked him. This error is most likely due to the following library not being linked against the current python Version (which is 2.7.6. in my case):

$ otool -L /opt/local/lib/libboost_python-mt.dylib
/opt/local/lib/libboost_python-mt.dylib:
	/opt/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.0)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 56.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

otool reveals that the linking is done against python 2.7.0 here instead of 2.7.6

I tried

sudo port upgrade --force  boost +no-single +no-static +python27

but this didn't solve the linkage problem.

Does anybody have an idea, what to do that boost +python27 links against the current version of python?

Or other thoughts and ideas how to overcome this problem?

Thanks, Thomas

[mamoll (Mark Moll)]

This is not the problem. I also have python 2.7.6 installed. Otool reports that it is version 2.7.0, but it really isn't. What happens in rare cases is that you link against one library at compile, but a different version is found at runtime. This will lead to problems, but that is not the case here.

[t.m.isele@…]

By using

nm -a /opt/local/lib/libboost_python-mt.dylib

I found out, that the symbol

__ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFNSt3__14pairIPvS2_EES5_E

is indeed not contained there. Nevertheless, there is a very similar symbol:

__ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFSt4pairIPvS2_ES4_E

if that does mean anything.

I also checked older versions of boost, namely

  boost @1.49.0_0+python27
  boost @1.53.0_1+no_single+no_static+python27
  boost @1.53.0_2+no_single+no_static+python27
  boost @1.54.0_0+no_single+no_static+python27
  boost @1.55.0_1+no_single+no_static+python27
  boost @1.55.0_2+no_single+no_static+python27

for the symbol in the mentioned file, by doing

port activate boost @xxx +yyy 

and then calling nm as above with the same result as above.

Also I updated my Xcode in the meantime and ran

port upgrade --force boost +no_static +no_single +python27
port upgrade --force py27-graphtool

but it still gives the same error. Any help or suggestions out there? Thanks, Thomas

[t.m.isele@…]

Hi there, in the meantime I deinstalled all inactive ports by

port uninstall inactive

Then I deinstalled python and its dependents by

port uninstall --follow_dependents python27

Then I installed graph tool anew by

port install py27-graph-tool

This operation has just finished. Still I get the same error.

Could it be that this bug is really some issue with these ports in their latest version, together with my operating system (OSX 10.8.5.)? Graph tool used to work a couple of months ago until I upgraded.

Could anybody with a working graph tool maybe have a look with

nm -a /opt/local/lib/libboost_python-mt.dylib

whether the symbol

__ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFNSt3__14pairIPvS2_EES5_E

is present or not? This would point the direction in which to investigate further (symbol there -> more likely an issue with graph-tool, symbol not there -> more likely an issue with boost).

The next thing I could imagine would only be to deinstall all ports completely, deinstall macports itself. And then reinstall everything again..

Any further suggestions?

Thanks,

Thomas

[neverpanic (Clemens Lang)]

Replying to t.m.isele@…:

I found out, that the symbol

__ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFNSt3__14pairIPvS2_EES5_E

is indeed not contained there. Nevertheless, there is a very similar symbol:

__ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFSt4pairIPvS2_ES4_E

if that does mean anything.

That actually gives me enough information to pinpoint this down to a C++ standard library issue. Compare:

$> # the symbol that's missing
$> echo '__ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFNSt3__14pairIPvS2_EES5_E' | c++filt
boost::python::objects::register_dynamic_id_aux(boost::python::type_info, std::__1::pair<void*, boost::python::type_info> (*)(void*))
$> # with the symbol that's present
$> echo '__ZN5boost6python7objects23register_dynamic_id_auxENS0_9type_infoEPFSt4pairIPvS2_ES4_E' | c++filt
boost::python::objects::register_dynamic_id_aux(boost::python::type_info, std::pair<void*, boost::python::type_info> (*)(void*))

The ::__1 you see there is part of libc++, clang's new C++ standard library. It seems graph tool deliberately chooses to install against libc++, but your version of boost is compiled against (the older) libstdc++, which doesn't support C++11. So either, if graph tool doesn't need C++11, it shouldn't use libc++, or, if it needs C++11, it cannot be built on systems below Mavericks (except with a lot of effort, see ​http://apple.stackexchange.com/questions/118550/define-local-keyword-globally-in-a-macports-config/122997#122997).

Note that a symbol mismatch like this is actually one of the nicer problems you can encounter in this situation. I could also just have crashed randomly.

I don't understand why libgraph_tool_core.so links against /usr/lib/libstdc++.6.dylib, though.

[count0 (Tiago de Paula Peixoto)]

Indeed this looks like a libstdc++/libc++ mismatch problem. Graph-tool needs C++11 in recent versions. Shouldn't it be possible to compile boost with libc++? If not, that simply means that graph-tool can't be built with anything older than Mavericks... Maybe a warning could be incorporated in the Portfile?

[neverpanic (Clemens Lang)]

Replying to tiago@…:

Shouldn't it be possible to compile boost with libc++?

It is, but you'd have to rebuild the whole MacPorts world against libc++. That's what the link I pasted was about. That's entirely unsupported and if it breaks you get to keep the pieces, though.

Replying to tiago@…:

If not, that simply means that graph-tool can't be built with anything older than Mavericks... Maybe a warning could be incorporated in the Portfile?

Or, the Portfile could automatically install the latest version that did work on < Mavericks.

[mamoll (Mark Moll)]

Can you try the attached Portfile (i.e., replace py-graph-tool/Portfile with this one)? You'd have to uninstall py-graph-tool first. This Portfile will install version 2.2.26 on OS X 10.8 and below if clang is used as the compiler.

[t.m.isele@…]

Hi guys,

it's too late now to try this now. This morning I capitulated and installed Mavericks on my machine. A couple of hours later, MacPorts was through. And indeed, now it works. I'm quite relieved, because I needed graph-tool for my PhD Thesis.. Thanks for all your help!

Thomas

[mamoll (Mark Moll)]

Committed in r119268

[absima@…]

Then my case ​https://trac.macports.org/ticket/43054 which has been closed as a duplicate could not be solved as Thomas's. My early 2008 macbook can no longer be upgraded to Mavericks. Lion OS X is the furthest it could be upgraded, not even mountain lion. But before the current version of graph-tool for e.g. 2.22 used to work just fine. Any help?

[neverpanic (Clemens Lang)]

Are you saying that the fix committed in r119268 doesn't work for you and you cannot build graph-tool 2.2.26 (which should be what the Portfile installs on your system)? Are you seeing the hang you described in #43054 or the missing symbols as mentioned in this ticket?