vtk5 -r90101 expecting a vtk library that isn't there. /opt/local/lib/vtk-5.8/libvtkftgl.5.8.dylib

[shinko.cheng@…]

Hello,

I tried upgrading vtk5 -r90101 to the latest 5.8 today via ports and found the following error:

 station:~ sycheng$ sudo port install vtk5
 Password:
 --->  Computing dependencies for vtk5
 --->  Fetching archive for vtk5
 --->  Attempting to fetch vtk5-5.8.0_0+python27+tcltk+x11.darwin_10.x86_64.tbz2  from http://packages.macports.org/vtk5
 --->  Fetching vtk5
 --->  Verifying checksum(s) for vtk5
 --->  Extracting vtk5
 --->  Configuring vtk5
 --->  Building vtk5
 Error: Target org.macports.build returned: shell command failed (see log for details)
 Log for vtk5 is at:  /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_rel ease_ports_graphics_vtk5/vtk5/main.log
 Error: Status 1 encountered during processing.
 To report a bug, see <http://guide.macports.org/#project.tickets>

Then I navigated to the log file and found the error occured when executing this command:

 :info:build shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_graphics_vtk5/vtk5/work/VTK" && /usr/bin/make -j4 -w all " returned error 2

So I went to the directory and typed 'sudo make' to found this to be the problem:

 Linking CXX executable ../../bin/vtkpython
 cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_graphics_vtk5/vtk5/work/VTK/Wrapping/Python && /opt/local/bin/cmake -E cmake_link_script CMakeFiles/vtkpython.dir/link.txt --verbose=1
 /usr/bin/g++-4.2   -pipe -O2 -arch x86_64   -Wno-deprecated -O3 -DNDEBUG -arch x86_64 -isysroot / -mmacosx-version-min=10.6 -Wl,-search_paths_first -Wl,-headerpad_max_install_names  -L/opt/local/lib -arch x86_64  -flat_namespace -undefined suppress -u _PyMac_Error CMakeFiles/vtkpython.dir/vtkPythonAppInit.cxx.o  -o ../../bin/vtkpython  /opt/local/lib/libtk.dylib /opt/local/lib/libtcl.dylib -lm -lpthread /opt/local/lib/libpython2.7.dylib ../../bin/libvtksys.5.8.0.dylib ../../bin/libvtkCommon.5.8.0.dylib ../../bin/libvtkFiltering.5.8.0.dylib ../../bin/libvtkIO.5.8.0.dylib ../../bin/libvtkGraphics.5.8.0.dylib ../../bin/libvtkImaging.5.8.0.dylib /usr/lib/libutil.dylib ../../bin/libvtkRendering.5.8.0.dylib ../../bin/libvtkVolumeRendering.5.8.0.dylib ../../bin/libvtkHybrid.5.8.0.dylib ../../bin/libvtkWidgets.5.8.0.dylib ../../bin/libvtkCharts.5.8.0.dylib ../../bin/libvtkGeovis.5.8.0.dylib ../../bin/libvtkInfovis.5.8.0.dylib ../../bin/libvtkViews.5.8.0.dylib ../../bin/libvtkInfovis.5.8.0.dylib ../../bin/libvtkWidgets.5.8.0.dylib ../../bin/libvtkVolumeRendering.5.8.0.dylib ../../bin/libvtkHybrid.5.8.0.dylib ../../bin/libvtkRendering.5.8.0.dylib ../../bin/libvtkIO.5.8.0.dylib ../../bin/libvtkGraphics.5.8.0.dylib ../../bin/libvtkImaging.5.8.0.dylib ../../bin/libvtkFiltering.5.8.0.dylib ../../bin/libvtkCommon.5.8.0.dylib ../../bin/libvtksys.5.8.0.dylib -lm -lm -lpthread /opt/local/lib/libpython2.7.dylib /usr/lib/libutil.dylib 
 ld: warning: duplicate dylib /usr/lib/libutil.dylib
 ld: file not found: /opt/local/lib/vtk-5.8/libvtkftgl.5.8.dylib
 collect2: ld returned 1 exit status
 make[2]: *** [bin/vtkpython] Error 1
 make[1]: *** [Wrapping/Python/CMakeFiles/vtkpython.dir/all] Error 2
 make: *** [all] Error 2

Any suggestions? Thanks.

Shinko

[shinko.cheng@…]

Cc Me!

[shinko.cheng@…]

What's bizarre is that libvtkftgl.5.8.dylib i imagine is a library that vtk5 will install, but it's somehow expecting the library to be in /opt/local/lib even before the library is even finished installing. What exactly is ftgl?

[shinko.cheng@…]

port install log

[shinko.cheng@…]

I tried port installing on a different mac with Mac OS X Lion instead of Snow Leopard, and the same problem exists.

It looks like an issue with the build for vtkpython. It's expecting some library that depends on libvtkftg and it's not finding it. I looked and the vtkftgl liibrary is there, under bin, but apparently it's not finding it. The CMakeLists.txt file has so many variables, too many for me to follow and too long between runs for me to find out where the culprit is. More precision that what I have is needed to locate the error in the configuration i think.

[shinko.cheng@…]

I found another thing tonight. I downloaded vtk-5.6 Portfile and attempted to go back a version. The program fails at a similar location in vtkpython, complaining about this time libvtkxml2 cannot be found. So i Added -L../../bin to the g++ command in Wrappers/Python when building vtkPython, the build succeeds. I suspect if I did the same thing with vtk-5.8 build, i can succeeded as well. however, this means I need to port install vtk5 until it failed, go into Wrapeprs/Python, run the command, and port install again to finish. Seems like a workaround, but still just a workaround.

 /usr/bin/g++-4.2   -pipe -O2 -arch x86_64   -Wno-deprecated -O3 -DNDEBUG -arch x86_64 -isysroot / -mmacosx-version-min=10.6 -Wl,-search_paths_first -Wl,-headerpad_max_install_names  -L/opt/local/lib -arch x86_64  -flat_namespace -undefined suppress -u _PyMac_Error CMakeFiles/vtkpython.dir/vtkPythonAppInit.cxx.o  -o ../../bin/vtkpython  /opt/local/lib/libtk.dylib /opt/local/lib/libtcl.dylib -lm -lpthread /opt/local/lib/libpython2.7.dylib ../../bin/libvtksys.5.8.0.dylib ../../bin/libvtkCommon.5.8.0.dylib ../../bin/libvtkFiltering.5.8.0.dylib ../../bin/libvtkIO.5.8.0.dylib ../../bin/libvtkGraphics.5.8.0.dylib ../../bin/libvtkImaging.5.8.0.dylib /usr/lib/libutil.dylib ../../bin/libvtkRendering.5.8.0.dylib ../../bin/libvtkVolumeRendering.5.8.0.dylib ../../bin/libvtkHybrid.5.8.0.dylib ../../bin/libvtkWidgets.5.8.0.dylib ../../bin/libvtkCharts.5.8.0.dylib ../../bin/libvtkGeovis.5.8.0.dylib ../../bin/libvtkInfovis.5.8.0.dylib ../../bin/libvtkViews.5.8.0.dylib ../../bin/libvtkInfovis.5.8.0.dylib ../../bin/libvtkWidgets.5.8.0.dylib ../../bin/libvtkVolumeRendering.5.8.0.dylib ../../bin/libvtkHybrid.5.8.0.dylib ../../bin/libvtkRendering.5.8.0.dylib ../../bin/libvtkIO.5.8.0.dylib ../../bin/libvtkGraphics.5.8.0.dylib ../../bin/libvtkImaging.5.8.0.dylib ../../bin/libvtkFiltering.5.8.0.dylib ../../bin/libvtkCommon.5.8.0.dylib ../../bin/libvtksys.5.8.0.dylib -lm -lm -lpthread /opt/local/lib/libpython2.7.dylib /usr/lib/libutil.dylib  \ '''-L../../bin'''

This finishes the build for vtkpython. I then go back to run sudo port install (where the vtk5 Portfile resided) and the install succeeded.

[shinko.cheng@…]

Yep confirmed. vtkpython can't find the built vtk libraries in vtk-5.8. Once i gave it a -L../../bin, and go back to building with port, all succeeded.

[shinko.cheng@…]

Confirmed on a second mac. Providing the following command to get past the error with building vtkpython will allow me to execute sudo port install vtk5 successfully immediately following it.

cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Views && /usr/bin/g++-4.2   -DvtkViewsTCL_EXPORTS -DVTK_IN_VTK -pipe -O2 -arch x86_64   -Wno-deprecated -O3 -DNDEBUG -arch x86_64 -isysroot / -mmacosx-version-min=10.6 -fPIC -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Common -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/VolumeRendering -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Rendering -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Charts -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/vtkalglib -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Wrapping/Python -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Infovis -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Geovis -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Views -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Hybrid -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Widgets -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Rendering/Testing/Cxx -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/IO -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Imaging -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Graphics -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/GenericFiltering -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Filtering -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Common/Testing/Cxx -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/vtklibproj4 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/DICOMParser -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/vtkfreetype/include -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/MaterialLibrary -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/verdict -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/vtkhdf5 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/vtkhdf5/src -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/utf8/source -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/ftgl/src -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/ftgl -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Utilities/TclTk/internals/tk8.5 -I/opt/local/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7    -o CMakeFiles/vtkViewsTCL.dir/vtkDataRepresentationTcl.cxx.o -c /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/work/VTK/Views/vtkDataRepresentationTcl.cxx
-L../../bin

Looks like i'll get to keep my day job for a little longer.

[shinko.cheng@…]

Correction: It's actually this command:

sudo /usr/bin/g++-4.2   -pipe -O2 -arch x86_64   -Wno-deprecated -O3 -DNDEBUG -arch x86_64 -isysroot / -mmacosx-version-min=10.6 -Wl,-search_paths_first -Wl,-headerpad_max_install_names  -L/opt/local/lib -arch x86_64  -flat_namespace -undefined suppress -u _PyMac_Error CMakeFiles/vtkpython.dir/vtkPythonAppInit.cxx.o  -o ../../bin/vtkpython  /opt/local/lib/libtk.dylib /opt/local/lib/libtcl.dylib -lm -lpthread /opt/local/lib/libpython2.7.dylib ../../bin/libvtksys.5.8.0.dylib ../../bin/libvtkCommon.5.8.0.dylib ../../bin/libvtkFiltering.5.8.0.dylib ../../bin/libvtkIO.5.8.0.dylib ../../bin/libvtkGraphics.5.8.0.dylib ../../bin/libvtkImaging.5.8.0.dylib /usr/lib/libutil.dylib ../../bin/libvtkRendering.5.8.0.dylib ../../bin/libvtkVolumeRendering.5.8.0.dylib ../../bin/libvtkHybrid.5.8.0.dylib ../../bin/libvtkWidgets.5.8.0.dylib ../../bin/libvtkCharts.5.8.0.dylib ../../bin/libvtkGeovis.5.8.0.dylib ../../bin/libvtkInfovis.5.8.0.dylib ../../bin/libvtkViews.5.8.0.dylib ../../bin/libvtkInfovis.5.8.0.dylib ../../bin/libvtkWidgets.5.8.0.dylib ../../bin/libvtkVolumeRendering.5.8.0.dylib ../../bin/libvtkHybrid.5.8.0.dylib ../../bin/libvtkRendering.5.8.0.dylib ../../bin/libvtkIO.5.8.0.dylib ../../bin/libvtkGraphics.5.8.0.dylib ../../bin/libvtkImaging.5.8.0.dylib ../../bin/libvtkFiltering.5.8.0.dylib ../../bin/libvtkCommon.5.8.0.dylib ../../bin/libvtksys.5.8.0.dylib -lm -lm -lpthread /opt/local/lib/libpython2.7.dylib /usr/lib/libutil.dylib -L../../bin
ld: warning: duplicate dylib /usr/lib/libutil.dylib \
-L../../bin

Recap:

I had to run the following commands:

> sudo port install vtk5
> sudo /usr/bin/g++-4.2   -pipe -O2 -arch x86_64   -Wno-deprecated -O3 -DNDEBUG -arch x86_64 -isysroot / -mmacosx-version-min=10.6 -Wl,-search_paths_first -Wl,-headerpad_max_install_names  -L/opt/local/lib -arch x86_64  -flat_namespace -undefined suppress -u _PyMac_Error CMakeFiles/vtkpython.dir/vtkPythonAppInit.cxx.o  -o ../../bin/vtkpython  /opt/local/lib/libtk.dylib /opt/local/lib/libtcl.dylib -lm -lpthread /opt/local/lib/libpython2.7.dylib ../../bin/libvtksys.5.8.0.dylib ../../bin/libvtkCommon.5.8.0.dylib ../../bin/libvtkFiltering.5.8.0.dylib ../../bin/libvtkIO.5.8.0.dylib ../../bin/libvtkGraphics.5.8.0.dylib ../../bin/libvtkImaging.5.8.0.dylib /usr/lib/libutil.dylib ../../bin/libvtkRendering.5.8.0.dylib ../../bin/libvtkVolumeRendering.5.8.0.dylib ../../bin/libvtkHybrid.5.8.0.dylib ../../bin/libvtkWidgets.5.8.0.dylib ../../bin/libvtkCharts.5.8.0.dylib ../../bin/libvtkGeovis.5.8.0.dylib ../../bin/libvtkInfovis.5.8.0.dylib ../../bin/libvtkViews.5.8.0.dylib ../../bin/libvtkInfovis.5.8.0.dylib ../../bin/libvtkWidgets.5.8.0.dylib ../../bin/libvtkVolumeRendering.5.8.0.dylib ../../bin/libvtkHybrid.5.8.0.dylib ../../bin/libvtkRendering.5.8.0.dylib ../../bin/libvtkIO.5.8.0.dylib ../../bin/libvtkGraphics.5.8.0.dylib ../../bin/libvtkImaging.5.8.0.dylib ../../bin/libvtkFiltering.5.8.0.dylib ../../bin/libvtkCommon.5.8.0.dylib ../../bin/libvtksys.5.8.0.dylib -lm -lm -lpthread /opt/local/lib/libpython2.7.dylib /usr/lib/libutil.dylib -L../../bin
ld: warning: duplicate dylib /usr/lib/libutil.dylib -L../../bin
> sudo port install vtk5

The way I found out this command is by looking at the log file, probably located here:

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_vtk5/vtk5/main.log

And looked for the error pertaining to ld: Couldn't find /opt/local/lib/libvtkftgl.dylib or libvtkxml2.dylib.

There's a command that cd's into the Wrapper/Python director and builds vtkpython. That's the command I execute with the adder "-L../../bin"

[adfernandes (Andrew Fernandes)]

Hmm... googling around (and seeing #33465) this has the bad smell of a parallel-build bug.

Could you please try building with

sudo port install vtk5 build.jobs=1

in the Portfile?

Thanks for doing the background digging on this, too, by the way!

[shinko.cheng@…]

Of course. I'm hoping in the end, a new Portfile that works gets committed :)

I gave the build.jobs=1 a try, and it still fails to find a library for vtkpython. But this time it fails to find

ld: file not found: /opt/local/lib/vtk-5.8/libvtkDICOMParser.5.8.dylib for architecture x86_64

Now the DICOMParser library has shown up in my web searches. I'm not sure if they're relevant though.

[tommccullough-tenica]

I'm fairly sure this is redundant with Ticket #33465

[shinko.cheng@…]

It does look similar. I prefer my workaround involving no symlinks though.

[seanfarley (Sean Farley)]

patch to fix the upstream bug of incorrect library search directories and also add all compatible python variants

[seanfarley (Sean Farley)]

The patch I just sent fixes vtk5's build issues (#33490 and #33465) with python by adding variants robustly (just like boost) and fixing the library search path based on Shinko's suggestion. This is still a hack but at least it doesn't do any funky linking outside the build tree. The proper fix needs to come from upstream (Kitware) by fixing their cmake variables for the python wrapping, though, this patch makes it work for now.

[adfernandes (Andrew Fernandes)]

Wow - that's quite a patch; much, much appreciated.

I was wracking my brain trying to come up with a way of fixing things that didn't hack outside the build tree; doing so is a Really Bad Idea.

Let me patch, verify my build, and I'll commit.

Again - thanks to everyone who helped; I'm not actually an active vtk5 user anymore, so this is really awesome.

[adfernandes (Andrew Fernandes)]

Committed in r90726. Thanks!

[seanfarley (Sean Farley)]

Replying to adfernandes@…:

Committed in r90726. Thanks!

Ah, thanks for the quick turn around! Sorry about the huge code review but I thought it would be best to fix the python block in one go. For those wondering, like I did, vtk still doesn't support python 3 (but at least now it'll be easy to add support for those variants!)