vtk 5.4.0 (vtk-devel)

[dweber@…]

See Portfile attached, sorry it's a mess and not ready for commit.

[ dweber@elegans ~ ]$ sudo port install vtk54
Password:
--->  Fetching vtk54
--->  Attempting to fetch vtk-5.4.0.tar.gz from http://distfiles.macports.org/vtk54
--->  Attempting to fetch vtk-5.4.0.tar.gz from http://www.vtk.org/files/release/5.4
--->  Attempting to fetch vtkdata-5.4.0.tar.gz from http://distfiles.macports.org/vtk54
--->  Attempting to fetch vtkdata-5.4.0.tar.gz from http://www.vtk.org/files/release/5.4
--->  Verifying checksum(s) for vtk54
--->  Extracting vtk54
--->  Configuring vtk54
--->  Building vtk54
--->  Staging vtk54 into destroot
--->  Installing vtk54 @5.4.0_0+darwin_9+examples+tclSys+testing
--->  Activating vtk54 @5.4.0_0+darwin_9+examples+tclSys+testing
Error: Target org.macports.activate returned: Image error: vtk54 @5.4.0_0+darwin_9+examples+tclSys+testing not installed as an image.
Error: Status 1 encountered during processing.

[dweber@…]

Portfile for vtk54

[raimue (Rainer Müller)]

Why not just update the existing vtk5? I would prefer to avoid versioning ports as much as possible.

[dweber@…]

Replying to raimue@…:

Why not just update the existing vtk5? I would prefer to avoid versioning ports as much as possible.

I think vtk 5.2 may be more stable than this latest release of vtk 5.4.0. I'm not aware of other port depedencies on vtk5 and I would not like to break them by changing the upstream version from 5.2 to 5.4. Also, given the failure of this 5.4 port to install as an image, it seems safer to put it into a different port. If we get it fully working (including the shared library installation, with python wrapping), perhaps then it might become THE vtk5 port. Even if this port for 5.4.0 works, it might be wise to wait for 5.4.1 or other bug fixes before it becomes THE vtk5 port.

In any case, all the include and lib install paths have the major.minor versions to allow for simultaneous installations. Apart from issues of disk space, I can't see any problem with doing that and there are advantages to this for downstream ports that may require a specific version of the library. This appears to be the case with postgresql, for instance. I don't believe anything more specific than major.minor is required.

[dweber@…]

Replying to raimue@…:

Why not just update the existing vtk5? I would prefer to avoid versioning ports as much as possible.

Also note that additional work is under way on vtk5, to address various issues, eg: ​http://trac.macports.org/ticket/19111

[raimue (Rainer Müller)]

It's just I would prefer ports without a version suffix to be the latest and greatest. Only older versions should have the suffix, so it is easier to drop them out later if they are no longer required.

[tgamblin@…]

I second raimue's comment that we should have one vtk5 port. Can't we make vtk vtk4 and merge vtk 5.4 into vtk5, then call it vtk to clear all this up? If people want the old version when this is released they can do:

port install vtk @5.2.blah.

Also since kitware claims the latest stable release is 5.4.0, I see nothing wrong with making 5.4 the head now (especially if people can always install the prev version).

[dweber@…]

Replying to tgamblin@…:

I second raimue's comment that we should have one vtk5 port. Can't we make vtk vtk4 and merge vtk 5.4 into vtk5, then call it vtk to clear all this up? If people want the old version when this is released they can do:

port install vtk @5.2.blah.

Also since kitware claims the latest stable release is 5.4.0, I see nothing wrong with making 5.4 the head now (especially if people can always install the prev version).

I'm not so certain this is the way to "clear" this up. I would prefer to see major.minor version numbers in every library port, as in vtkXY where X are major and Y are minor version numbers.

Even if we resolve to put VTK-5-4-0 (or whatever the latest stable release is) into THE 'vtk' port, what is the convention for the library installation paths? Do we drop all the major.minor indicators? I would hope not. The macports installation tree should contain the potential for many simultaneous version installations, particularly for a library like VTK. I see clear advantages for client ports to be able to specify a dependency on a major.minor port version of VTK. With those major.minor version numbers in the port name, it is very easy to specify the dependency without having to resort to a port variant or something to get the dependency correct. Note that I'm talking about ports with dynamic library dependency on SOME version of the VTK library, I'm not talking about a user trying to specify the port during an interactive port install process.

What about a solution where we have MULTIPLE ports with vtkXY (as in postgresql83) and just ONE 'symbolic' port that is just 'vtk' and it manually or perhaps automatically points to the highest stable release number? So, if you run port install vtk it would tell you that this is resolved into {{port install vtkXY}}} to get the latest version. I see this kind of proxy package all over the place in Debian (Ubuntu) and perhaps it serves just this kind of purpose of keeping up with the latest versions, while also providing clarity about port (package) versions within the port (package) name. In this way, a 'client' port that depends on VTK can specify a version specific dependency (as in vtkXY) or the latest version (as in just vtk).

Aside from these port naming issues, I'm having difficulty with getting shared libraries installed for vtk through macports. I can do a shared library install directly from source (without macports) and everything works fine, including the java, tcl, and python wrapping. However, once the destroot comes into play with macports, all my configure options are broken. I've yet to resolve how to fix this and I'm sort-a stuck in a quagmire of trying to understand the best way to do it, either with some kind of rpath configuration (or postdestroot hacking with install_name_tool) or some environment variable settings for DYLD (without any rpath config). I don't understand frameworks yet, so I've focused entirely on static and dynamic builds into /opt/local/include/vtk-5.x and /opt/local/lib/vtk-5.x

Take care, Darren

[tgamblin@…]

Cc Me!

[tgamblin@…]

Even if we resolve to put VTK-5-4-0 (or whatever the latest stable release is) into THE 'vtk' port, what is the convention for the library installation paths? Do we drop all the major.minor indicators? I would hope not. The macports installation tree should contain the potential for many simultaneous version installations, particularly for a library like VTK.

Can't you do this without versions in the port name, even if they're all listed as "VTK"? e.g. I could do:

port install vtk@5.2.1
port install vtk@5.4.0

VTK already keeps the includes and libraries separate for these ($prefix/include/vtk-X.Y, $prefix/lib/vtk-X.Y, etc), so that shouldn't be a problem, but there are going to be problems with VTKData and VTK's binary utilities (vtkWrapPython, vtkpython, vtkEncodeString, etc), which go under /bin and /share without version-specific subdirectories. Things in these directories will conflict if you try to simultaneously install two versions, and this is going to be a problem even if you have separate names for the ports.

On the other hand, I believe you CAN have multiple versions built at once under this scheme. You just can't have them active:

port install vtk@5.2.1
port build vtk@5.4.0
port deactivate vtk@5.2.1
port activate vtk@5.4.0
... etc

What about a solution where we have MULTIPLE ports with vtkXY (as in postgresql83) and just ONE 'symbolic' port that is just 'vtk' and it manually or perhaps automatically points to the highest stable release number?

I don't have a really strong preference here, but raimue seems to. Also like I said, I don't think this is the problem. I think it's that two VTK installs are going to need entirely separate prefixes to be installed together. raimue?

Maybe it would make sense to come up with some patches so that things get installed in a non-conflicting way, but I think this is a big change and a fair amount of work. I would rather focus on getting the thing working an the names consolidated before working on the ability to have concurrent versions. What do you think?

Aside from these port naming issues, I'm having difficulty with getting shared libraries installed for vtk through macports. I can do a shared library install directly from source (without macports) and everything works fine, including the java, tcl, and python wrapping. However, once the destroot comes into play with macports, all my configure options are broken. I've yet to resolve how to fix this and I'm sort-a stuck in a quagmire of trying to understand the best way to do it, either with some kind of rpath configuration (or postdestroot hacking with install_name_tool) or some environment variable settings for DYLD (without any rpath config). I don't understand frameworks yet, so I've focused entirely on static and dynamic builds into /opt/local/include/vtk-5.x and /opt/local/lib/vtk-5.x

I'm not sure I understand what the problem here is -- can't we base things on the vtk5 Portfile, which lets cmake do all the hard work? I've got vtk-5.2 installed and the libraries seem to work just fine. Can you be a little more specific? What doesn't run?

[ryandesign (Ryan Carsten Schmidt)]

You cannot ask MacPorts to install a specific version of a port. It always installs the current version that is available. See the InstallingOlderPort how-to for instructions on how to work around this missing MacPorts feature and install an older version of a port if necessary.

Discussion of versioning ports is now ​occurring on the mailing list.

If 5.4.0 is not ready to be the vtk5 port yet, then don't commit it to the vtk5 port until you fix it, or consider making a vtk5-devel port where you can fix it up, and merge that to vtk5 when ready.

[dweber@…]

I'm doing a review of all the settings used in vtk (4.x) and vtk5, with regard to variables for RPATH and INSTALL_DIR in cmake (also reviewing the cmake book to make sure I understand these variables). There are some differences in these settings between the two ports, with vtk5 using a LOT more of these settings than vtk (4.4), and vtk5 sneaks in a DYLD environment setting into the build.env (I wonder if that is required to sort-of "undo" the BUILD_WITH_INSTALL_RPATH setting). vtk5 uses INSTALL_RPATH_USE_LINK_PATH - will that point back to ${worksrcpath} or the destroot; what happens if they are removed? The post-configure hack in vtk5 seems a curiousity to me - why isn't that handled by a cmake variable? I'm not clear on how all these settings are working during the build and install into destroot (I assume cmake settings are independent of the activation phase); I need to test a fake destroot build or step through each of the macports phases for my peace of mind.

In vtk5, it might be better to have some of the post-destroot stuff as variants for +doc +data +examples? The data install path is generic rather than version specific, ie: -DVTK_DATA_ROOT:PATH=${prefix}/share/VTKData/. vtk5 +python is using python25 - should we update to python26? (I assume python30 is a bleeding edge.)

I would prefer to see cocoa as the default variant, rather than x11. Perhaps I'm missing something about additional wrapping or something - why is x11 the default variant in vtk5?

These comments should probably go into a specific ticket on vtk (4.4), but I'm feeling lazy... According to the cmake book, CMAKE_BUILD_WITH_INSTALL_RPATH over-rides CMAKE_SKIP_RPATH, so these settings in vtk (4.4.2) may be redundant (they might be effectively consistent anyhow). I'm not clear why it should BUILD_WITH_INSTALL_RPATH - shouldn't the RPATH get reset during the installation? It might be wise to change CMAKE_INSTALL_NAME_DIR to a version specific path in vtk (4.4), how about ${prefix}/lib/vtk-4.4? vtk (4.4) sets ${worksrcdir} - should that be ${worksrcpath}? Should vtk (4.4) use ${x11prefix} instead of ${prefix} in the +x11 variant and should that be a default variant? In vtk (4.4), all the -D config args are followed by a space, whereas vtk5 omits the space (the latter seems to make it easier to config delete in variants - no quotes required around the "-D <var>:<type>=<val>").

In the long_description, it would seem useful to include some comments about the variants. Although variants can specify 'requires' and 'conflicts' clauses, it might help users to know about those before selecting a variant.

I'm content to leave vtk5 alone. My focus is on my local repository version of vtk54 for testing a few things. When I test and clarify my local repository version, I'll run an svn update on the trunk version of both vtk5 and vtk54 (to get all the latest work), double-check my local version with a diff against the trunk, and then try to consolidate into an svn commit for vtk54 (leaving vtk5 alone). If that works and we have a consensus about vtkXY version naming, then (a) vtk54 might become vtk5 (and we remove any ports for vtk5Y), or (b) vtk5 becomes vtk52 and vtk54 remains (or vtk5 becomes vtk52 and vtk54 becomes vtk5).

Please feel free to work on vtk54 as you please (it has openmaintainer!).

[dweber@…]

From discussions on the mailing list, a possible solution to naming is:

vtk becomes vtk4 or vtk44 @4.4.2 vtk5 becomes vtk @5.2.1 vtk54 becomes vtk-devel @5.4.0

I could muck around with the svn trunk to make these changes, but I would prefer Ryan to do it, given more experience with macports. I'm not entirely clear on how best to do this, as it's not only changes to the svn port tree directories, but it also requires changes to the Portfile names and other variables.

If I work on anything in particular in the svn trunk, it will be only vtk-devel (currently vtk54).

Best, Darren

[dweber@…]

Created and committed a first attempt to adapt vtk into vtk44, see svn trunk revision 50221.

[dweber@…]

vtk54 has been modified and committed as vtk-devel into the svn trunk, ie:

svn commit -m "The default installation with shared libraries appears to work now; still lots of work to be done on wrapping for java, python, tcl, among other things." vtk-devel Adding vtk-devel Adding vtk-devel/Portfile Transmitting file data . Committed revision 50222.

[(none)]

Milestone Port Enhancements deleted

[dweber@…]

A note from David Cole at kitware:

There are some API changes from VTK 5.2 to 5.4... Although there are not many, I would still say the current dylib compatibility settings are correct. The 5.4 dylibs cannot be loaded by a program built against 5.2 and work flawlessly.

Some VTK 5.4 API changes are explicitly listed here: ​http://www.vtk.org/Wiki/VTK_5.4_Release_Planning

And I would guess there are a handful of others that are simply not listed anywhere, but would take source code analysis to uncover...

The dylib properties are set by the code in VTK's CMakeLists.txt that looks like this:

# Append the library version information to the library target # properties. A parent project may set its own properties and/or may # block this. IF(NOT VTK_NO_LIBRARY_VERSION)

SET(VTK_LIBRARY_PROPERTIES ${VTK_LIBRARY_PROPERTIES}

VERSION "${VTK_VERSION}" SOVERSION "${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}" )

ENDIF(NOT VTK_NO_LIBRARY_VERSION)

The library VERSION property is the source of the dylib versions...

HTH, David

[dweber@…]

svn commit -m "Many modifications; All the core variants are working, incl. data, doc, examples, shared libs, testing; the data, doc, examples, testing files are installed to /opt/local/share/vtk-${branch}/ and install_name_tool is used to map the binary rapths for examples and testing to the correct library location such that /opt/local/share/vtk-${branch}/*/bin/ contains executables that run the examples and tests" Portfile 
Sending        Portfile
Transmitting file data .
Committed revision 50417.

This version has been developed and tested this week. Some additional features are enabled (GL2PS, N_WAY_ARRAYS). The post-destroot phase is now coded for specifically for each variant that requires it (see the hacks in the examples and testing variants to set the installation paths, copy files, and reset the rpath for binaries; I wish there were an easier way to patch the CMakeLists.txt files for the installation of examples and testing, but I don't have time to figure that out).

Still lots of work to be done on the language wrapping (java, tcl, python) and other rendering variants (carbon, x11; I suppose x11 has a higher priority than carbon). With regard to the x11, the previous commit contained two variants (x11, mesaOpenGL) that are now rolled into one variant (x11). With regard to the language wrappers, there are now two variants for python25 and python26 (nothing yet for python30 - that may be too adventurous). The tcl variants were renamed to tcl (macports) and tcl_apple.

There are numerous variants for databases, etc. and all of them require testing and clarification of their dependencies.

Darren

[dweber@…]

Note: some interesting vtk package information for debian: ​http://packages.qa.debian.org/v/vtk.html

[dweber@…]

Should be able to checkin a new revision this week. Currently have tcl, python, java wrapping working with cocoa (although no extensive testing of all the functionality has been done). At this point, all the build and installation routines are coming along well.

Tested: vtk-devel @5.4.0_0+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing

Testing today: vtk-devel @5.4.0_0+cocoa+darwin_9+data+doc+examples+java+py25+shared+tcl+testing

In the course of testing the py26 variant, discovered a VTK-5.4 egg in the python 2.6 framework site-packages (not sure where it came from, maybe specific to my system or another py26 port is trying to provide vtk). That egg did not work because it didn't link the .so libraries with an rpath. The extra egg was overriding the install from this package, somehow. After removing the other egg, this variant install works ($prefix/bin/vtk-5.4-python26 works, and ipython -> import vtk works).

Renaming the $prefix/bin/vtk to $prefix/bin/vtk-5.4-tcl/. Other binaries in $prefix/bin (eg, vtkWrap*) may be renamed too, if it will not interfere with functionality. If it does, the actual binaries my be renamed and symlinks can be created for version specific defaults (maybe part of a vtk_select port).

[dweber@…]

vtk-devel is now committed at 5.4.0, revision 3. Most of it appears to be working.

It's time to close this ticket. There may be additional issues to resolve, like whether or when the content now in vtk-devel replaces the content of vtk5. In that regard, I've done nothing to date to incorporate the variants from vtk5 into vtk-devel. I think we need more specific tickets for each issue as it arises. The main difference may be that vtk-devel currently has cocoa as the default, whereas vtk5 uses x11 as the default. One would expect that vtk will move to cocoa on OSX. If we find that too many things are still broken in vtk-devel using cocoa, the solutions may be to patch the 5.4.0 release, wait for another 5.4.x release to fix the problem upstream, or, as a last resort, switch the default to x11 (if that still makes sense for the new releases on OSX).

Regards, Darren