RFE: Make Python more Framework Friendly

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

To ease the transition when python25 enabled the Framework option (in r37817), it was decided to move the actual library and include directories to ${prefix}/lib and ${prefix}/include respectively.

foreach dir { lib include } {
    file rename ${destroot}${framewdir}/${dir}/python${branch} ${destroot}${prefix}/${dir}
    ln -s ${prefix}/${dir}/python${branch} ${destroot}${framewdir}/${dir}/python${branch}
}

This can cause a problem because some scripts ask python directly where the library directory is. Python believes that the library directory is on the Framework directory. There is a discussion on the mailing list ​on this problem.

We can put the effort into making ports work with the current python setup, but I humbly suggest that a better long term solution is to to modify the various ports to install into the Framework directory.

Attached is a proposed change to the python25 Portfile as well as the python25 group.

The advantages are:

• The python program will not have the wrong library path.
• It is much easier to avoid conflicts between the versions of python.

The main disadvantage is that many ports will have to be modified (only minor changes, however, are often sufficient).

Of the py25-* ports I have tested, the most common changes required are along the lines of:

configure.pre_args-delete  --prefix=${prefix}
set python_prefix ${prefix}/Library/Frameworks/Python.framework/Versions/2.5
configure.pre_args-append  --prefix=${python_prefix}
configure.args-append      --includedir=${python_prefix}/include/python2.5
configure.env-append PKG_CONFIG_PATH=${python_prefix}/lib/pkgconfig

[ryandesign (Ryan Carsten Schmidt)]

If changes are required to all py25-* ports, wouldn't it make more sense to make those changes in the python25 portgroup file?

Also, we do have the variable ${configure.pkg_config_path}

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

One of the changes I proposed was to the python25 portgroup file (python25-1.0.tcl.diff), but several of the py25-* ports do not use the python25 portgroup. Therefore, other changes are needed.

Thanks for pointing out the variable ${configure.pkg_config_path}. Having never used it, I had forgotten about its existence.

-Marcus

[raimue (Rainer Müller)]

As I did the framework transition, I was planing if this would be possible. It requires changes to the python port group which can only be done with a release currently (see #14553). I didn't want to wait with the framework transition until we get a new release finally.

Not all software is aware of frameworks. They expect python libraries and headers in ${prefix}/lib/python2.5/ respectively ${prefix}/include/python2.5/ and they are not using the python port groups. I think it will be some work to patch them to link with the framework instead of using the standard locations. Therefore there have to be symlinks to the framework at these places.

The big problem is now: To create these symlinks, all users have to uninstall all installed py25-* ports, upgrade python25 and then reinstall their py25-* ports. MacPorts cannot replace an existing directory with a symlink. Not thinking of dependents, most probably a reinstall of all py25-* dependents is necessary, too.

And what you plan for python25 here should also be applied to python24, python26-devel and python30.

[raimue (Rainer Müller)]

Ah, and now I forgot to mention another problem I already discovered when preparing the transition:

The python25 port group is not only used by py25-* ports, for example also by mercurial or duplicity. When using --prefix=${prefix}/Library/Frameworks/Python.framework/Versions/2.5 binaries will end up in ${prefix}/Library/Frameworks/Python.framework/Versions/2.5/bin and will need manual symlinking in post-destroot to ${prefix}/bin. Might also apply to other files and not only binaries, I did not check again.

[mf2k (Frank Schima)]

Cc Me!

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

I agree that what I have proposed is a significant undertaking, but I think that in the long run it might make thinks easier.
I have made these changes on my local build, and of the twenty or so packages which I have tested (which depend on python), most of them only required minor changes.
This is a small sample, but it gives me some hope.

I would argue having binaries install into ${prefix}/Library/Frameworks/Python.framework/Versions/2.5/bin could actually be though of as a good thing.
It makes conflicts easier to manage.
py25-numpy, for example, now requires in post-destroot

move ${destroot}${prefix}/bin/f2py ${destroot}${prefix}/bin/f2py2.5

but, if installed in the framework directory, would only need

ln -s ${prefix}/Library/Frameworks/Python.framework/Versions/2.5/bin/f2py ${destroot}${prefix}/bin/f2py2.5

The actual f2py binaries for the various version are safely separated.

[skymoo (Adam Mercer)]

Cc Me!

[bryan@…]

Cc Me!

[mf2k (Frank Schima)]

Can you please update your patches for the latest versions of the files? They don't work now. I want to try them locally.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Replying to macsforever2000@…:

Can you please update your patches for the latest versions of the files? They don't work now. I want to try them locally.

Attached are the updated versions of the patches.
Please ignore patch-Makefile.diff.
I can not remember why it was there in the first place.

[mf2k (Frank Schima)]

I tried the 2 patchfiles and attempted to reinstall python25 but I see the following errors:

--->  Activating python25 @2.5.4_0+darwin_9+macosx
Error: Target org.macports.activate returned: Image error: /opt/local/include/python2.5 already exists and does not belong to a registered port.  Unable to activate port python25.
Error: Status 1 encountered during processing.
Hyperspace:dports frank$ port installed python25
The following ports are currently installed:
  python25 @2.5.4_0+darwin_9+macosx
$ sudo port -f activate python25
--->  Activating python25
Warning: File /opt/local/include/python2.5 already exists.  Moving to: /opt/local/include/python2.5.mp_1234153366.
Warning: File /opt/local/lib/python2.5 already exists.  Moving to: /opt/local/lib/python2.5.mp_1234153366.
Error: port activate failed: Is a directory

[blb@…]

I wonder if those 'already exists' are from the confusion created when installing ports that result in the 'Not a directory' error message?

[mf2k (Frank Schima)]

I started from scratch and re-applied the patches and this time I am able to install python25. After successfully installing a few py25-* ports, I hit an error:

--->  Staging py25-scientific into destroot
Error: Target org.macports.destroot returned: error renaming "/opt/local/var/macports/build/_opt_mports_trunk_dports_python_py25-scientific/work/destroot/opt/local/bin/bsp_virtual": no such file or directory
Error: The following dependencies failed to build: py25-scientific
Error: Status 1 encountered during processing.

How can I fix that?

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Replying to macsforever2000@…:

I started from scratch and re-applied the patches and this time I am able to install python25. After successfully installing a few py25-* ports, I hit an error: How can I fix that?

I know that boost and graphviz work around the fact that the current python25 uses a hybrid Framework and library installation.
Neither is likely to work with the patches you have applied.
Judging from the differences between py26-scientific and py25-scientific, this seems to be the case with py25-scientific.

You can probably track down the exact problem using py26-scientific.

In any event, based on the amount of time this ticket has been open, I suspect the proposed changes will not be accepted.
There are good reasons for this which are outlined above.

If you wish to keep using the patches, it is entirely possible you will keep running into these kinds of errors.

[stromnov (Andrey Stromnov)]

Cc Me!

[sidney@…]

Cc Me!

[arthurk@…]

Cc Me!

[dbsgeo@…]

Cc Me!

[(none)]

Milestone Port Enhancements deleted

[michaelld (Michael Dickens)]

Cc Me!

[raimue (Rainer Müller)]

Calling this a wontfix for python25.