Split python25 into python25-core and modules

[nirs@…]

$ python2.5
Python 2.5.1 (r251:54863, Jul 29 2007, 14:28:00) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sha
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/lib/python2.5/sha.py", line 6, in <module>
    from hashlib import sha1 as sha
  File "/opt/local/lib/python2.5/hashlib.py", line 133, in <module>
    md5 = __get_builtin_constructor('md5')
  File "/opt/local/lib/python2.5/hashlib.py", line 60, in __get_builtin_constructor
    import _md5
ImportError: No module named _md5
>>> 

sha is important core module, many apps will break without it.

There was no problem with Python 2.5.0.

Environment: Mac OS X 10.4.8

[nirs@…]

Seems that it is caused by ​setup.py patch

 # This global variable is used to hold the list of modules to be disabled.
7	-disabled_module_list = []
8	+disabled_module_list = ["zlib","_hashlib","_ssl","_bsddb","_sqlite3","_tkinter","bz2","gdbm","readline","_curses","_curses_panel"]

The comment is quite useless - the variable name is self explaining. The comment should describe WHY these modules are disabled.

Why these modules are disabled?

[nox@…]

Python _hashlib core module has been disabled in python itself because they are available through specific ports. As they should be core modules, I think their ports should be removed of the tree and reenabled in python25 portfiles.

The module ports are:

• py25-zlib
• py25-hashlib
• py25-bsddb
• py25-sqlite3
• py25-tkinter
• py25-bz2
• py25-gdbm
• py25-readline
• py25-curses

[nox@…]

I've added the reporter in the Cc list, at least, he will now he have to install py25-hashlib

[raimue (Rainer Müller)]

Tickets #11931, #12586, #12369 are duplicates describing the same issue. The resolution is to install py25-hashlib at the moment. As nox suggested we could also move this back into the main python25 port. How should we handle it?

[nirs@…]

Why these modules were made available as separate modules?

[mww@…]

If all modules of python were to be activated, python would have dependencies on:

tcl, tk, openssl, db44, bzip2, ncursesw, zlib, gdbm, readline & sqlite3

Which is quite heavy if you just want a python interpreter (perhaps even unknowingly as a dependency). If someone is keen on having all these in a bundle, please create a 'python25-full' meta port which drags in all the py25-modules you think are important - including perhaps even more modules than come with the standard installation.

[nirs@…]

Python 2.5 port should be exactly the same as Python distributed from ​http://python.org. If you want to create a lightweight version, create a python-lite port that does not include these dependencies, and add proper docs how to get the missing components.

[nox@…]

...or just add proper docs on how to get the missing component in this port.

[mww@…]

So what are the dependencies of the "same as the Python distribution"? Depends a lot on what exactly the configure-script finds, so the only deterministic solution is to provide everything.

People coming from e.g. FreeBSD would expect the python port to be exactly like it is - the FreeBSD ports also have a 'Python-lite standard' installation.

Also you totally ingored the fact that a "python-full" + python (like it is currently) would be compatible while your "python-lite" and "python-full" is NOT. You can only install one at a time, while you can "upgrade" from the current port to a "full" port.

If you're missing the docs: This is an open project, so feel free to join and provide them. It was not clear to me that people w/o a FreeBSD background would not find out themselves whats the current state.

[nirs@…]

Well this is the problem, not many users have FreeBSD background and do not expect to get python25-lite and additional ports. Any user that use previous MacPorts versions expect a full Python port.

What the problem in having the same port as python24? Did anyone complained about 2.4 being too big?

I already submitted a patch to add ​docs to the port 2 month ago.

If you think that the docs are missing, why did you close this bug?

[mww@…]

I think this is a misunderstanding: "docs" is the abbreviation for "documentation" and I assumed you wanted the behavior of python25 documented, which is btw. the same as python 2.4 (see py-bsddb, py-tkinter, etc.). I assumed you wanted to add some lines to the wiki or have a post-install message about modules that are a good idea to install, too.

Anyway, to come to a solution: You want a "full" size port while I think a slim port with extras is more reasonable. We can get both what we want if you just create a "python25-with-extras" port (I dont care about the name). If I just bloat the main port, it is not possible anymore to have a slim installation (because an additional python25-lite port would conflict with the python25 port).

So why dont you just create a "python25-full" port so everybody can get what they want?

[nirs@…]

Python 2.5 is documented now, but was not documented when this bug was reported. However, the docs are not accessible enough. You need online docs, accessible from the ports tool.

I'm not sure why python2.5 can not work like python24, which does not require separate ports to get basic functionality like md5 to work. I'm also not convinced that there is a need for slim python25 port. but I don't have time to research this myself now.

About practical solution in your direction, how about this:

1. Rename python25 to python25-core
2. Add python25 port that depends on python25-core and all those py25-xxx dependencies

I should have time to work on this in 2-3 of weeks.

[matt macports@…]

+1 for nirs's python25 / python25-core split

This makes it much easier for end-users to "port install python25" and be able to follow standard instructions for installing software not packaged in MacPorts. Packagers on the other hand have a deeper understanding of the dependencies of their app and can choose python25-core + the few extra packages they need instead of pulling in all of the Python libs.

[justin1@…]

This is going get a lot more attention soon because of Google's App Engine. The GAE development server forbids importing C extensions, so _hashlib is blocked with MacPorts, causing lots of problems with the server.

[sasha@…]

I'm attempting to use virtualenv under python25. I followed the instructions in this ticket and installed py25-hashlib. But virtualenv --no-site-packages errors out with the error bellow.

I believe the reason is the the copy of python executable copied to the new environment by the virtualenv script only has access to hashlib.py that's part of the crippled python25 port and not to the hashlib that's installed as part of the py25-hashlib port.

is there a way around this?

New python executable in ./bin/python
Installing setuptools...........
  Complete output from command ./bin/python -c "#!python
\"\"\"Bootstrap setuptoo...




" /opt/local/Library/Frameworks/...5.egg:
  Traceback (most recent call last):
  File "<string>", line 267, in <module>
  File "<string>", line 226, in main
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/virtualenv-1.2-py2.5.egg/support-files/setuptools-0.6c8-py2.5.egg/setuptools/command/easy_install.py", line 21, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/virtualenv-1.2-py2.5.egg/support-files/setuptools-0.6c8-py2.5.egg/setuptools/package_index.py", line 3, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib2.py", line 91, in <module>
    import hashlib
ImportError: No module named hashlib
----------------------------------------
...Installing setuptools...done.
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/bin/virtualenv", line 8, in <module>
    load_entry_point('virtualenv==1.2', 'console_scripts', 'virtualenv')()
  File "/opt/local/lib/python2.5/site-packages/virtualenv-1.2-py2.5.egg/virtualenv.py", line 389, in main
    unzip_setuptools=options.unzip_setuptools)
  File "/opt/local/lib/python2.5/site-packages/virtualenv-1.2-py2.5.egg/virtualenv.py", line 638, in create_environment
    install_setuptools(py_executable, unzip=unzip_setuptools)
  File "/opt/local/lib/python2.5/site-packages/virtualenv-1.2-py2.5.egg/virtualenv.py", line 282, in install_setuptools
    cwd=cwd)
  File "/opt/local/lib/python2.5/site-packages/virtualenv-1.2-py2.5.egg/virtualenv.py", line 450, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command ./bin/python -c "#!python
\"\"\"Bootstrap setuptoo...




" /opt/local/Library/Frameworks/...5.egg failed with error code 1

Replying to nirs@…:

$ python2.5
Python 2.5.1 (r251:54863, Jul 29 2007, 14:28:00) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sha
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/lib/python2.5/sha.py", line 6, in <module>
    from hashlib import sha1 as sha
  File "/opt/local/lib/python2.5/hashlib.py", line 133, in <module>
    md5 = __get_builtin_constructor('md5')
  File "/opt/local/lib/python2.5/hashlib.py", line 60, in __get_builtin_constructor
    import _md5
ImportError: No module named _md5
>>> 

sha is important core module, many apps will break without it.

There was no problem with Python 2.5.0.

Environment: Mac OS X 10.4.8

[casbon@…]

It's worth mentioning that this split creates problems if you use 'python -S'.

From the manpage:

       -S     Disable  the import of the module site and the site-dependent manipulations of sys.path that it
              entails.

So if you have software that invokes itself through 'python -S' it will not get any modules outside of the built python distribution. The times I have seen this, it is to prevent picking up any local customisations. Unfortunately, if the software uses hashlib which it is fair to assume *should* be there, it will fail, and there is no way of fixing it using macports. You have to manually copy the hashlib module from where macports puts it to where python expects it.

-1 for this split. Keep it as it's shipped.

[nirsof@…]

The ticket name is wrong - it should be Rename python to python-core, add python port that depends on all py25-xxx ports.

The current python port is already split - and this is the problem. A user that install python expect it to contain everything installed by the regular python shipped by python.org.

[nirsof@…]

Cc Me!

[mf2k (Frank Schima)]

Cc Me!

[bryan@…]

Cc Me!

[akitada@…]

Cc Me!

[blb@…]

Cc Me!

[skymoo (Adam Mercer)]

Cc Me!

[free.as.in.speech@…]

Cc Me!

[anthony.theocharis@…]

Cc Me!

[arthurk@…]

Cc Me!

[jezdez@…]

Any news when this brain dead error is going to be fixed?

[jezdez@…]

Patch to make the python25 port to behave as expected, e.g. like python24, python26, Apple's Python and python.org's Python.

[jezdez@…]

That patch is a result of a discussion on IRC about #18567 that made me wonder why Python isn't build with hashlib etc.

[jezdez@…]

New patch that refers to ​http://lists.macosforge.org/pipermail/macports-dev/2009-April/008236.html and only removes the tk/tcl dependency.

[jezdez@…]

Updated patch for r49645

[(none)]

Milestone Port Enhancements deleted

[arthurk@…]

"python25-reenable-libs.4.diff" removes the dependencies on core module ports and updates python25 to enable core modules.

[arthurk@…]

enabled and added dep on tk

[arthurk@…]

fixed ncurses dep on python25 port

[jmroot (Joshua Root)]

There's no point rev bumping all the (former) dependents. The dependency relationship will still stay in the registry because of #13054.

[arthurk@…]

Remove rev-bumping

[arthurk@…]

Okay, I didn't know that. "python25-reenable-libs.7.diff" removes all the rev-bumping.

[arthurk@…]

The latest patch turns the core module ports into stubs.

[blb@…]

py25-opengl and py25-yum-metadata-parser have 'port:port:' typos in dependencies, other than that it looks good to me; unless Markus says otherwise I think it definitely falls into maintainer timeout.

[arthurk@…]

Fixed 'port:port:' typos

[macports@…]

how can I apply this patch ? is it safe to use it ? what happen if I run a safeupgrade ? cheers

[macports@…]

Cc Me!

[macports@…]

Cc Me!

[macports@…]

Cc Me!

[arthurk@…]

Update patch to r51758

[arthurk@…]

If there are no objections, I'm going to commit the patch next weekend.

[mf2k (Frank Schima)]

I'm against this patch because I need py25-tkinter based on tk 8.4. I am able to modify my local py25-tkinter portfile to manage this. See Ticket #14488. Unless someone can allow this in the new arrangement.

[blb@…]

Replying to macsforever2000@…:

I'm against this patch because I need py25-tkinter based on tk 8.4. I am able to modify my local py25-tkinter portfile to manage this. See Ticket #14488. Unless someone can allow this in the new arrangement.

Can you replace your local py25-tkinter with a local python25 which uses an 8.4 version of tk instead?

[mf2k (Frank Schima)]

Replying to blb@…:

Replying to macsforever2000@…:

I'm against this patch because I need py25-tkinter based on tk 8.4. I am able to modify my local py25-tkinter portfile to manage this. See Ticket #14488. Unless someone can allow this in the new arrangement.

Can you replace your local py25-tkinter with a local python25 which uses an 8.4 version of tk instead?

After chatting with blb on irc, and looking at python26 with tkinter, i am convinced it will work fine for me. So i withdraw my objection!

[raimue (Rainer Müller)]

Your patch has trailing whitespace after a backslash at depends_lib in the Portfile for python25. This is fatal and not going to work.

[arthurk@…]

Replying to raimue@…:

Your patch has trailing whitespace after a backslash at depends_lib in the Portfile for python25. This is fatal and not going to work.

Whoops! Thanks for spotting this. The .11 patch fixes this. The python25 port was the only one where this error occured.

[arthurk@…]

Checked in with r51955

[mf2k (Frank Schima)]

.