vim: breaks with two Python versions

[ml@…]

Running:

• Mac OS 10.8.5, XCode 5 (iMac 2013)
• Mac OS 10.7.5, XCode 4.5 (Macbook Air 2011)

Related Issues (probably?):

• ticket:39207 vim @7.3.762_2+cscope+huge+perl+python27+python33+tcl is broken after upgrading MacPorts python27 from 2.7.3 to 2.7.5

Python 2.7 and 3.3 work fine, as do vim @7.4.052_0+breakindent+huge+python27 and vim @7.4.052_0+breakindent+huge+python33

However, when both Python options are set, Vim does not function as expected.

:python3 import sys
E370: Could not load library Python.framework/Versions/3.3/Python
E263: Sorry, this command is disabled, the Python library could not be loaded.

:python import sys
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 548, in <module>
main()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 530, in main
known_paths = addusersitepackages(known_paths)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 266, in addusersitepackages
user_site = getusersitepackages()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 241, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 231, in getuserbase
USER_BASE = get_config_var('userbase')
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sysconfig.py", line 516, in get_config_var
return get_config_vars().get(name)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sysconfig.py", line 449, in get_config_vars
import re
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/re.py", line 105, in <module>
import sre_compile
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sre_compile.py", line 14, in <module>
import sre_parse
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sre_parse.py", line 17, in <module>
from sre_constants import *
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sre_constants.py", line 18, in <module>
from _sre import MAXREPEAT
ImportError: cannot import name MAXREPEAT

I am unclear on what is happening, as the path to Python looks correct. Furthermore, I have no problem importing MAXREPEAT from _sre.

I attempted a clean install of Macports on the Macbook Air (10.7), but the problem persists.

When I rebuild from source with: $ sudo port -d -s install vim +breakindent +huge +python27 +python33

I successfully see:

checking Python version... 2.7
checking Python is 2.3 or better... yep
checking Python's install prefix... /opt/local/Library/Frameworks/Python.framework/Versions/2.7
checking Python's execution prefix... /opt/local/Library/Frameworks/Python.framework/Versions/2.7
checking Python's configuration directory... /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config
checking if -pthread should be used... no
checking if compile and link flags for Python are sane... yes
checking --enable-python3interp argument... yes
checking --with-python3 argument... /opt/local/bin/python3.3
checking Python version... 3.3
checking Python is 3.0 or better... yep
checking Python's abiflags... m
checking Python's install prefix... /opt/local/Library/Frameworks/Python.framework/Versions/3.3
checking Python's execution prefix... /opt/local/Library/Frameworks/Python.framework/Versions/3.3
checking Python's configuration directory... /opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/config-3.3m
checking if -pthread should be used... no
checking if compile and link flags for Python 3 are sane... yes
checking whether we can do without RTLD_GLOBAL for Python... no
checking whether we can do without RTLD_GLOBAL for Python3... yes

I've attached the full debug/source output from both versions of the OS.

[ml@…]

Debug Output for Lion Install

[ml@…]

Debug Output for Mountain Lion Install

[raimue (Rainer Müller)]

Please be aware that only one version of python can be loaded at a time in vim anyway. This is the correct way this is supposed to work and documented in :help python-2-and-3.

But the problems you encountered are probably really broken behavior. I could reproduce the issue with :python3 import sys and it's clearly missing the path to the frameworks directory here. I am looking into this.

However, I could not reproduce the second issue with :python import sys and MAXREPEAT on Mavericks. It could be you are seeing an upstream issue introduced in 2.7.4 (​http://bugs.python.org/issue18050). Was your vim built against python27 @2.7.4 and then python27 was upgraded later on?

[jambonrose (Andrew Pinkham)]

My understanding is that I can switch between Python versions by closing vim. For example, if I have invoked Python 2.7, I can close&reopen vim and invoke Python 3, which will be loaded dynamically. Is this not the case?

If this is the case, then the errors above are still valid. I invoked import sys for the two Python versions in different runs of vim.

On 10.8, I upgraded to Python 2.7.5 (from 2.7.3, I believe) and then installed vim. However, on 10.7, I reinstalled Macports from scratch, and then installed the latest versions directly (and still saw the error).

I've taken a look at the Python bug. It does seem related, but I don't think my case fits into any of the ones laid out by Ned Deily: ​http://bugs.python.org/issue18050#msg190132. Specifically:

• vim is not statically compiling Python
• I have seen these errors inside and outside of a virtual environment
• Python, virtualenvwrapper (and associated), and even pip are all the latest version, and correctly set in the terminal environment.

Bottom line: not sure what to make of it.

I will probably be upgrading to Mavericks this weekend. If I do, I will let you know if I continue to see the error.

[raimue (Rainer Müller)]

Oh well, it's an old problem returning. Turns out that both DYNAMIC_PYTHON3_DLL and DYNAMIC_PYTHON_DLL are set to relative paths to the framework. That makes vim pick up the Python 2.7 from /System/Library/Frameworks -- which works on Mavericks, as it is 2.7.5, but fails on Mountain Lion and earlier as it is an incompatible <= 2.7.3. Of course, it fails for Python 3.3, as that does not exist in OS X itself.

I am pretty sure this is broken since r108762 (#39223), which changed the LINKFORSHARED variable and removed the part that previously added an absolute path to the framework. I am working on a patch.

[raimue (Rainer Müller)]

Here is a patch that works for me for +python27 +python33. I can run both :py print "Hello" and :py3 print("Hello").

However, it only works without my .vimrc and plugins. Otherwise, python 3.x causes a Segmentation fault... I am not sure if the linking is still wrong or it's some other problem.

[jambonrose (Andrew Pinkham)]

I am currently on Mac OS 10.9.2 and Xcode 5.1 (Build version 5B130a according to xcodebuild -version).

For the sake of clarity:

$ port echo requested
py27-pip                       @1.5.4_0 
py27-virtualenvwrapper         @4.0_0 
py33-pip                       @1.5.4_0 
py33-virtualenvwrapper         @4.0_0 
python27                       @2.7.6_0 
python33                       @3.3.5_0 
vim                            @7.4.052_0+breakindent+cscope+huge+python27+python33

It was not immediately apparent to me how to apply the patch. For the benefit of others:

$ sudo port uninstall vim
$ cd /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/editors/vim/files
$ sudo curl -O https://trac.macports.org/raw-attachment/ticket/41344/vim-python-linkage.diff
$ sudo patch < vim-python-linkage.diff
$ sudo rm vim-python-linkage.diff
$ sudo port install -s vim +breakindent +cscope +huge +python27 +python33

I started without a .vimrc or .vim/ bundle directory. Starting vim and entering the following results in a segfault.

:py print "Hello"
:py3 print("Bonjour")

Specifically, the terminal prints:

Vim: Caught deadly signal SEGV
Vim: Finished.
Segmentation fault: 11

However, the following runs absolutely fine:

:py3 print("Bonjour")
:py print "Hello"
:py3 print("Konichiwa")

What's more, after running the commands above, I am able to see the following:

:python print sys.prefix
/opt/local/Library/Frameworks/Python.framework/Versions/2.7
:python3 print(sys.prefix)
/opt/local/Library/Frameworks/Python.framework/Versions/3.3

I rebuilt my .vimrc file, checking the commands and bundles as I went. None of the native settings or commands I use in vim caused any problems. The only (vundle) package that caused problems was Lokaltog/powerline, which caused any command to :py3 to segfault as above. The rest of my bundles (below) work fine.

• gmarik/vundle
• scrooloose/nerdtree
• ervandew/supertab
• tpope/vim-fugitive
• scrooloose/syntastic
• davidhalter/jedi-vim
• vim-scripts/django.vim

Notably, jedi-vim caused no issue, but I was unable to test whether it was using omnicompletion for Python version 2 or 3. I would love help here: is there a command that will tell me which Python version is loaded? has() will attempt to dynamically load Python, and is not useful in this context.

The behaviors described above are erratic and undesirable. I'm increasingly under the impression that this is not a MacPorts problem, and much more a vim problem. Is the version of vim being used in MacPorts the Unix version, or one of the Mac OS X flavors? Essentially: who should I go to with this bug?

[raimue (Rainer Müller)]

As I stated above, I saw the same segmentation fault happening when using my plugins. At the moment, I would recommend to use either +python2x or +python3x, as using both at the same time seems to be a bit unstable.

Yes, this is the Unix version of vim, so I would kindly direct you to the ​vim_dev list. There is also ​vim_mac, but it mostly focuses on the MacVim GUI.

[jambonrose (Andrew Pinkham)]

Apologies for the confusion: I was simply confirming the behavior you'd mentioned, expanding on some of the details, and seeking a direction to take this in.

I will try to see whether I can recreate the problem with the current version of vim for Unix, and then contact the vim_dev list depending on my results.

I'll let you know if I find anything interesting.

[raimue (Rainer Müller)]

I fixed the linking issue in r136909. However, I still see a segmentation fault when using :python3 in such a setup.

[raimue (Rainer Müller)]

Has duplicate #48392.

Upstream ticket for MacVim: ​​https://github.com/macvim-dev/macvim/issues/58 (reported to be fixed in snapshot-80)

[raimue (Rainer Müller)]

I don't know if this issue still exists in vim @8.0.0596, but it is definitely something that should be handled by upstream.

[raimue (Rainer Müller)]

vim no longer causes a segmentation fault when using :py3 after :py when both variants are enabled. I consider this as working as intended by upstream.