llvm-X.Y: fail to build with python3 selected

[cjones051073 (Chris Jones)]

Title says it all ... llvm-3.1 fails to build for me on OSX 10.7 with Xcode 4.2

log file attached.

Chris

[mf2k (Frank Schima)]

In the future, please Cc the maintainer(s).

[jeremyhu (Jeremy Huddleston Sequoia)]

:info:build make: Entering directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_llvm-3.1/llvm-3.1/work/trunk'
:info:build llvm[0]: Constructing LLVMBuild project information.
:info:build Traceback (most recent call last):
:info:build   File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_llvm-3.1/llvm-3.1/work/trunk/utils/llvm-build/llvm-build", line 3, in <module>
:info:build     import llvmbuild
:info:build   File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_llvm-3.1/llvm-3.1/work/trunk/utils/llvm-build/llvmbuild/__init__.py", line 1, in <module>
:info:build     from main import main
:info:build ImportError: No module named main

It builds fine for me here. This looks like python related madness, so /shrug ... =/

[cjones051073 (Chris Jones)]

Sorry about the cc - I thought the bug system would automatically cc the maintainers of the port ... Seems odd I have to do it by hand...

Anyway, could it matter which python version I am using. I have macports python32 install and have 'port selected' it .... ?

Chris

[cjones051073 (Chris Jones)]

The problem does seem to be related to my use of python32 as my default python version. If I switch back toy either macports python27, or the system python version, the build goes OK.

If llvm-3.1 depends on python during the build phase, wouldn't it be best to explicitly depend on a macports python version (2.7, as that seems to work) and use that, rather than just pickup whatever the user has as their 'python' version via $PATH ?

cheers Chris

[jeremyhu (Jeremy Huddleston Sequoia)]

Ok, then it seems the llvm ports (and probably clang, can you please test clang-3.1) need some python magic... yick.

[cjones051073 (Chris Jones)]

clang-3.1 does indeed suffer from the same problem. log attached, in case its useful ...

Chris

[jeremyhu (Jeremy Huddleston Sequoia)]

llvm Portfile patch

[jeremyhu (Jeremy Huddleston Sequoia)]

Does this patch to the llvm-3.1 Portfile address the problem for you?

[cjones051073 (Chris Jones)]

Sorry for the delay in getting back.

I'm afraid it don't work. Same error as before. See attached log.

Note, you can (I hope) reproduce this yourself by just installing python32 and setting it as your default, using

sudo port select python python32

Chris

[cjones051073 (Chris Jones)]

Any progress on fixing this ?

[jeremyhu (Jeremy Huddleston Sequoia)]

I don't use python3, and there's an easy workaround, so it's not a priority for me. Patches are welcome.

Try setting the PYTHON_EXECUTABLE environment variable:

$ grep -R PYTHON .
./CMakeLists.txt:if( NOT PYTHONINTERP_FOUND )
./CMakeLists.txt:Please install Python or specify the PYTHON_EXECUTABLE CMake variable.")
./CMakeLists.txt:  COMMAND ${PYTHON_EXECUTABLE} ${LLVMBUILDTOOL}
./test/CMakeLists.txt:set(PYTHON_EXECUTABLE ${PYTHON_EXECUTABLE})
./test/CMakeLists.txt:  COMMAND ${PYTHON_EXECUTABLE}
./test/lit.cfg:# Propagate PYTHON_EXECUTABLE into the environment
./test/lit.cfg:config.environment['PYTHON_EXECUTABLE'] = getattr(config, 'python_executable',
./test/lit.site.cfg.in:config.python_executable = "@PYTHON_EXECUTABLE@"
./test/Makefile:	@$(ECHOPATH) s=@PYTHON_EXECUTABLE@=python=g >> lit.tmp
./test/Scripts/coff-dump.py.bat:%PYTHON_EXECUTABLE% -u %LLVM_SRC_ROOT%\test\Scripts\coff-dump.py %1 %2 %3 %4 %5 %6 %7 %8 %9
./test/Scripts/elf-dump.bat:%PYTHON_EXECUTABLE% -u %LLVM_SRC_ROOT%\test\Scripts\elf-dump %1 %2 %3 %4 %5 %6 %7 %8 %9
./test/Scripts/macho-dumpx.bat:%PYTHON_EXECUTABLE% -u %LLVM_SRC_ROOT%\test\Scripts\macho-dump %1 %2 %3 %4 %5 %6 %7 %8 %9
./tools/clang/bindings/python/README.txt:$ env PYTHONPATH=$(echo ~/llvm/tools/clang/bindings/python/) \
./tools/clang/test/CMakeLists.txt:if(PYTHONINTERP_FOUND)
./tools/clang/test/CMakeLists.txt:    COMMAND ${PYTHON_EXECUTABLE}
./tools/clang/test/CMakeLists.txt:      COMMAND ${PYTHON_EXECUTABLE}
./tools/clang/test/Makefile:	@ $(PYTHON) $(LLVM_SRC_ROOT)/utils/lit/lit.py \
./utils/lit/lit/TestingConfig.py:                        'PYTHONUNBUFFERED' : '1',

[cjones051073 (Chris Jones)]

Totally fair point ;)

If and when I have time, I'll see if the PYTHON_EXECUTABLE workaround works or not.

cheers Chris

[jmroot (Joshua Root)]

It doesn't look like modifying anything in the environment will work. It executes llvm-build directly, not as an argument to an interpreter, so the #!/usr/bin/env python in the script gets used. Just patching that to /usr/bin/python should be fine.

Upstream should probably add an AM_PATH_PYTHON to configure, and a $(PYTHON) to the invocation in Makefile.rules.

[russel@…]

Cc Me!

[russel@…]

Just a note that I was also seeing this problem, or a directly related problem, and tweeted about it. @neverpanic replied pointing me here. With my usual python32 as default I cannot install LLVM-3.1, switch to python27 and it works fine.

[samkass-macports@…]

If you have python installed through MacPorts, you can do this to make it work:

sudo port select python python27

[cjones051073 (Chris Jones)]

Replying to samkass-macports@…:

If you have python installed through MacPorts, you can do this to make it work:

sudo port select python python27

Well yes.... But that is not the point really. llvm should really be insensitive at build time to the version of python a user has opted to use.

[larryv (Lawrence Velázquez)]

Has duplicate #39700 against llvm-3.3.

[larryv (Lawrence Velázquez)]

Cc Me!

[deesto (John S. De Stefano Jr.)]

Confirmed that switching the port Python version to 2.7 gets around this, even in {llvm|clang}-3.3, but it's kind of a messy hack.

[larryv (Lawrence Velázquez)]

Has duplicate #38800 against llvm-3.2. I can’t reproduce with llvm-3.0, though.

[deesto (John S. De Stefano Jr.)]

Unfortunately a port select switch to python 2.7 does not completely work after all in my case; after a python 2.7 port switch, and port clean, clang-3.3 fails to upgrade:

In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/projects/compiler-rt/lib/profile/GCDAProfiling.c:26:
/opt/local/include/sys/stat.h:75:10: fatal error: 'Availability.h' file not found
#include <Availability.h>
         ^
1 error generated.
make[4]: *** [/opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/tools/clang/runtime/compiler-rt/clang_darwin/profile_osx/i386/SubDir.lib__profile/GCDAProfiling.o] Error 1
make[4]: *** Waiting for unfinished jobs....
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/projects/compiler-rt/lib/profile/GCDAProfiling.c:26:
/opt/local/include/sys/stat.h:75:10: fatal error: 'Availability.h' file not found
#include <Availability.h>

See attached debug log. I can file a separate ticket if warranted.

[jeremyhu (Jeremy Huddleston Sequoia)]

You are missing your system headers.

[deesto (John S. De Stefano Jr.)]

Replying to jeremyhu@…:

You are missing your system headers.

Thanks Jeremy. How've I been running ports for >5 years without them? :/

[larryv (Lawrence Velázquez)]

Replying to deesto@…:

Unfortunately a port select switch to python 2.7 does not completely work after all in my case; after a python 2.7 port switch, and port clean, clang-3.3 fails to upgrade:

In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/projects/compiler-rt/lib/profile/GCDAProfiling.c:26:
/opt/local/include/sys/stat.h:75:10: fatal error: 'Availability.h' file not found
#include <Availability.h>
         ^
1 error generated.
make[4]: *** [/opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/tools/clang/runtime/compiler-rt/clang_darwin/profile_osx/i386/SubDir.lib__profile/GCDAProfiling.o] Error 1
make[4]: *** Waiting for unfinished jobs....
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/projects/compiler-rt/lib/profile/GCDAProfiling.c:26:
/opt/local/include/sys/stat.h:75:10: fatal error: 'Availability.h' file not found
#include <Availability.h>

See attached debug log. I can file a separate ticket if warranted.

This is #39605.

[mymacports@…]

Cc Me!

[mymacports@…]

same problem here.

clang-3.3 fails to upgrade.

OS X 10.8.4

Availability.h is in /usr/include

yes, this is 39605.

unfortunately 39605 was (prematurely, imo) marked as 'invalid' ..

Replying to larryv@…:

Replying to deesto@…:

Unfortunately a port select switch to python 2.7 does not completely work after all in my case; after a python 2.7 port switch, and port clean, clang-3.3 fails to upgrade:

In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/projects/compiler-rt/lib/profile/GCDAProfiling.c:26:
/opt/local/include/sys/stat.h:75:10: fatal error: 'Availability.h' file not found
#include <Availability.h>
         ^
1 error generated.
make[4]: *** [/opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/tools/clang/runtime/compiler-rt/clang_darwin/profile_osx/i386/SubDir.lib__profile/GCDAProfiling.o] Error 1
make[4]: *** Waiting for unfinished jobs....
In file included from /opt/local/var/macports/build/_opt_local_var_macports_sources_svn.macports.org_trunk_dports_lang_llvm-3.3/clang-3.3/work/llvm-3.3.src/projects/compiler-rt/lib/profile/GCDAProfiling.c:26:
/opt/local/include/sys/stat.h:75:10: fatal error: 'Availability.h' file not found
#include <Availability.h>

See attached debug log. I can file a separate ticket if warranted.

This is #39605.

[larryv (Lawrence Velázquez)]

Replying to mymacports@…:

yes, this is 39605.

unfortunately 39605 was (prematurely, imo) marked as 'invalid'

Irrelevant. Keep your comments on that ticket if your issue is with Availability.h.

[cooljeanius (Eric Gallager)]

Cc Me!

[larryv (Lawrence Velázquez)]

Has duplicate #40115.

[samuelandjw]

Cc Me!

[larryv (Lawrence Velázquez)]

Has duplicate #40850.

[michaelbartz@…]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #40966.

[jeremyhu (Jeremy Huddleston Sequoia)]

3.3 and 3.4 fixed with r113409

[schmunk@…]

Cc Me!

[schmunk@…]

Same here with python34 and llvm-3.3. I had to switch back to python27.

[jeremyhu (Jeremy Huddleston Sequoia)]

We don't want to add extra build dependencies to llvm where we can avoid it as it is too low-level in the toolchain. We should just force using the system python instead if needed.

[ryandesign (Ryan Carsten Schmidt)]

Replying to jeremyhu@…:

3.3 and 3.4 fixed with r113409

This only changed the pythonXY variants in the clang-X.Y subports. Those pythonXY variants have since been deleted and the clang-X.Y subports now always use python27. But the bug report was about the llvm-X.Y ports, not clang-X.Y. With python 3.4.2 selected, I had no problem just now building llvm-3.7 @3.7-r235803 or llvm-3.5 @3.5.2 or llvm-3.4 @3.4.2. But llvm-3.3 @3.3 still has the problem. So I guess in llvm-3.4 they fixed the incompatibility with python 3.

[larryv (Lawrence Velázquez)]

Has duplicate #48167.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicates #48256, #48257.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

I just ran into this.
Attached is a proposed solution.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Cc Me!

[cjones051073 (Chris Jones)]

Replying to ryandesign:

Replying to jeremyhu@…:

3.3 and 3.4 fixed with r113409

This only changed the pythonXY variants in the clang-X.Y subports. Those pythonXY variants have since been deleted and the clang-X.Y subports now always use python27. But the bug report was about the llvm-X.Y ports, not clang-X.Y. With python 3.4.2 selected, I had no problem just now building llvm-3.7 @3.7-r235803 or llvm-3.5 @3.5.2 or llvm-3.4 @3.4.2. But llvm-3.3 @3.3 still has the problem. So I guess in llvm-3.4 they fixed the incompatibility with python 3.

So...., if this issue is fixed in llvm-3.4 and newer, can we now close this ticket as we probably no longer really care about the older versions ?

[pmetzger (Perry E. Metzger)]

cjones051073: you opened the ticket. If you feel it should be closed, I'll close it.

[cjones051073 (Chris Jones)]

be gone !!