Opened 5 years ago
Last modified 3 years ago
#59785 assigned defect
llvm ports opportunistically build ocaml binding even when +ocaml variant is not selected; leads to conflicts between them
Reported by: | cooljeanius (Eric Gallager) | Owned by: | jeremyhu (Jeremy Huddleston Sequoia) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.6.2 |
Keywords: | Cc: | larryv (Lawrence Velázquez) | |
Port: | llvm-7.0 llvm-9.0 |
Description
This bug applies to multiple llvm ports, but let's use 7.0 as an example. Trying to activate it leads to:
Error: port activate failed: Image error: /opt/local/lib/ocaml/META.llvm is being used by the active llvm-9.0 port. Please deactivate this port first, or use 'port -f activate llvm-7.0' to force the activation.
As you can see, none of my llvm ports have ocaml variants active, so none of them should be installing ocaml files:
$ port installed llvm* The following ports are currently installed: llvm-3.3 @3.3_10+universal (active) llvm-3.4 @3.4.2_12+universal (active) llvm-3.5 @3.5.2_9+assertions+universal (active) llvm-3.7 @3.7.1_4+universal (active) llvm-3.9 @3.9.1_5+universal llvm-4.0 @4.0.1_2+universal llvm-5.0 @5.0.2_1+universal llvm-6.0 @6.0.1_1+universal llvm-7.0 @7.0.1_2+emulated_tls+universal llvm-8.0 @8.0.1_0+emulated_tls+universal (active) llvm-9.0 @9.0.0_0+emulated_tls llvm-9.0 @9.0.0_0+emulated_tls+universal (active) llvm-gcc42 @2336.11_3+universal (active) llvm_select @2_0 (active)
And yet:
$ ls /opt/local/lib/ocaml/*llvm* /opt/local/lib/ocaml/META.llvm /opt/local/lib/ocaml/META.llvm_Hexagon /opt/local/lib/ocaml/META.llvm_PowerPC /opt/local/lib/ocaml/META.llvm_X86 /opt/local/lib/ocaml/META.llvm_AArch64 /opt/local/lib/ocaml/META.llvm_Lanai /opt/local/lib/ocaml/META.llvm_RISCV /opt/local/lib/ocaml/META.llvm_XCore /opt/local/lib/ocaml/META.llvm_AMDGPU /opt/local/lib/ocaml/META.llvm_MSP430 /opt/local/lib/ocaml/META.llvm_Sparc /opt/local/lib/ocaml/META.llvm_ARM /opt/local/lib/ocaml/META.llvm_Mips /opt/local/lib/ocaml/META.llvm_SystemZ /opt/local/lib/ocaml/META.llvm_BPF /opt/local/lib/ocaml/META.llvm_NVPTX /opt/local/lib/ocaml/META.llvm_WebAssembly /opt/local/lib/ocaml/llvm: libllvm.a llvm.mli llvm_Lanai.cmxa llvm_Sparc.cmx llvm_analysis.cmi llvm_linker.cma libllvm_AArch64.a llvm_AArch64.a llvm_Lanai.mli llvm_Sparc.cmxa llvm_analysis.cmx llvm_linker.cmi libllvm_AMDGPU.a llvm_AArch64.cma llvm_MSP430.a llvm_Sparc.mli llvm_analysis.cmxa llvm_linker.cmx libllvm_ARM.a llvm_AArch64.cmi llvm_MSP430.cma llvm_SystemZ.a llvm_analysis.mli llvm_linker.cmxa libllvm_BPF.a llvm_AArch64.cmx llvm_MSP430.cmi llvm_SystemZ.cma llvm_bitreader.a llvm_linker.mli libllvm_Hexagon.a llvm_AArch64.cmxa llvm_MSP430.cmx llvm_SystemZ.cmi llvm_bitreader.cma llvm_passmgr_builder.a libllvm_Lanai.a llvm_AArch64.mli llvm_MSP430.cmxa llvm_SystemZ.cmx llvm_bitreader.cmi llvm_passmgr_builder.cma libllvm_MSP430.a llvm_AMDGPU.a llvm_MSP430.mli llvm_SystemZ.cmxa llvm_bitreader.cmx llvm_passmgr_builder.cmi libllvm_Mips.a llvm_AMDGPU.cma llvm_Mips.a llvm_SystemZ.mli llvm_bitreader.cmxa llvm_passmgr_builder.cmx libllvm_NVPTX.a llvm_AMDGPU.cmi llvm_Mips.cma llvm_WebAssembly.a llvm_bitreader.mli llvm_passmgr_builder.cmxa libllvm_PowerPC.a llvm_AMDGPU.cmx llvm_Mips.cmi llvm_WebAssembly.cma llvm_bitwriter.a llvm_passmgr_builder.mli libllvm_RISCV.a llvm_AMDGPU.cmxa llvm_Mips.cmx llvm_WebAssembly.cmi llvm_bitwriter.cma llvm_scalar_opts.a libllvm_Sparc.a llvm_AMDGPU.mli llvm_Mips.cmxa llvm_WebAssembly.cmx llvm_bitwriter.cmi llvm_scalar_opts.cma libllvm_SystemZ.a llvm_ARM.a llvm_Mips.mli llvm_WebAssembly.cmxa llvm_bitwriter.cmx llvm_scalar_opts.cmi libllvm_WebAssembly.a llvm_ARM.cma llvm_NVPTX.a llvm_WebAssembly.mli llvm_bitwriter.cmxa llvm_scalar_opts.cmx libllvm_X86.a llvm_ARM.cmi llvm_NVPTX.cma llvm_X86.a llvm_bitwriter.mli llvm_scalar_opts.cmxa libllvm_XCore.a llvm_ARM.cmx llvm_NVPTX.cmi llvm_X86.cma llvm_executionengine.a llvm_scalar_opts.mli libllvm_all_backends.a llvm_ARM.cmxa llvm_NVPTX.cmx llvm_X86.cmi llvm_executionengine.cma llvm_target.a libllvm_analysis.a llvm_ARM.mli llvm_NVPTX.cmxa llvm_X86.cmx llvm_executionengine.cmi llvm_target.cma libllvm_bitreader.a llvm_BPF.a llvm_NVPTX.mli llvm_X86.cmxa llvm_executionengine.cmx llvm_target.cmi libllvm_bitwriter.a llvm_BPF.cma llvm_PowerPC.a llvm_X86.mli llvm_executionengine.cmxa llvm_target.cmx libllvm_executionengine.a llvm_BPF.cmi llvm_PowerPC.cma llvm_XCore.a llvm_executionengine.mli llvm_target.cmxa libllvm_ipo.a llvm_BPF.cmx llvm_PowerPC.cmi llvm_XCore.cma llvm_ipo.a llvm_target.mli libllvm_irreader.a llvm_BPF.cmxa llvm_PowerPC.cmx llvm_XCore.cmi llvm_ipo.cma llvm_transform_utils.a libllvm_linker.a llvm_BPF.mli llvm_PowerPC.cmxa llvm_XCore.cmx llvm_ipo.cmi llvm_transform_utils.cma libllvm_passmgr_builder.a llvm_Hexagon.a llvm_PowerPC.mli llvm_XCore.cmxa llvm_ipo.cmx llvm_transform_utils.cmi libllvm_scalar_opts.a llvm_Hexagon.cma llvm_RISCV.a llvm_XCore.mli llvm_ipo.cmxa llvm_transform_utils.cmx libllvm_target.a llvm_Hexagon.cmi llvm_RISCV.cma llvm_all_backends.a llvm_ipo.mli llvm_transform_utils.cmxa libllvm_transform_utils.a llvm_Hexagon.cmx llvm_RISCV.cmi llvm_all_backends.cma llvm_irreader.a llvm_transform_utils.mli libllvm_vectorize.a llvm_Hexagon.cmxa llvm_RISCV.cmx llvm_all_backends.cmi llvm_irreader.cma llvm_vectorize.a llvm.a llvm_Hexagon.mli llvm_RISCV.cmxa llvm_all_backends.cmx llvm_irreader.cmi llvm_vectorize.cma llvm.cma llvm_Lanai.a llvm_RISCV.mli llvm_all_backends.cmxa llvm_irreader.cmx llvm_vectorize.cmi llvm.cmi llvm_Lanai.cma llvm_Sparc.a llvm_all_backends.mli llvm_irreader.cmxa llvm_vectorize.cmx llvm.cmx llvm_Lanai.cmi llvm_Sparc.cma llvm_analysis.a llvm_irreader.mli llvm_vectorize.cmxa llvm.cmxa llvm_Lanai.cmx llvm_Sparc.cmi llvm_analysis.cma llvm_linker.a llvm_vectorize.mli $ ls /opt/local/lib/ocaml/*.llvm* | xargs port provides /opt/local/lib/ocaml/META.llvm is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_AArch64 is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_AMDGPU is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_ARM is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_BPF is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_Hexagon is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_Lanai is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_MSP430 is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_Mips is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_NVPTX is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_PowerPC is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_RISCV is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_Sparc is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_SystemZ is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_WebAssembly is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_X86 is provided by: llvm-9.0 /opt/local/lib/ocaml/META.llvm_XCore is provided by: llvm-9.0
Unfortunately the logfiles that originally showed ocaml being detected during configure got overwritten. Attaching the relevant log lines (found by grep -i ocaml /opt/local/var/macports/logs/_opt_local_var_macports_*_llvm-*/*/main.log > llvm_ocaml_log_lines.txt
) anyways.
Attachments (1)
Change History (13)
Changed 5 years ago by cooljeanius (Eric Gallager)
Attachment: | llvm_ocaml_log_lines.txt added |
---|
comment:1 Changed 5 years ago by mf2k (Frank Schima)
Cc: | larryv added; jeremyhu removed |
---|---|
Owner: | set to jeremyhu |
Status: | new → assigned |
comment:2 follow-up: 3 Changed 5 years ago by kencu (Ken)
the various bindings (ocaml, go, python) that come with llvm* are detected opportunistically during a source installation, and are unfortunately not as simple to turn off as you might expect them to be. There should just be some simple cmake switches for them, and there are switches that seem related, but they don't really effectively turn the bindings off.
And things, as I recall, changed in this area at certain points in the llvm life cycle.
Last time I looked into this about 6 months ago, it was looking to take some cmake surgery to get this done. As these issues never seem to affect the buildbot versions of clang/llvm, which are built in clean environments where these go/ocaml etc bindings are never opportunistically detected, it doesn't come up for most people.
I see you're building +universal so that must be why you are seeing this, and most don't.
It is a bit of a pain, it should be fixed, upstream has declined to fix it and says things are fine as they are (in the email exchange I read about this last go round).
comment:3 Changed 5 years ago by cooljeanius (Eric Gallager)
Replying to kencu:
the various bindings (ocaml, go, python) that come with llvm* are detected opportunistically during a source installation, and are unfortunately not as simple to turn off as you might expect them to be. There should just be some simple cmake switches for them, and there are switches that seem related, but they don't really effectively turn the bindings off.
And things, as I recall, changed in this area at certain points in the llvm life cycle.
Last time I looked into this about 6 months ago, it was looking to take some cmake surgery to get this done. As these issues never seem to affect the buildbot versions of clang/llvm, which are built in clean environments where these go/ocaml etc bindings are never opportunistically detected, it doesn't come up for most people.
I see you're building +universal so that must be why you are seeing this, and most don't.
It is a bit of a pain, it should be fixed, upstream has declined to fix it and says things are fine as they are (in the email exchange I read about this last go round).
The llvm mailing lists have public archives, right? Do you have a link to where in the archives this exchange occurred that I could read?
comment:4 Changed 5 years ago by kencu (Ken)
not that good a memory :>
There are a few hooks in the llvm tree (LLVM_ENABLE_BINDINGS).
Give it a go!
comment:5 Changed 5 years ago by kencu (Ken)
There is more info about our attempts at turning the bindings on and off here #58917
comment:6 Changed 5 years ago by kencu (Ken)
Cc: | kencu added |
---|
comment:7 Changed 4 years ago by cooljeanius (Eric Gallager)
Note that this is now causing llvm-11 to fail to build for me with this error:
File "llvm_executionengine.mli", line 1: Error: /opt/local/lib/ocaml/site-lib/ctypes/ctypes.cmi is not a compiled interface for this version of OCaml. It seems to be for an older version of OCaml. make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_llvm-11/llvm-11/work/build' /Library/Developer/CommandLineTools/usr/bin/make -f lib/Target/WebAssembly/AsmParser/CMakeFiles/LLVMWebAssemblyAsmParser.dir/build.make lib/Target/WebAssembly/AsmParser/CMakeFiles/LLVMWebAssemblyAsmParser.dir/build make[2]: *** [bindings/ocaml/executionengine/llvm_executionengine.cma] Error 2 make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_llvm-11/llvm-11/work/build' make[1]: *** [bindings/ocaml/executionengine/CMakeFiles/ocaml_llvm_executionengine.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs....
port provides
says /opt/local/lib/ocaml/site-lib/ctypes/ctypes.cmi
is from ocaml-ctypes. Maybe a conflicts-build line needs to be added?
comment:8 Changed 4 years ago by kencu (Ken)
Yeah, still a PITA for those rare birds who build llvm/clang from source like you. I have no real idea what is generating your error exactly, but when I have tried to build llvm with go installed, it fails sometimes and works sometimes. I have never installed anything ocaml.
If the royal we feels that shutting down all go/ocaml/python bindings for everyone forever is the better way to go, we can conflicts build things I guess. I have no idea if anyone ever uses them for anything.
Upstream seems more-or-less-uninterested, but it's been a while since I dug into the CMAKE (again) to see if they had changed anything that might make the situation better.
patches welcome.
Here are some links you were asking about, found in a minute or two using google advanced search and this <https://www.google.ca/search?q=LLVM+ENABLE+BINDINGS+site:https://lists.llvm.org/pipermail/llvm-dev/>
https://lists.llvm.org/pipermail/llvm-dev/2016-January/093989.html
https://lists.llvm.org/pipermail/llvm-dev/2017-March/110977.html
https://lists.llvm.org/pipermail/llvm-dev/2016-January/093843.html
https://lists.llvm.org/pipermail/llvm-dev/2016-December/108382.html
https://lists.llvm.org/pipermail/llvm-dev/2016-January/093984.html
comment:9 Changed 4 years ago by cooljeanius (Eric Gallager)
Just checking back to note that the combination of disabling ocaml-ctypes plus turning on trace mode allowed my build of llvm-11 to succeed. Also, trace mode reports a bit more opportunistic port usage besides ocaml, too; after configure it says this:
Warning: The following existing files were hidden from the build system by trace mode: /opt /opt/local/bin/git /opt/local/bin/go /opt/local/bin/install_name_tool /opt/local/bin/ld /opt/local/bin/llvm-ar-mp-9.0 /opt/local/bin/llvm-ranlib-mp-9.0 /opt/local/bin/ocamlfind /opt/local/bin/pkg-config /opt/local/include/__libunwind_config.h /opt/local/include/mach-o/loader.h /opt/local/include/unwind.h /opt/local/libexec/llvm-11/bin/llvm-addr2line /opt/local/libexec/llvm-11/bin/llvm-ar /opt/local/libexec/llvm-11/bin/llvm-dlltool /opt/local/libexec/llvm-11/bin/llvm-objcopy /opt/local/libexec/llvm-11/bin/llvm-ranlib /opt/local/libexec/llvm-11/bin/llvm-readelf /private/var/select/sh /usr/X11R6
And also, after snipping the nonexistent ones, this:
Warning: The following files inside the MacPorts prefix not installed by a port were accessed: /opt/local/bin/llvm-ar /opt/local/bin/llvm-ranlib
Some additional dependencies or conflicts-build entries to add? Not sure which of them it's good to have hidden and which it'd be ok to unhide...
comment:10 Changed 4 years ago by kencu (Ken)
there is a git check along the way during the cmake configure phase, but we can ignore that. It'll find this git or that git if there is a git. Or no git.
go is opportunistically found, just like ocaml, which is what this ticket is about.
unwind.h is accessed during the build, and it'll find the one in /opt/local/include first if it exists. That's OK, rather than yet another "conflicts build" with libunwind-headers that drives everyone bananas with the gcc builds all the time.
pkg-config -- I added that dep then removed it. Not sure why it's being accessed now.
ld is ld. It'll find the first one in the path, which is the idea.
all the llvm tools are expected to be there when you're upgrading -- not sure why trace mode feels the need to hide them. But it doesn't matter one way or the other, I think.
I'm not seeing anything here that looks like it needs fixing, other than the initial headaches, ocaml and go, which upstream can't seem to fix properly.
comment:11 Changed 3 years ago by cooljeanius (Eric Gallager)
Note that the opportunistic building of ocaml bindings affects other llvm-based ports, too, such as ispc-clang, as shown in this error:
-- Installing: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_ispc/ispc-clang/work/destroot/opt/local/libexec/ispc-clang/bin/yaml2obj CMake Error at docs/cmake_install.cmake:41 (file): file INSTALL cannot find "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_ispc/ispc-clang/work/build/docs/ocamldoc/html/.": No such file or directory. Call Stack (most recent call first): cmake_install.cmake:75 (include) make: *** [install/fast] Error 1 make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_ispc/ispc-clang/work/build' Command failed: cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_ispc/ispc-clang/work/build" && /usr/bin/make -w install/fast DESTDIR=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_ispc/ispc-clang/work/destroot Exit code: 2 Error: Failed to destroot ispc-clang: command execution failed DEBUG: Error code: CHILDSTATUS 8436 2 DEBUG: Backtrace: command execution failed while executing "system {*}$notty {*}$callback {*}$nice $fullcmdstring" invoked from within "command_exec -callback portprogress::target_progress_callback destroot" (procedure "portdestroot::destroot_main" line 2) invoked from within "$procedure $targetname" Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_ispc/ispc-clang/main.log for details.
Manually creating the directory in question allows the build to proceed.
comment:12 Changed 3 years ago by kencu (Ken)
Cc: | kencu removed |
---|
relevant portions of surviving logs