virtualbox: build failure when curl is not installed universal

[mf2k (Frank Schima)]

I see the following error building virtualbox on Mac OS X 10.6.8.

/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTDirQueryInfo-generic.cpp
:info:build /opt/local/include/curl/curlrules.h:143: error: size of array ‘__curl_rule_01__’ is negative
:info:build /opt/local/include/curl/curlrules.h:153: error: size of array ‘__curl_rule_02__’ is negative
:info:build kBuild: Compiling VBoxRT - /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTDirSetTimes-generic.cpp
:info:build kBuild: Compiling VBoxRT - /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTFileMove-generic.cpp
:info:build kBuild: Compiling VBoxRT - /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTLogWriteDebugger-generic.cpp
:info:build kBuild: Compiling VBoxRT - /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTProcDaemonize-generic.cpp
:info:build kBuild: Compiling VBoxRT - /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTThreadGetAffinity-stub-generic.cpp
:info:build kBuild: Compiling VBoxRT - /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTThreadSetAffinity-stub-generic.cpp
:info:build kBuild: Compiling VBoxRT - /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTThreadSetAffinityToCpu-generic.cpp
:info:build kBuild: Compiling VBoxRT - /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/generic/RTTimeLocalNow-generic.cpp
:info:build kmk: *** [/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/out/darwin.x86/release/obj/VBoxRT/common/misc/s3.o] Error 1
:info:build The failing command:
:info:build @g++-4.0 -c -O2 -g -pipe -pedantic -Wshadow -Wall -Wextra -Wno-missing-field-initializers -Wno-unused -Wno-trigraphs  -Wno-long-long -Wno-variadic-macros -O2 -mtune=generic -fno-omit-frame-pointer -fno-strict-aliasing -fvisibility=hidden -DVBOX_HAVE_VISIBILITY_HIDDEN -DRT_USE_VISIBILITY_DEFAULT -fvisibility-inlines-hidden -Werror -mmacosx-version-min=10.6 -isysroot /Developer/SDKs/MacOSX10.6.sdk -m32 -I/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/include -I/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/libs/liblzf-3.4 -I/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/libs/kStuff/kStuff/include -I/opt/local/include -I/opt/local/include/libxml2 -I/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/libs/boost-1.37.0 -I/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/include -I/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/out/darwin.x86/release -DVBOX -DVBOX_WITH_DEBUGGER -DVBOX_OSE -DVBOX_WITH_64_BITS_GUESTS -DRT_OS_DARWIN -D__DARWIN__ -DRT_ARCH_X86 -D__X86__ -D_REENTRANT -DBOOST_DISABLE_ASSERTS -DBUILDING_LIBCURL -DIN_RING3 -DHC_ARCH_BITS=32 -DGC_ARCH_BITS=64 -DIN_RING3 -DHC_ARCH_BITS=32 -DGC_ARCH_BITS=64 -DIN_RT_R3 -DIN_SUP_R3 -DLDR_WITH_NATIVE -DLDR_WITH_ELF32 -DLDR_WITH_PE -DRT_WITH_VBOX -DLDR_WITH_KLDR -DRT_WITH_ICONV_CACHE -DIN_SUP_R3 -DIN_SUP_R3 -DMAC_OS_X_VERSION_MIN_REQUIRED=1060 -DMAC_OS_X_VERSION_MAX_ALLOWED=1060 -DPIC -Wp,-MD,/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/out/darwin.x86/release/obj/VBoxRT/common/misc/s3.o.dep -Wp,-MT,/opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/out/darwin.x86/release/obj/VBoxRT/common/misc/s3.o -Wp,-MP -o /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/out/darwin.x86/release/obj/VBoxRT/common/misc/s3.o /opt/local/var/macports/build/_opt_mports_trunk_dports_emulators_virtualbox/virtualbox/work/VirtualBox-4.1.2_OSE/src/VBox/Runtime/common/misc/s3.cpp
:info:build kmk: *** Waiting for unfinished jobs....
:info:build kmk: *** Exiting with status 2

[ryandesign (Ryan Carsten Schmidt)]

Replying to macsforever2000@…:

:info:build /opt/local/include/curl/curlrules.h:143: error: size of array ‘__curl_rule_01__’ is negative
:info:build /opt/local/include/curl/curlrules.h:153: error: size of array ‘__curl_rule_02__’ is negative

Looks like virtualbox has an undeclared dependency on curl. Is curl installed with the same archs for which you're building virtualbox?

These lines reference CURL_SIZEOF_LONG and CURL_SIZEOF_CURL_OFF_T. If the archs of curl and virtualbox match, can you try rebuilding curl anyway?

[mf2k (Frank Schima)]

$ port installed curl
The following ports are currently installed:
  curl @7.21.7_0+ssl (active)

I'm not installing with universal. Reinstalling curl did not change the error.

[carsomyr@…]

What happens if you write a dummy program and include curlrules.h? I'm wondering if it's a VirtualBox problem in the first place.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign@…:

Looks like virtualbox has an undeclared dependency on curl.

r83164.

[ryandesign (Ryan Carsten Schmidt)]

Replying to macsforever2000@…:

I'm not installing with universal. Reinstalling curl did not change the error.

I am able to reproduce this problem when curl @7.21.7_0+ssl is active, but virtualbox builds fine when using curl @7.21.7_0+ssl+universal.

[ryandesign (Ryan Carsten Schmidt)]

Rebuilding curl universal is also the workaround to this issue offered on the ​virtualbox forums.

[ryandesign (Ryan Carsten Schmidt)]

virtualbox is not the only software experiencing this situation; rebuilding curl universal is also suggested to ​build curb. So it's sounding like a curl bug. Looking at the diff between the x86_64-only curl install and the universal one, I don't understand the problem, however.

[ryandesign (Ryan Carsten Schmidt)]

Ok. Not a curl bug. "size of array ‘__curl_rule_01__’ is negative" is curl's cute way of saying the architecture doesn't match. Turns out virtualbox builds for i386 only. So I've fixed that in the Portfile in r83169 so now this will force curl to rebuild to include i386.

[carsomyr@…]

I am against the change of forcing VirtualBox to build only i386, feeling that it is heavy-handed. The problem with cURL seems to be an interaction of Snow Leopard with cURL and not with downstream Ports. By forcing the i386 variant, that essentially means that all dependencies (I'm looking at you, qt4-mac) are forced to build universal on 64-bit kernels. Taking a long time aside, building universal for so many dependencies is a potentially fertile source for more problems. After Googling extensively, I made the following observations and inferences:

1. The problem only arose on Snow Leopard.
2. Snow Leopard is a transitionary OS from 32-bit to 64-bit.
3. The word length mismatch arises from LP64 being defined and CURL_SIZEOF_LONG set to a value not matching sizeof(long).
4. Inference: Those building on 32-bit Snow Leopard are seeing this problem.

Actually, I think the fix may be quite simple: Take out the offending sanity checks in the cURL Port and maybe file a bug with upstream. I say may because the sanity checks might be preventing illegal operations from happening.

[ryandesign (Ryan Carsten Schmidt)]

Replying to carsomyr@…:

I am against the change of forcing VirtualBox to build only i386, feeling that it is heavy-handed.

I did not force virtualbox to build i386. Virtualbox forces itself to build i386. I merely updated the portfile to correspond to the reality of how virtualbox builds itself. Requests to have a 64-bit build of virtualbox must be met by the developers of virtualbox, unless there's a configuration switch they've already built in to do that, but I doubt it, since they're deliberately ignoring and overwriting MacPorts' existing request to build 64-bit (which happens by setting -arch flags in CFLAGS, CPPFLAGS, and LDFLAGS, which works on most other software).

The problem with cURL seems to be an interaction of Snow Leopard with cURL and not with downstream Ports.

Only because as of Snow Leopard the compiler builds 64-bit by default instead of 32-bit as before.

By forcing the i386 variant, that essentially means that all dependencies (I'm looking at you, qt4-mac) are forced to build universal on 64-bit kernels.

The bitness of your kernel is irrelevant to everything except what bitness of kernel extensions you can run. Apple produced years worth of 64-bit Intel Macs that boot to a 32-bit kernel, which again is no detriment to being able to run 64-bit software on those machines.

Taking a long time aside, building universal for so many dependencies is a potentially fertile source for more problems.

If you find problems building ports universal (or any other manner), file bug reports. I have been building ports universal for two years, since Snow Leopard came out; the dependencies of virtualbox build universal without problems.

[carsomyr@…]

Replying to ryandesign@…:

Replying to carsomyr@…:

I am against the change of forcing VirtualBox to build only i386, feeling that it is heavy-handed.

I did not force virtualbox to build i386. Virtualbox forces itself to build i386. I merely updated the portfile to correspond to the reality of how virtualbox builds itself. Requests to have a 64-bit build of virtualbox must be met by the developers of virtualbox, unless there's a configuration switch they've already built in to do that, but I doubt it, since they're deliberately ignoring and overwriting MacPorts' existing request to build 64-bit (which happens by setting -arch flags in CFLAGS, CPPFLAGS, and LDFLAGS, which works on most other software).

The VirtualBox binaries are showing up as 64-bit for me (see the attachment "word_size"). Perhaps you mean not so much as forcing i386 as building whatever the default architecture is?

The problem with cURL seems to be an interaction of Snow Leopard with cURL and not with downstream Ports.

Only because as of Snow Leopard the compiler builds 64-bit by default instead of 32-bit as before.

By forcing the i386 variant, that essentially means that all dependencies (I'm looking at you, qt4-mac) are forced to build universal on 64-bit kernels.

The bitness of your kernel is irrelevant to everything except what bitness of kernel extensions you can run. Apple produced years worth of 64-bit Intel Macs that boot to a 32-bit kernel, which again is no detriment to being able to run 64-bit software on those machines.

VirtualBox was been building fine for me on 64-bit Snow Leopard. If it's not bitness, then what would you identify as the cause of the build failure?

Taking a long time aside, building universal for so many dependencies is a potentially fertile source for more problems.

If you find problems building ports universal (or any other manner), file bug reports. I have been building ports universal for two years, since Snow Leopard came out; the dependencies of virtualbox build universal without problems.

[carsomyr@…]

A listing of the word sizes for VirtualBox artifacts.

[carsomyr@…]

Another issue that I foresee is that 32-bit kernel extensions will be generated in 64-bit kernels. That effectively breaks VirtualBox for anyone running a 64-bit kernel.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign@…:

The bitness of your kernel is irrelevant to everything except what bitness of kernel extensions you can run. Apple produced years worth of 64-bit Intel Macs that boot to a 32-bit kernel, which again is no detriment to being able to run 64-bit software on those machines.

What I wrote here is true, but I failed to realize that:

1. virtualbox includes kernel extensions, and
2. virtualbox's configure script runs uname -m to determine whether your kernel is 32-bit or 64-bit, and adopts the same architecture when building the rest of virtualbox

This is not a behavior I believe we have encountered before in MacPorts so we don't have anything prepared for dealing with it, but I'll see what I can do.

[ryandesign (Ryan Carsten Schmidt)]

I'm testing whether simply using

supported_archs     [exec uname -m]

would work.

[carsomyr@…]

Just to be conceptually clear, what is the sequence of events that leads to the build failing on cURL headers on 32-bit Snow Leopard with 64-bit compilation?

[ryandesign (Ryan Carsten Schmidt)]

AFAICT, the failure occurs when running Snow Leopard (for which the default compilation architecture is 64-bit) on a 64-bit Mac that uses the 32-bit kernel (like my MacBookPro3,1). curl and all other ports that can be built 64-bit will have been built 64-bit, but because virtualbox does things special, it wants to build 32-bit on 32-bit kernels. Again, the curl error message should be translated as: "the architecture for which you built curl does not match the architecture for which you're building whatever you're building now" (i.e. virtualbox).

[carsomyr@…]

I see the

#define CURL_SIZEOF_LONG 8

now. That's surely going to cause problems with the sanity check when those headers are imported and sizeof(long) doesn't match.

[ryandesign (Ryan Carsten Schmidt)]

The header says "#define CURL_SIZEOF_LONG 8" because curl is built 64-bit. That will be fine when building virtualbox on 64-bit kernels. For users like me with 32-bit kernels, curl will have to be recompiled universal. The header will then contain an #ifdef to define CURL_SIZEOF_LONG correctly depending on architecture.

[carsomyr@…]

It seems that the supported_archs fix should work, given what's been deduced.

[ryandesign (Ryan Carsten Schmidt)]

r83173.