py37-zmq @17.1.2: Build failure on OSX PPC: can't locate installed zmq

[SerpentChris (Chris Calderon)]

It looks like the project tries to run it's own script to locate the installed zmq library, and failing that it defaults to a version it has bundled with it's own source code. I'm building this on a Power Mac G5 running Mac OS X 10.5.8.

[kencu (Ken)]

These two build lines fail:

895	:info:build /usr/bin/gcc-4.2 -DNDEBUG -g -fwrapv -O3 -Wall -arch ppc -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.macosx-10.5-ppc-3.7/scratch/vers.c -o build/temp.macosx-10.5-ppc-3.7/scratch/vers.o
896	:info:build /usr/bin/gcc-4.2 -arch ppc -undefined dynamic_lookup -Wl,-rpath -Wl,/opt/local/lib build/temp.macosx-10.5-ppc-3.7/scratch/vers.o -L/opt/local/lib -lzmq -o build/temp.macosx-10.5-ppc-3.7/scratch/vers
897	:info:build Error running version detection script:
898	:info:build error: Error running version detection script:

I can't see what the actual error is, though. Might have to go into the build directory and run those manually to see what the issue is.

If it builds the vers executable, try running it and see what happens...

[michaelld (Michael Dickens)]

I can't replicate this issue on -any- Intel OSX, so I'm tweaking the title to be just for PPC.

[michaelld (Michael Dickens)]

This issue seems to be with the whole py3*-zmq series, not just 3.7. This issue is not present with the py27-zmq. Seems to be very specific to Py3 and PPC.

[SerpentChris (Chris Calderon)]

Update: I tried downloading the pyzmq source code off github, and compiling it that way. The problem is that when it tests the installed library, it compiles a C file into an executable, but when I run the executable it says "Illegal Instruction" and exits with code 132. I don't fully understand why it takes two steps to build the executable though. Here are the lines:

/usr/bin/gcc-4.2 -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -arch ppc -I/opt/local/include -Izmq/utils -Izmq/backend/cython -Izmq/devices -c buildutils/vers.c -o vers.o
/usr/bin/gcc-4.2 -arch ppc -undefined dynamic_lookup -Wl,-rpath -Wl,/opt/local/lib vers.o -L/opt/local/lib -lzmq -o vers

I tried making my own very similar C program and running it and I got the same error. Here are the contents:

#include <stdio.h>
#include <zmq.h>

int
main(int argc, char *argv[])
{
  int major, minor, patch;
  zmq_version(&major, &minor, &patch);
  printf("Current ZMQ version is %d.%d.%d\n", major, minor, patch);
  return 0;
}

I compiled this:

gcc-4.2 -I/opt/local/include -L/opt/local/lib -lzmq zmq_vers.c -o zmq_vers

I ran the resulting executable and got the same error, Illegal Instruction. This is all on my Power Mac G5 running OS X 10.5.8 btw.

[SerpentChris (Chris Calderon)]

I do notice that when I install zmq, there is a warning: "C++11 ports are compiling with GCC. EXPERIMENTAL."

[kencu (Ken)]

Michael says the py27 series works; is it an option for you to use that while we sort this out?

[SerpentChris (Chris Calderon)]

I'm actually getting the same result if I try to install py27-zmq.

[kencu (Ken)]

zmq 4.25 works OK:

$ cat zmq_vers.c
#include <stdio.h>
#include <zmq.h>

int
main(int argc, char *argv[])
{
  int major, minor, patch;
  zmq_version(&major, &minor, &patch);
  printf("Current ZMQ version is %d.%d.%d\n", major, minor, patch);
  return 0;
}

$gcc-4.2 -I/opt/local/include -L/opt/local/lib -lzmq zmq_vers.c -o zmq_vers

$ ./zmq_vers
Current ZMQ version is 4.2.5

I'll upgrade that machine to zmq 4.3.0 and see what happens when I try that.

[kencu (Ken)]

Yep, zmq @4.3.0 currently seems well and truly broken on 10.5 PPC, at least. I get the same result -- "Illegal Instruction". Investigating. For now, your only option is to roll back to @4.2.5:

sudo port -v activate zmq

and pick @4.2.5 from the list.

We'll keep this ticket updated if it turns out to be fixable.

[kencu (Ken)]

In the Portfile, there is a bit added on to end of it for some special PPC building:

# special building for PPC/PPC64 for zmq and zmq-devel

if {(${subport} eq "zmq" || ${subport} eq "zmq-devel") &&
    (${build_arch} eq "ppc" || ${build_arch} eq "ppc64")} {

    # force c++11
    PortGroup cxx11 1.1

    configure.args-replace --disable-cxx11 --enable-cxx11
}

I'm not sure why that bit is there, but if I delete that block, zmq builds through to completion, and at least delivers up it's version correctly.

$ gcc-4.2 -I/opt/local/include -L/opt/local/lib -lzmq zmq_vers.c -o zmq_vers
LeopardG5:~ cunningh$ ./zmq_vers
Current ZMQ version is 4.3.0

[kencu (Ken)]

Here is the error and gdb run with the stock version, built with that block in:

$ ./zmq_vers
Illegal instruction

$ gdb ./zmq_vers
GNU gdb 6.3.50-20050815 (Apple version gdb-967) (Tue Jul 14 02:15:14 UTC 2009)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...Reading symbols for shared libraries .... done

(gdb) run
Starting program: /Users/cunningh/zmq_vers 
Reading symbols for shared libraries +++.. done

Program received signal EXC_BAD_INSTRUCTION, Illegal instruction/operand.
0x0008d034 in _mh_dylib_header ()
(gdb) bt
#0  0x0008d034 in _mh_dylib_header ()
#1  0x000b41e0 in std::_Rb_tree<std::string, std::pair<std::string const, zmq::own_t* (zmq::session_base_t::*)(zmq::io_thread_t*, bool)>, std::_Select1st<std::pair<std::string const, zmq::own_t* (zmq::session_base_t::*)(zmq::io_thread_t*, bool)> >, std::less<std::string>, std::allocator<std::pair<std::string const, zmq::own_t* (zmq::session_base_t::*)(zmq::io_thread_t*, bool)> > >::_M_insert_unique<std::pair<std::string const, zmq::own_t* (zmq::session_base_t::*)(zmq::io_thread_t*, bool)>*> ()
#2  0x000edbfc in _GLOBAL__sub_I_session_base.cpp ()
#3  0x8fe13834 in __dyld__ZN16ImageLoaderMachO18doModInitFunctionsERKN11ImageLoader11LinkContextE ()
#4  0x8fe0f248 in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEj ()
#5  0x8fe0f198 in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEj ()
#6  0x8fe0f36c in __dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextE ()
#7  0x8fe03848 in __dyld__ZN4dyld24initializeMainExecutableEv ()
#8  0x8fe08144 in __dyld__ZN4dyld5_mainEPK11mach_headermiPPKcS5_S5_ ()
#9  0x8fe01774 in __dyld__ZN13dyldbootstrap5startEPK11mach_headeriPPKcl ()
#10 0x8fe01048 in __dyld__dyld_start ()
(gdb) quit
The program is running.  Exit anyway? (y or n) y

I have no idea at this moment what is going on to cause this illegal instruction error when zmq is build with gcc6 in -std=c++11 mode, but whatever it is goes away when zmq is built with gcc-4.2 in non-c++11 mode.

Now we have to see why that block forcing gcc6 / c++11 for PPC is in the Portfile in the first place...

[kencu (Ken)]

Michael -- I don't know the slightest thing about zmq -- but I can confirm that with the block forcing gcc6/c++11 removed, both zmq and zmq-devel build and don't crash when running the version test program above.

I see it was put in a few months ago to fix a build failure noted at that time. The log is no longer available on the buildbot, so I don't know what was happening back then, but it seems to be gone (for now .... ).

Suggest we disable the force, and revbump (only the PPC version needs a revbump, if you want to do it like that).

[michaelld (Michael Dickens)]

Interesting. Here's the commit: ​https://github.com/macports/macports-ports/commit/c0c1c89f851334207b481fcaa58329ac5e8247e1 . Let me test tomorrow at work (where my PPC box is), but it seems like the ZMQ folks might have fixed whatever the issue was (regardless of intentionality). I'd definitely prefer to not require C++11 if it's not required! It -is- pretty strange that using C++11 results in crashes; that's probably a compiler issue!

[michaelld (Michael Dickens)]

OK well it took me more than a day to get to this point ... fighting the !@#$ internet at my shared workspace. When I remove per Ken's advice and reinstall zmq (or zmq-devel), then py*-zmq does indeed install cleanly. I wonder what the difference is in what ZMQ installs when C++11 is forced versus when it is not ... maybe ports using ZMQ are also required to use C++11? Anyway, I'll fix up the ZMQ Portfile with this fix shortly.

[kencu (Ken)]

Replying to michaelld:

I wonder what the difference is in what ZMQ installs when C++11 is forced versus when it is not ... maybe ports using ZMQ are also required to use C++11?

Worse than that, I think -- some kind of full crashing bug when it's built with gcc6. Usually these things are optimization related, but ???

[michaelld (Michael Dickens)]

In 351847800b26a7d34c122c43f86f3e7647678747/macports-ports (master):

zmq[-devel]: remove "special building for PPC/PPC64"

+ although ZMQ builds forcing C++11, some dependent ports do not;
+ forcing C++11 is no longer necessary due to code changes from upstream;
+ rev-bump zmq and zmq-devel for possible change.

Closes: #57724