MacPorts gcc gives malloc error on correct program using boost, Apple's gcc doesn't

[yusuhail@…]

Consider the following program that uses the Boost port version 1.52:

#include <boost/program_options.hpp>
#include <iostream>
using namespace std;

int main(int ac, char* av[])
{
        boost::program_options::options_description desc("Allowed options");
        desc.add_options()  ("help", "this is a help message")  ;
        cout << desc << endl;
}

If I use the mp gcc:

$ sudo port select gcc mp-gcc46
Selecting 'mp-gcc46' for 'gcc' succeeded. 'mp-gcc46' is now active.
$ g++ -L/opt/local/lib -I/opt/local/include -lboost_program_options-mt b.cpp
$ ./a.out
Allowed options:
a.out(45587) malloc: *** error for object 0x7fff70ca3500: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap

However, if I use Apple's gcc I get

$ sudo port select gcc gcc42
Selecting 'gcc42' for 'gcc' succeeded. 'gcc42' is now active.
$ g++ -L/opt/local/lib -I/opt/local/include -lboost_program_options-mt b.cpp
$ ./a.out
Allowed options:
  --help                this is a help message

[bgrupe27]

Have you build boost with mp-gcc as well?

It is known to be problematic to mix clang-built boost with newer gccs as their C++ libraries are not compatible.

[yusuhail@…]

Replying to bgrupe@…:

Have you build boost with mp-gcc as well?

I don't know. I just did

$ sudo port install boost

Currently I have

$ port installed boost
The following ports are currently installed:
  boost @1.50.0_0+no_single+no_static
  boost @1.52.0_1+no_single+no_static+python27 (active)

How do I find out the gcc a port has been built with, and how do I change that?

[seanfarley (Sean Farley)]

Replying to yusuhail@…:

Replying to bgrupe@…:

Have you build boost with mp-gcc as well?

I don't know. I just did

$ sudo port install boost

Currently I have

$ port installed boost
The following ports are currently installed:
  boost @1.50.0_0+no_single+no_static
  boost @1.52.0_1+no_single+no_static+python27 (active)

How do I find out the gcc a port has been built with, and how do I change that?

Currently, it's not easy to change the compiler for boost. I have some patches in my side project that adds all the gcc variants but won't have much time this week to work on submitting them :-(

[bgrupe27]

Replying to sean@…:

Currently, it's not easy to change the compiler for boost. I have some patches in my side project that adds all the gcc variants but won't have much time this week to work on submitting them :-(

Have you looked into the possibility to add a "boost-gcc" subport? It would be ideal to have both gcc-compiled boost (e.g. for mkvtoolnix) and clang-compiled boost active at the same time (plus a subport could easily be used as a dependency).

I know that this would add even more work though :-(

[yusuhail@…]

Replying to sean@…:

Currently, it's not easy to change the compiler for boost. I have some patches in my side project that adds all the gcc variants but won't have much time this week to work on submitting them :-(

Which compiler does it currently use? Does it depend on the "port select gcc" done beforehand?

[seanfarley (Sean Farley)]

Replying to bgrupe@…:

Replying to sean@…:

Currently, it's not easy to change the compiler for boost. I have some patches in my side project that adds all the gcc variants but won't have much time this week to work on submitting them :-(

Have you looked into the possibility to add a "boost-gcc" subport? It would be ideal to have both gcc-compiled boost (e.g. for mkvtoolnix) and clang-compiled boost active at the same time (plus a subport could easily be used as a dependency).

I have thought about it but am not sure about it yet. It would throw a wrench into any dependent port of boost that looks in ${prefix}/include (or lib) which seems (to me at least) to run against the macports design. Perhaps, the design could be changed but that would be quite a challenge. Basically, you'd need the ability to ask for a port's dependent libraries and headers.

Therefore, in keeping with the motto "try not to rock the boat," I added the extra compilers as variants. This definitely has its own problems.

I know that this would add even more work though :-(

Actually, since I wrote them as a port group, it'd initially be easy (at least for generating the subports) but would be subtle to fix the dependencies for reasons stated above

[seanfarley (Sean Farley)]

Replying to yusuhail@…:

Replying to sean@…:

Currently, it's not easy to change the compiler for boost. I have some patches in my side project that adds all the gcc variants but won't have much time this week to work on submitting them :-(

Which compiler does it currently use? Does it depend on the "port select gcc" done beforehand?

It adds variants to boost to allow selecting +gcc44 all the way to +gcc47, also +clang31 all the way to +clang33, and +dragonegg3X as well. It uses the full path, so it doesn't need 'port select' at all.

[yusuhail@…]

Replying to sean@…:

It adds variants to boost to allow selecting +gcc44 all the way to +gcc47, also +clang31 all the way to +clang33, and +dragonegg3X as well. It uses the full path, so it doesn't need 'port select' at all.

I thank you for patiently replying, but I can't seem to find any compiler variants since

$ port variants boost
boost has the variants:
   debug: Builds debug versions of the libraries as well
[+]no_single: Disable building single-threaded libraries
[+]no_static: Disable building static libraries
   openmpi: Build Boost.MPI
     * conflicts with debug
   python25: Build Boost.Python for Python 2.5
     * conflicts with debug python26 python27 python31 python32 python33
   python26: Build Boost.Python for Python 2.6
     * conflicts with debug python25 python27 python31 python32 python33
[+]python27: Build Boost.Python for Python 2.7
     * conflicts with debug python25 python26 python31 python32 python33
   python31: Build Boost.Python for Python 3.1
     * conflicts with debug openmpi python25 python26 python27 python32 python33
   python32: Build Boost.Python for Python 3.2
     * conflicts with debug openmpi python25 python26 python27 python31 python33
   python33: Build Boost.Python for Python 3.3
     * conflicts with debug openmpi python25 python26 python27 python31 python32
   regex_match_extra: Enable access to extended capture information of submatches in Boost.Regex
   universal: Build for multiple architectures

I I try to install it with a gcc variant it doesn't do anything

$ sudo port install boost +gcc46
--->  Computing dependencies for boost
--->  Cleaning boost
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.

[seanfarley (Sean Farley)]

Replying to yusuhail@…:

Replying to sean@…:

It adds variants to boost to allow selecting +gcc44 all the way to +gcc47, also +clang31 all the way to +clang33, and +dragonegg3X as well. It uses the full path, so it doesn't need 'port select' at all.

I thank you for patiently replying, but I can't seem to find any compiler variants since

$ port variants boost
boost has the variants:
   debug: Builds debug versions of the libraries as well
[+]no_single: Disable building single-threaded libraries
[+]no_static: Disable building static libraries
   openmpi: Build Boost.MPI
     * conflicts with debug
   python25: Build Boost.Python for Python 2.5
     * conflicts with debug python26 python27 python31 python32 python33
   python26: Build Boost.Python for Python 2.6
     * conflicts with debug python25 python27 python31 python32 python33
[+]python27: Build Boost.Python for Python 2.7
     * conflicts with debug python25 python26 python31 python32 python33
   python31: Build Boost.Python for Python 3.1
     * conflicts with debug openmpi python25 python26 python27 python32 python33
   python32: Build Boost.Python for Python 3.2
     * conflicts with debug openmpi python25 python26 python27 python31 python33
   python33: Build Boost.Python for Python 3.3
     * conflicts with debug openmpi python25 python26 python27 python31 python32
   regex_match_extra: Enable access to extended capture information of submatches in Boost.Regex
   universal: Build for multiple architectures

I I try to install it with a gcc variant it doesn't do anything

$ sudo port install boost +gcc46
--->  Computing dependencies for boost
--->  Cleaning boost
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.

See my previous comment, "I have some patches in my side project that adds all the gcc variants but won't have much time this week to work on submitting them :-("

[jmroot (Joshua Root)]

Please remember to Cc the maintainers. As per the ticket guidelines, the High priority is reserved for the use of MacPorts team members.

[ecronin (Eric Cronin)]

I don't believe boost is sadistic enough yet to have bjam process the headers installed, so multiple compilers could share the same $prefix/include/boost. Boost also supports versioned library names which allow you to install boost for multiple compilers into the standard location at the cost of needing -lboost_system-gcc47-mt instead of just -lboost_system. I don't know how painful it would be to make every port that depends on boost find the correct lib if we started doing this, and the work to make bjam build them is not insignificant. But then we could have a boost-headers (sub)port and boost-clang31 boost-gcc47, etc subports (here it's not just the libc++ vs libstdc++ issue, boost is so tightly coupled with the compiler used to build it you really really don't want to be using anything but exactly the same compiler with exactly the same -std= flags).

This would be a significant undertaking that adfernandes has indicated in the past he's not planning to tackle (can't blame him, I'm not signing up either) but is realistically the only way to solve this problem that keeps coming up. I wish there was a way for the boost libs to just fail completely with a useful message when there is a mismatch...

For mkvtoolnix, has anyone tried the latest mp-clang-devel? I thought it was C++11 compliant enough to build it now?

[ryandesign (Ryan Carsten Schmidt)]

Replying to sean@…:

It adds variants to boost to allow selecting +gcc44 all the way to +gcc47, also +clang31 all the way to +clang33, and +dragonegg3X as well. It uses the full path, so it doesn't need 'port select' at all.

Adding compiler variants to boost would not be an acceptable solution because it would give users the simple ability to break all existing ports using boost; I don't want it to be that easy for users to shoot themselves in the foot.

Replying to ecronin@…:

Boost also supports versioned library names which allow you to install boost for multiple compilers into the standard location at the cost of needing -lboost_system-gcc47-mt instead of just -lboost_system.

That sounds good.

I don't know how painful it would be to make every port that depends on boost find the correct lib if we started doing this, and the work to make bjam build them is not insignificant.

We wouldn't need to change most ports, just the one or two (i.e. mkvtoolnix) that need boost built with a nonstandard compiler. Actually I don't know of any other than mkvtoolnix, since any port needing that would currently be broken.

For mkvtoolnix, has anyone tried the latest mp-clang-devel? I thought it was C++11 compliant enough to build it now?

That's a discussion for #34806 or elsewhere I think.

[seanfarley (Sean Farley)]

Replying to ryandesign@…:

Replying to sean@…:

It adds variants to boost to allow selecting +gcc44 all the way to +gcc47, also +clang31 all the way to +clang33, and +dragonegg3X as well. It uses the full path, so it doesn't need 'port select' at all.

Adding compiler variants to boost would not be an acceptable solution because it would give users the simple ability to break all existing ports using boost; I don't want it to be that easy for users to shoot themselves in the foot.

You actually already have this because of the +openmpi variant. A user can just,

$ port install openmpi +gcc47 $ port install boost +openmpi

I think this is, unfortunately, the price to pay for wanting to use something that depends on boost :-/ If the whole variant dependence issue was solved, that would mitigate some of these issues, but that's a whole other issue (get it? issue? hah!).

[ecronin (Eric Cronin)]

Replying to sean@…:

Replying to ryandesign@…:

Replying to sean@…:

It adds variants to boost to allow selecting +gcc44 all the way to +gcc47, also +clang31 all the way to +clang33, and +dragonegg3X as well. It uses the full path, so it doesn't need 'port select' at all.

Adding compiler variants to boost would not be an acceptable solution because it would give users the simple ability to break all existing ports using boost; I don't want it to be that easy for users to shoot themselves in the foot.

You actually already have this because of the +openmpi variant. A user can just,

$ port install openmpi +gcc47 $ port install boost +openmpi

I think this is, unfortunately, the price to pay for wanting to use something that depends on boost :-/ If the whole variant dependence issue was solved, that would mitigate some of these issues, but that's a whole other issue (get it? issue? hah!).

boost +openmpi was probably added before +gcc45 was a default variant to openmpi, but even back then foot shooting was possible as you noted. Today I think it's all but guaranteed since installing boost +openmpi without a version of openmpi already installed will default to be broken. It should probably at least be using the active_variants group to ensure that things are ok at build/activate, but I don't think we can prevent issues after the fact from activating a different openmpi (which is why non-conflicting subports are better than variants)

[seanfarley (Sean Farley)]

Replying to ecronin@…:

Replying to sean@…:

Replying to ryandesign@…:

Replying to sean@…:

It adds variants to boost to allow selecting +gcc44 all the way to +gcc47, also +clang31 all the way to +clang33, and +dragonegg3X as well. It uses the full path, so it doesn't need 'port select' at all.

Adding compiler variants to boost would not be an acceptable solution because it would give users the simple ability to break all existing ports using boost; I don't want it to be that easy for users to shoot themselves in the foot.

You actually already have this because of the +openmpi variant. A user can just,

$ port install openmpi +gcc47 $ port install boost +openmpi

I think this is, unfortunately, the price to pay for wanting to use something that depends on boost :-/ If the whole variant dependence issue was solved, that would mitigate some of these issues, but that's a whole other issue (get it? issue? hah!).

boost +openmpi was probably added before +gcc45 was a default variant to openmpi, but even back then foot shooting was possible as you noted. Today I think it's all but guaranteed since installing boost +openmpi without a version of openmpi already installed will default to be broken. It should probably at least be using the active_variants group to ensure that things are ok at build/activate, but I don't think we can prevent issues after the fact from activating a different openmpi (which is why non-conflicting subports are better than variants)

The problems the active_variants approach (especially with compiler wrappers) is which variant to use? +gcc44, 45, 46, 47? +clang31, 32, 33? etc. Subports would iron out this dependency issue but introduce a drastic change in core (i.e. need a way to get a subport's libs / headers).

[adfernandes (Andrew Fernandes)]

Replying to ecronin@…:

I don't believe boost is sadistic enough yet to have bjam process the headers installed, so multiple compilers could share the same $prefix/include/boost. Boost also supports versioned library names which allow you to install boost for multiple compilers into the standard location at the cost of needing -lboost_system-gcc47-mt instead of just -lboost_system.

This is correct. The major problem with boost is that it has one-header-for-all that changes all sorts of interface assumptions about how the libraries were built, and it's terrible with encapsulation. Basically, this means that the compiler that processes the boost headers needs to be the same compiler that built the libraries. That's why they added the brain-dead -lboost_system-gcc47-mt stuff.

MPI is a different issue. The openmpi headers encapsulate the interfaces quite well, so you can do things like mix dylibs compiled with different compilers. As long as the headers correctly specify the (constant) interfaces to the dylibs, it doesn't matter what compiler creates the dylibs. (There are side-issues with this, such as people using different c++ std libraries by accident, but those are side issues.)

Basically, my experiences with boost are that it really his horrible to use as a system library if you have multiple compilers. It was designed and tested in one configuration - the one that uses the system compiler. The "one system compiler to rule them all" is kind-of an old-time concept... but that's what they do.

Any variants of boost that specify the compiler would just introduce a reverse-bug; all projects using the boost libraries would then need to use the same compiler... :-)

By the way - I just ran the code sample in the report using system gcc-4.2.1, system clang and gcc47 from macports, and all three compiled and ran without problems, so marking as worksforme (OS 10.8.2, latest Xcode tools, etc.)

[yusuhail@…]

Does it work for anyone else? Is there any way to know why it works for adfernandes but not for me?

[neverpanic (Clemens Lang)]

Works for me, too. 10.8.2, latest Xcode, also with system clang and gcc and gcc47 from macports.

[adfernandes (Andrew Fernandes)]

Replying to yusuhail@…:

Does it work for anyone else? Is there any way to know why it works for adfernandes but not for me?

Hard to tell. The type of error being produced makes me think that somewhere, somehow, the wrong c++ standard library is being used.

Try force-reinstalling gcc46. Or, what might be better, is try comping and running with gcc47.

It also depends what version of Mac OS you're running. I think boost tests 10.7 and 10.8, but 10.6 and earlier probably are buggy (not sure about the exact versions supported, etc. though.)