Boost.Regex with GCC broken on Mavericks?

[akimd (Akim Demaille)]

Hi,

First of all, this seems to be strongly related to this ticket: #38984.

On Mavericks (with a fresh install of MacPorts, not a plain upgrade), I have the following behavior which turns out to be very alike the one I had reported in the aforementioned ticket, except that this time GCC is broken, but Clang is not:

akim@padam /tmp $ port installed | grep gcc
  gcc43 @4.3.6_9 (active)
  gcc44 @4.4.7_8 (active)
  gcc45 @4.5.4_9 (active)
  gcc46 @4.6.4_3 (active)
  gcc47 @4.7.3_3 (active)
  gcc48 @4.8.2_0 (active)
  gcc49 @4.9-20131117_0
  gcc49 @4.9-20131124_0 (active)
  gcc_select @0.1_8 (active)
  libgcc-devel @4.9-20131117_0
  libgcc-devel @4.9-20131124_0 (active)
  libgcc45 @4.5.4_9 (active)
akim@padam /tmp $ cat foo.cc
#include <boost/regex.hpp>

int
main ()
{
  boost::regex exp("*");
  boost::regex_match("foo", exp);
  return 0;
}
akim@padam /tmp $ g++-mp-4.8 foo.cc -L /opt/local/lib -I /opt/local/include -lboost_regex-mt
Undefined symbols for architecture x86_64:
  "boost::re_detail::perl_matcher<char const*, std::allocator<boost::sub_match<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::construct_init(boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags)", referenced from:
      boost::re_detail::perl_matcher<char const*, std::allocator<boost::sub_match<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::perl_matcher(char const*, char const*, boost::match_results<char const*, std::allocator<boost::sub_match<char const*> > >&, boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags, char const*) in ccaSC2nT.o
  "boost::re_detail::perl_matcher<char const*, std::allocator<boost::sub_match<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::match()", referenced from:
      bool boost::regex_match<char const*, std::allocator<boost::sub_match<char const*> >, char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >(char const*, char const*, boost::match_results<char const*, std::allocator<boost::sub_match<char const*> > >&, boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags) in ccaSC2nT.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
akim@padam /tmp $ clang++ foo.cc -L /opt/local/lib -I /opt/local/include -lboost_regex-mt
akim@padam /tmp $

[ryandesign (Ryan Carsten Schmidt)]

You cannot mix C++ software compiled with libstdc++ (i.e. anything compiled with FSF GCC, i.e. your foo.cc) with C++ software compiled with libc++ (i.e. boost and most other ports compiled using clang, llvm-gcc, or Apple GCC on Mavericks or later). Compile your foo.cc with clang instead of FSF GCC.

[akimd (Akim Demaille)]

Hi Ryan,

If I read you correctly, we are now forced to use clang on Mac OS X, instead of GCC? What can be done so that GCC still be usable on Mac OS X? How about providing a means to compile and install libraries for GCC in parallel to those of Clang?

[ryandesign (Ryan Carsten Schmidt)]

You're not forced to use anything, but Apple has decided with Mavericks to switch the system's C++ library from its previous custom version of libstdc++ (which was subtly incompatible with FSF GCC's libstdc++) to libc++ (which is utterly incompatible with FSF GCC's libstdc++). Software compiled with MacPorts simply follows the system's convention.

If you absolutely must use FSF GCC to compile C++ code that uses other C++ code from MacPorts, then I can only suggest to uninstall all ports, install MacPorts trunk (MacPorts 2.2.x does not contain the following setting), and set 'cxx_stdlib libstdc++' in ${prefix}/etc/macports/macports.conf, and then reinstall ports. I have no idea if this will work, and it's probably not a good idea to try; you're going from the supported configuration that everyone else is using on Mavericks to an unsupported configuration that probably nobody else has tried so you're bound to run into lots of problems.

Mavericks is designed for libc++; my personal recommendation is to either use libc++ or don't use Mavericks.

[cooljeanius (Eric Gallager)]

Cc Me!

[akimd (Akim Demaille)]

Hi Ryan,

Thanks for clarifying the issue to me.

I do not want to try to compromise the stability of this Mavericks machine by attempting to substitute its native libc++, I was asking wether it would be possible to introduce support for both C++ compilers, possibly with a different /opt/local/lib for libstdc++.

[ryandesign (Ryan Carsten Schmidt)]

If you don't wish to make these changes in your primary MacPorts installation in /opt/local, you could maintain a second MacPorts installation in a different prefix (i.e. compile MacPorts from source with ./configure --prefix=/somewhere/else), and set its cxx_stdlib to libstdc++ as above, but as I said, you will probably run into problems with that MacPorts installation by doing so.

[akimd (Akim Demaille)]

Well, this is really a pity. I wonder how the GNU/Linux distros handle this issue when they deploy Boost.

Is the following issue related to the current discussion? That's weird, since this machine is a Mountain Lion, so it is my understanding that it should work properly with GCC, and less easily with Clang:

$ cat /tmp/foo.cc
#include <boost/filesystem.hpp>
#include <string>

int main()
{
  boost::filesystem::path directory("abc");
  return directory.is_absolute();
}
$ g++-mp-4.9 -Wall foo.cc -lboost_filesystem-mt -lboost_system-mt -L/opt/local/lib -I/opt/local/include
$ ./a.out
a.out(16530) malloc: *** error for object 0x109283570: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
zsh: abort      ./a.out
$ clang++-mp-3.4 -Wall foo.cc -lboost_filesystem-mt -lboost_system-mt -L/opt/local/lib -I/opt/local/include
$ ./a.out
$ otool -L /opt/local/lib/libboost_system-mt.dylib
/opt/local/lib/libboost_system-mt.dylib:
	/opt/local/lib/libboost_system-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 56.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

[ryandesign (Ryan Carsten Schmidt)]

I don't know if that's the same problem. But it could be. Per my remarks above, the Apple custom version libstdc++ that's in use by default on Mountain Lion and earlier is not the same libstdc++ that's in use with FSF GCC, and they are subtly incompatible with each other which can lead to problems. Don't mix and match them. You're using Boost's C++ code, which, if you've installed the boost port the normal way with MacPorts, has been compiled with the system compiler and is therefore linked with the system's libstdc++. Therefore, compile your own code using the system's libstdc++ too; don't use FSF GCC's libstdc++.

[akimd (Akim Demaille)]

Replying to ryandesign@…:

Therefore, compile your own code using the system's libstdc++ too; don't use FSF GCC's libstdc++.

I'd be happy to do that, but as of today, this is not possible on Mac Ports, because the system's lib for C++ is libstdc++, but if I don't pass -stdlib=c++, then c++ (the native C++ compiler) does not support C++11. So:

• GCC is not an option, as it does not use the system's library:

  $ cat /tmp/foo.cc
  #include <memory>
  #include <boost/filesystem.hpp>
  #include <iostream>
  
  int main()
  {
    auto p = std::make_shared<boost::filesystem::path>("abc");
    std::cerr << p->is_absolute() << std::endl;
    std::cerr << absolute(*p) << std::endl;
  }
  $ g++-mp-4.8 -std=c++11 -ggdb -Wall foo.cc -lboost_filesystem-mt -lboost_system-mt -L/opt/local/lib -I/opt/local/include
  $ ./a.out
  a.out(82426) malloc: *** error for object 0x108bc4570: pointer being freed was not allocated
  *** set a breakpoint in malloc_error_break to debug
  zsh: abort      ./a.out
  

• clang is not an option, where from Xcode or from MacPorts, because its system's library does not support C++11

  $ c++ -std=c++11 -ggdb -Wall foo.cc -lboost_filesystem-mt -lboost_system-mt -L/opt/local/lib -I/opt/local/include
  foo.cc:7:17: error: no member named 'make_shared' in namespace 'std'
    auto p = std::make_shared<boost::filesystem::path>("abc");
             ~~~~~^
  foo.cc:7:52: error: expected '(' for function-style cast or type construction
    auto p = std::make_shared<boost::filesystem::path>("abc");
                              ~~~~~~~~~~~~~~~~~~~~~~~^
  2 errors generated.
  $ c++ -std=c++11 -stdlib=libc++ -ggdb -Wall foo.cc -lboost_filesystem-mt -lboost_system-mt -L/opt/local/lib -I/opt/local/include
  $ ./a.out
  zsh: segmentation fault  ./a.out
  

So, unless I missed an option, one cannot use Boost and C++11 on MacPorts today.

[ryandesign (Ryan Carsten Schmidt)]

Replying to akim.demaille@…:

• clang is not an option, where from Xcode or from MacPorts, because its system's library does not support C++11

  $ c++ -std=c++11 -ggdb -Wall foo.cc -lboost_filesystem-mt -lboost_system-mt -L/opt/local/lib -I/opt/local/include
  foo.cc:7:17: error: no member named 'make_shared' in namespace 'std'
    auto p = std::make_shared<boost::filesystem::path>("abc");
             ~~~~~^
  foo.cc:7:52: error: expected '(' for function-style cast or type construction
    auto p = std::make_shared<boost::filesystem::path>("abc");
                              ~~~~~~~~~~~~~~~~~~~~~~~^
  2 errors generated.
  

This works fine for me.

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.9
BuildVersion:	13A603
$ c++ -v
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix
$ c++ -std=c++11 -ggdb -Wall foo.cc -lboost_filesystem-mt -lboost_system-mt -L/opt/local/lib -I/opt/local/include
$ ./a.out
0
"/private/tmp/abc"

$ c++ -std=c++11 -stdlib=libc++ -ggdb -Wall foo.cc -lboost_filesystem-mt -lboost_system-mt -L/opt/local/lib -I/opt/local/include
$ ./a.out
zsh: segmentation fault  ./a.out

As does this, which it should, since all you've added is "-stdlib=libc++" which is the default on Mavericks anyway.

This makes me think your c++ is not Apple LLVM version 5.0.

[akimd (Akim Demaille)]

Replying to ryandesign@…:

This works fine for me. […] As does this, which it should, since all you've added is "-stdlib=libc++" which is the default on Mavericks anyway.

This makes me think your c++ is not Apple LLVM version 5.0.

Hi Ryan,

My bad, I should have repeated that, as said earlier in this "thread", these last examples are about a Moutain Lion machine. My latest comments as "impossible to Boost in C++11 in MacPorts" is about Mountain Lion. I do expect this to work on the other, Mavericks, machine, which I don't have access to right now.

[ryandesign (Ryan Carsten Schmidt)]

Oh I see.

On Mountain Lion it will be the opposite problem: you cannot compile your code with libc++ and link it with boost compiled with the system standard C++ library libstdc++.

My understanding is that clang's C++11 support requires libc++ and does not work with libstdc++. So you cannot use C++11 with clang on Mountain Lion or earlier, unless you rebuild all ports with libc++, as the ticket you referenced in your opening remarks says.

[ryandesign (Ryan Carsten Schmidt)]

For more on this topic, see this thread: ​https://lists.macosforge.org/pipermail/macports-dev/2013-December/025360.html. That's probably a better place to continue the discussion.

[cjones051073 (Chris Jones)]

Replying to akim.demaille@…:

Well, this is really a pity. I wonder how the GNU/Linux distros handle this issue when they deploy Boost.

Simple, they don't mix c++ runtimes. They pick one (I guess 99.99% of the time its a libstdc++ runtime, are there any linux distros based on libc++ yet ?) and compile *everything* against that 'one true runtime'.

MacPorts doesn't get to help Apple decide what runtimes to use. With OSX10.8 the default was apples stdlibc++, with 10.9 its libc++. The best thing MacPorts can do is go with the flow. This means on OSX10.9 use libc++ and the system clang.

Why are you so intent on using GCC ? If you want to carry on support OSX, the best thing I would say you can do is invest time to get your code working with clang.

Chris

[cjones051073 (Chris Jones)]

Cc Me!

[akimd (Akim Demaille)]

Replying to jonesc@…:

Replying to akim.demaille@…:

Well, this is really a pity. I wonder how the GNU/Linux distros handle this issue when they deploy Boost.

Simple, they don't mix c++ runtimes. They pick one (I guess 99.99% of the time its a libstdc++ runtime, are there any linux distros based on libc++ yet ?) and compile *everything* against that 'one true runtime'.

FWIW, I can see a significant difference bw the Ubuntu I have at hand, and the MacPorts: on the Ubuntu, clang used in C++11 mode does support C++11 with stdlibc++ (the one from GCC). On Mountain Lion, when I use clang++ -std=c+11, it seems to use a pre-C++11 installation of libstdc++, so it does not support C++11. So I have to pass -stdlib=libc++, which in turn breaks everything, since that's not the "system" C++ library.

As a consequence, I can, on Ubuntu, use C++11 and Boost with both compilers (since the real issue is not the compiler, but the "standard" C++ library).

Can't MacPorts' deployments of clang be fixed so that they use the MacPorts' libstdc++?

Why are you so intent on using GCC ? If you want to carry on support OSX, the best thing I would say you can do is invest time to get your code working with clang.

I maintain free software (e.g., Bison and Vaucanson), and it is extremely important for me to test this software in all the possible configurations: this way I can anticipate the problems that other will face, and will report to me. It saves me a lot of time to fix issues before they spread in releases.

Also clang and g++ have different virtues. Sometimes I prefer Clang error messages, but GCC optimizes better my pieces of code.

I also confess that I feel safer with the GPL than with BSD, and I am worried to see GCC being "ejected" from Mac OS X; this is certainly something that Apple is happy about, and I am not.

[ryandesign (Ryan Carsten Schmidt)]

Replying to akim.demaille@…:

FWIW, I can see a significant difference bw the Ubuntu I have at hand, and the MacPorts: on the Ubuntu, clang used in C++11 mode does support C++11 with stdlibc++ (the one from GCC). On Mountain Lion, when I use clang++ -std=c+11, it seems to use a pre-C++11 installation of libstdc++, so it does not support C++11. So I have to pass -stdlib=libc++, which in turn breaks everything, since that's not the "system" C++ library.

I assume Apple's libstdc++ is based on the one used in GCC 4.2.1, since that's the last version of GCC before the switch to GPLv3 which Apple appears to have decided as a company not to look at.

Can't MacPorts' deployments of clang be fixed so that they use the MacPorts' libstdc++?

Please join the discussion on the mailing list that I mentioned above. However, we are unlikely to use libstdc++ on Mavericks even if it were possible; we are not here to second-guess Apple's decisions. What we are however discussing is whether we can switch Lion and Mountain Lion to libc++.

[cjones051073 (Chris Jones)]

Can't MacPorts' deployments of clang be fixed so that they use the MacPorts' libstdc++?

'Fixing' it implies its a bug, which it isn't, its intentional.

MacPorts clang compilers (currently) intentionally use the systems default c++ runtime, in order to allow libraries built with these compilers to be linked with others built using the default system compiler. On OSX10.8, this default runtime is the system libstdc++, which does not have full c++11 support.

Of course, this could theoretically be changed. These compilers could link against a MacPorts provided c++ runtime (although I am pretty sure if this where to happen, it would be a libc++ implementation and not libstdc++) but then we would break the above intercompatibity to some degree.

Currently, right now, the only OSX release that has c++11 support in MacPorts in OSX10.9. If you really value c++11 supoport, then honestly, upgrading to this is your best bet right now. Discussions are on going as to if it is possible to get c++11 support into older OSX versions, but its looking pretty complicated right now to do this.

Chris