cmake: fix build against libc++ on Lion

[mojca (Mojca Miklavec)]

Building CMake against libc++ seems to be broken on Lion. The reason is that the bootstrap cmake sometimes cuts the lines at 1024 characters.

See:

• ​http://public.kitware.com/pipermail/cmake-developers/2014-June/010712.html
• ​http://www.cmake.org/Bug/view.php?id=15039

The dirty workaround is to run a few commands manually, but it would be nice to find a proper patch.

[cooljeanius (Eric Gallager)]

Cc Me!

[mojca (Mojca Miklavec)]

graphite2 is also affected by this bug.

[mojca (Mojca Miklavec)]

I found out what the problem was. CMake uses complicated code (as a workaround to compiler bugs) to read lines and istream.gcount() in clang 3.3 with libc++ is buggy. The following code:

#include <iostream>
#include <fstream>
#include <string>

int main()
{
  std::ifstream is("test.txt");
  std::string line;
  const int bufferSize = 11;
  char buffer[bufferSize];
  is.getline(buffer, bufferSize);
  std::cout << is.gcount() << std::endl;
  return 0;
}

returns 11 instead of 10 (for input text with > 10 characters).

Until the problem gets fixed the workaround is to use clang 3.4 or 3.5.

Would something like that work?

    platform darwin 11 {
        if {${configure.cxx_stdlib} eq "libc++"} {
            # everything but macports-clang-3.4 macports-clang-3.5
            compiler.blacklist-append *gcc* {clang < 500} macports-clang-2.9 macports-clang-3.0 macports-clang-3.1 macports-clang-3.2 macports-clang-3.3
            compiler.fallback-append macports-clang-3.4 macports-clang-3.5
        }
    }

We probably need to blacklist just clang 3.3 ({clang < 500} macports-clang-3.3). (I'm not sure if other compilers even work with libc++.)

On the other hand we can simply wait for an upstream fix.

[jeremyhu (Jeremy Huddleston Sequoia)]

Good find. I don't see anything obviously releated in istream when comparing the STL (/opt/local/libexec/llvm-3.[34]/lib/c++/v1/istream), but if you figure out the cause, we can rev clang-3.3 with the fix.

Note that macports-clang-3.4 should already be in the fallback list, so you really just need the compiler.blacklist-append line.

[cooljeanius (Eric Gallager)]

Replying to jeremyhu@…:

Note that macports-clang-3.4 should already be in the fallback list

Could you remind me where the contents of the fallback list are documented again?

[larryv (Lawrence Velázquez)]

tags/release_2_3_1/base/src/port1.0/portconfigure.tcl#L426​

[mojca (Mojca Miklavec)]

The compile is even more broken than what I though:

• ​http://www.cmake.org/Bug/view.php?id=15039#c36490

Anyway, if we would start "officially supporting" libc++ for the sake of better support for C++11, I would suggest to pick clang-3.4 as the default compiler (if that is feasible).

[jeremyhu (Jeremy Huddleston Sequoia)]

That's a good idea. We can conditionalize the fallback list based on cxx_stdlib.

[mojca (Mojca Miklavec)]

Michael, I assigned the ticket to you now that you are the one that's actively maintaining the port.

I only experienced the problem and tested on 10.7, but maybe someone should check what's going on with 10.6 and 10.8 as well.

The following patch should work on 10.7:

if {${configure.cxx_stdlib} eq "libc++"} {
    compiler.blacklist-append *gcc* {clang < 500} macports-clang-2.9 macports-clang-3.0 macports-clang-3.1 macports-clang-3.2 macports-clang-3.3
}

Whether or not this should be inside a platform darwin 11 block, I have no idea.

[michaelld (Michael Dickens)]

Mojca: I'm running 10.8, and have never had an issue with CMake. How do I determine/test that there's an issue?

[mojca (Mojca Miklavec)]

You need to install MacPorts from scratch into a different prefix (also changing applications_dir etc.) and set libc++ as default stdlib.

These are my changes in macports.conf:

cxx_stdlib libc++
delete_la_files yes
buildfromsource always

If this is not documented anywhere on the wiki yet, it probably should be.

You would then want to remove configure.cxx_stdlib libstdc++ from the Portfile, but if you are on ML, it shouldn't have any effect anyway.

See r110452, r131111, r131116.

Running sudo port install cmake would trigger the error on Lion, but running sudo port install cmake configure.compiler=macports-clang-3.4 works. I would imagine that running

sudo port install cmake configure.compiler=macports-clang-3.3

should reproduce the error on Mountain Lion. And I would imagine that ML with an old Xcode would fail in the same way as Lion does. I didn't test on anything below Lion yet.

The problem seems to be caused by a *combination* of libc++ and clang < 3.4. (Maybe much older clang still works. And maybe it's different with different versions of libc++.)

[jeremyhu (Jeremy Huddleston Sequoia)]

Replying to mojca@…:

If this is not documented anywhere on the wiki yet, it probably should be.

It is:

LibcxxOnOlderSystems

[michaelld (Michael Dickens)]

Do you think running sudo port install cmake configure.compiler=macports-clang-3.3 on 10.10 will reproduce the error? I'm going to try it shortly, once clang 3.3 is installed.

[larryv (Lawrence Velázquez)]

Cc Me!

[michaelld (Michael Dickens)]

To answer my own question: Nope; does not seem like that does it. On 10.10, using clang-3.3 still links against the Xcode-provided /usr/lib/libc++.dylib, which does not contain the bug.

[michaelld (Michael Dickens)]

Does this summary make sense to folks?

Reading through the Portfile for libcxx (and libcxxabi), I see that it builds without forcing (via +replacemnt_libcxx) on 10.5 and 10.6 only; one has to force it on 10.7+; this means that on 10.7+ by default, we get whatever libc++ is provided by Xcode. I just tested Mojca's C++ stream code on 10.8 (Xcode 5.1.1, Clang 503.0.40), and it is OK -- which fits with the "blacklist" above too (do not use clang < 500). So, if folks are up to date on 10.8, then they should be OK. I don't know what Xcode is provided on 10.7, but it sounds like even the most recent not be recent enough to provide a usable libc++. But, if clang/llvm (per libcxx[abi]) uses the built-in libc++ as provided by Xcode, then it seems like we have an issue: even blacklisting (clang < 500) && (mp-clang < 3.3) won't do the trick because the libc++ is broken to begin with. What they need is an updated libc++, via the "libcxx +replacemnt_lbcxx" port / variant.

I really can't test out this hypothesis, since I don't have access to a 10.7- box (I have 10.8 and 10.10 boxes).

[michaelld (Michael Dickens)]

I attached a test .cc file, Mojca's modified to print out whether the gcount is OK or NOT OK. Simple to compile & execute.

[mojca (Mojca Miklavec)]

True, libc++ comes with the system.

But

clang++-mp-3.4 -stdlib=libc++ test.cxx -o test

works for me. And it links to the same stdlib:

> otool -L test
test:
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 28.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)

See XcodeVersionInfo.

[michaelld (Michael Dickens)]

Do you have access to a 10.7 box? If so, I'd be curious how to test code does when using different clang versions. I'd guess it'll always fail unless libcxx +replacemnt_libcxx is used.

See XcodeVersionInfo.

Cool; thanks!

[mojca (Mojca Miklavec)]

I don't have +replacement_libcxx activated.

[michaelld (Michael Dickens)]

Do you have +replacemnt_libcxx activated? (Note lacking the last 'e': "mnt", not "ment")? And, which OS are you on? If you're on 10.7+, then libcxx just installs a README file; see "port contents libcxx".

[larryv (Lawrence Velázquez)]

I have VMs for 10.6 through 10.9, but not with +replacement_libcxx. Let me know if you want me to try anything.

[michaelld (Michael Dickens)]

larryv: If you compile the attached .cc file on your 10.7 VM using the script embedded in it, then execute the resulting executable, what does it print? If you don't have "libcxx +replacemnt_libcxx" installed, then you're using the Xcode libc++, which version is < 500 and hence should have the issue reported here & the script should print "libc++ is NOT OK: is.gcount() == 11".

[mojca (Mojca Miklavec)]

• Yes, I'm on 10.7.
• No, I don't have +replacement_libcxx activated. Yes, libcxx is a placeholder package only (no contents other than the README).
• As I already explained:
  • clang++ fails (Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn))
  • clang++-mp-3.3 fails (I just installed and tested that one; I'm not eager to install < 3.3.)
  • clang++-mp-3.4 and clang++-mp-3.5 work (I assume higher versions do as well.)

I would assume that clang-3.4 fixed the problem even for the broken libc++ then ;)

If it was easier (=faster) to compile clang, I would happily do the bisection to find the relevant commit, but it's not relevant enough to spend days compiling.

It might be interesting to test on 10.6 and to test whether the old Xcode on 10.8 produces the error, but large amount of work would not justify the benefit of knowing the output.

[michaelld (Michael Dickens)]

Mojca: thanks & understood & agreed. It would be fascinating, but there's only so much time in the day. Interesting results; I was not expecteding clang++-mp-3.4 to fix an issue with libc++; just to use it. Maybe I don't understand how MP builds clang / llvm & hence how it uses libcxx / libc++. Anyway, if what you say is universally true, then it seems like we could do the following in the cmake Portfile:

# when using libc++, require Clang 3.4 or newer; Clang 3.3 produces
# bad stream reading code. See also MP ticket #44129:
# < https://trac.macports.org/ticket/44129 >

if {${configure.cxx_stdlib} eq "libc++"} {
    compiler.blacklist-append *gcc* {clang < 500} macports-clang-3.3
    compiler.fallback-append macports-clang-3.4 macports-clang-3.5
}

Is the fallback really necessary? Could also use whitelist.

I don't think we need a "platform darwin 11" around that, since it's universally true; maybe around the fallback list only?

[mojca (Mojca Miklavec)]

In comment:5 it was already noted that fallback should not be needed. In the meantime Jeremy figured out that libc++ doesn't work with default compiler on < 10.7 either (see r131111). I'm speculating that blacklisting compilers would solve the issue there as well, but we cannot be sure unless we try.

[mojca (Mojca Miklavec)]

Michael, are you willing to commit the change that you suggested in comment:27 or do you want me to do any further testing?

(I guess you don't need the fallback because macports-clang-3.4 is already on the list of defaults, but maybe double-check in the MacPorts sources just in case. It certainly is on the default fallback list on 10.7 and on < 10.7 you cannot use libc++ without clang 3.4 to begin with. But if you want to add clang 3.5, you could add all newer versions as well then.)

[jeremyhu (Jeremy Huddleston Sequoia)]

I'd avoid using 3.5. I actually want to remove it soon. It has codegen issues for i386.

[mojca (Mojca Miklavec)]

In 9eb9d7b3/macports-ports:

cmake: blacklist clang < 3.4 for libc++

CMake fails to compile with clang < 3.4 against libc++
due to a compiler bug. Blacklist old clang compilers.

See: ​https://cmake.org/Bug/view.php?id=15039
Closes: #44129