ghc @8.6.5_1: Built ghc not working on Catalina

[abusse (Anselm Busse)]

Compiled on macOS 10.15 with system compiler (Xcode 11.1) completes bootstrap, but the compiler later fails to run simple programs with a memory-related error:

ghc: mmap 131072 bytes at 0x0: Operation not permitted
ghc: Try specifying an address with +RTS -xm<addr> -RTS
ghc: internal error: m32_allocator_init: Failed to map
   (GHC version 8.6.5 for x86_64_apple_darwin)

Homebrew seems to have the same issue: ​https://github.com/Homebrew/homebrew-core/pull/44802

The issue is discussed upstream: ​https://gitlab.haskell.org/ghc/ghc/issues/17353 However, it seems to be an issue during building and not in the ghc code.

[cjones051073 (Chris Jones)]

Looks like just another instance of the Xcode 11 bug related to stack-check.

Simple solution, until either Apple fix it or upstream do something is blacklist Xcode 11 clang.

[essandess (Steve Smith)]

Another workaround: just install with the +prebuilt variant and hope that it runs.

[cjones051073 (Chris Jones)]

Replying to essandess:

Another workaround: just install with the +prebuilt variant and hope that it runs.

Last resort. Please try and fix the build itself. If it is just the Xcode 11 issue again, based on experience with other ports either just blacklisting Xcode 11 clang, so macports clang 9.0 is used, or appending the compiler option -fno-stack-check will do the job.

[mouse07410 (Mouse)]

IMHO, blacklisting Xcode-11 would be a very bad idea. That's the way Apple chose to proceed. I don't even know if XCode-10 would work on Catalina. We got to deal with it.

Adding -fstack-no-check appears the best solution/workaround, at least for now.

[cjones051073 (Chris Jones)]

An Apple insider as confirmed its a bug in Xcode 11. So blacklisting it is perfectly reasonable.

[cjones051073 (Chris Jones)]

Also note, just to be clear, the information we have is it should be fixed in Xcode 11.2 (time will tell on this). At this point the blacklist can be made specific to the clang version range affected.

Also, blacklisting Xcode 11 does not mean you have to install Xcode 10 !! It just means, for the port that does it, port will not use the Xcode clang versions blacklisted, and fallback to another compiler selection, which in this case will be MacPorts own build of clang 9.0. This works fine, we use it now in a number of places for exactly this issue.

[mouse07410 (Mouse)]

Chris, ok thanks, that makes sense. If Xcode-11.2 fixed the problem, it's fine to blacklist Xcode-11.{0,1}. Residually if having Macports clang-9 resolves the issue anyway

[ryandesign (Ryan Carsten Schmidt)]

Xcode 11.2.1 is out and is installed on our Catalina buildbot worker. But unfortunately ​ghc just got built with it and ​still has this problem.

[casouri (Yuan Fu)]

For other normal users, prebuilt variant works.

[essandess (Steve Smith)]

I asked over at ghc how to bootstrap completely from source. If it’s straightforward and more robust, that would be preferable to using the prebuilt binary to bootstrap. Thread: ​https://mail.haskell.org/pipermail/glasgow-haskell-users/2020-January/026898.html

[essandess (Steve Smith)]

Replying to mouse07410:

Adding -fstack-no-check appears the best solution/workaround, at least for now.

I'm not familiar with this flag. Doing this breaks configure completely:

    # Fix Xcode 11 issue; see https://trac.macports.org/ticket/59467
    configure.optflags-append \
                    -fstack-no-check

What's the recommended approach using Portfile code?

[essandess (Steve Smith)]

Replying to cjones051073:

Simple solution, until either Apple fix it or upstream do something is blacklist Xcode 11 clang.

I tried this, but it still produces a broken ghc binary, with exactly the same problem as with Apple clang as far as I can tell:

PortGroup           compiler_blacklist_versions 1.0
…
    # Fix Xcode 11 issue; see https://trac.macports.org/ticket/59467
    compiler.blacklist-append clang

Error (same one as with a clang compile):

ghci
GHCi, version 8.6.5: http://www.haskell.org/ghc/  :? for help
ghc: mmap 131072 bytes at 0x0: Operation not permitted
ghc: Try specifying an address with +RTS -xm<addr> -RTS
ghc: internal error: m32_allocator_init: Failed to map
    (GHC version 8.6.5 for x86_64_apple_darwin)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug
Abort trap: 6

Would you please post or do a PR for the Portfile you're using to get a working bootstrapped ghc on Catalina?

[kencu (Ken)]

the ghc catalina issue is different, as per <​https://mail.haskell.org/pipermail/glasgow-haskell-users/2020-January/026897.html>. Sounds like Ben knows the fix.

bootstrapping ghc from c source is difficult. Doubt we can do that.

[essandess (Steve Smith)]

This looks like an upstream issue with ghc. See ​https://gitlab.haskell.org/ghc/ghc/issues/17353.

I don’t know how Haskell world creates their prebuilt macOS ghc binary—cross compiled off a Linux box? If there’s some general approach that bootstraps from, say, Python, that could work. But yeah, that will likely be too difficult and too much work.

If anyone knows how to add -fstack-no-check to the Portfile, please post it.

[mf2k (Frank Schima)]

The guile18 portfile uses a ​patch file.

[cjones051073 (Chris Jones)]

just use the xcode_workaround Portgroup

​https://github.com/macports/macports-ports/blob/master/_resources/port1.0/group/xcode_workaround-1.0.tcl

[kencu (Ken)]

stack check has nothing to do with this

[kencu (Ken)]

I know how haskell generates their prebuilt binary, as I generated 10 of them to bring 10.6.8 up to current. Ben has a build script that does it, using an older ghc no more than two releases earlier to do it. i have the whole setup on my snowleopard system.

[kencu (Ken)]

For fun, the script is here, but NB I had to modify it slightly to work with Macports, as Ben uses homebrew:

<​https://github.com/bgamari/ghc-utils>

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #59990.

[kencu (Ken)]

this patch is supposed to fix this issue (untested) <​https://gitlab.haskell.org/ghc/ghc/merge_requests/2190/diffs?commit_id=c6fca740a708ad9db5fdb595ad06f727289cdc07>.

[essandess (Steve Smith)]

Replying to kencu:

I know how haskell generates their prebuilt binary, as I generated 10 of them to bring 10.6.8 up to current.

Ken, how is it done? I assumed that they used a cross-compiler, no? Can ghc be compiled from ghc source entirely on macOS?

If so, and it's easy and robust, that would be preferable to using the prebuilt compiler.

[essandess (Steve Smith)]

Also note that ghc 8.8.2 fails to build itself on Catalina. See ​https://mail.haskell.org/pipermail/glasgow-haskell-users/2020-January/026897.html.

[kencu (Ken)]

ghc bootstraps off a recent previous ghc; I understand cross compiling is a project. Ben's script I referenced above simplifies it.

the patch I referenced is meant to fix the issue on the mailing list you note, but i guess a new release is likely coming at some point anyway...

[essandess (Steve Smith)]

In 16bf91aad3fb09da600736391ac522752ee4227a/macports-ports (master):

ghc: Update to version 8.8.3

• Fixes: #59467