ghc & haskell-platform update?

[J.Gilbey@…]

Is it feasible to update the haskell-platform port at some point? Upstream is now at 7.10.2, which introduces some significant changes from what I can tell.

Thanks!

[neverpanic (Clemens Lang)]

Yes, I'm aware of it and I plan to update it.

[jason-johnson (Jason)]

Hello, any update on this?

[jason-johnson (Jason)]

Hello,

Is this still maintained? This issue is getting close to a year old.

Thanks

[neverpanic (Clemens Lang)]

I'd still like to do this, but time is limiting. If you're willing to help, I can give you some pointers.

[neverpanic (Clemens Lang)]

Status update: I've had a first successful build of the new GHC and am currently testing Debian's patch to fix an issue where the build path of the compiler leaks into the ABI hashes (see ​https://ghc.haskell.org/trac/ghc/ticket/10424), since that would affect us with every binary package we ship from the buildbots.

[larryv (Lawrence Velázquez)]

Cc Me!

[elventear (Pepe Barbe)]

Cc Me!

[larryv (Lawrence Velázquez)]

Adding reporters of #52221 and #52317.

[stromnov (Andrey Stromnov)]

Cc Me!

[andre.david@…]

Cc Me!

[arthur.michaut@…]

Cc Me!

[andre.david@…]

According to #52317, this is blocking ghc availability in OSX sierra (and consequently of pandoc).

[john@…]

Cc Me!

[machielbos]

Here's another user who needs ghc under macOS Sierra to install pandoc!

[larryv (Lawrence Velázquez)]

Has duplicate #52491.

[larryv (Lawrence Velázquez)]

Cc Me!

[grimreaper (Eitan Adler)]

Cc Me!

[neverpanic (Clemens Lang)]

See ​https://lists.macosforge.org/pipermail/macports-dev/2016-October/034073.html for why this isn't done yet. Help welcome.

[gorticus (Jason Mitchell)]

Cc Me!

[kprussing (Keith)]

Is a major hold up still the need to create all the Portfiles to build stack? I started on a few to get pandoc updated, but then I saw how many I would need to update. So, is there a communal place where the edits are being worked on (per the first comment in the linked message)?

[informatimago (Pascal J. Bourguignon)]

It looks like by just adding:

#ifdef darwin_HOST_OS
#include <mach/mach_time.h>
#endif

in rts/posix/GetTime.c , the following errors can be eliminated.

:info:build "inplace/bin/ghc-stage1" -optc-m64 -optc-fno-stack-protector -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-fno-strict-aliasing -optc-fno-common -optc-DDTRACE -optc-O2 -optc-fomit-frame-pointer -optc-DRtsWay=\"rts_v\" -static  -H32m -O -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -package-name rts -dcmm-lint  -DDTRACE     -i -irts -irts/dist/build -irts/dist/build/autogen -Irts/dist/build -Irts/dist/build/autogen           -O2    -c rts/posix/GetTime.c -o rts/dist/build/posix/GetTime.o
:info:build 
:info:build rts/posix/GetTime.c:44:5:
:info:build      error: use of undeclared identifier 'mach_timebase_info_data_t'
:info:build         mach_timebase_info_data_t info;
:info:build         ^
:info:build 
:info:build rts/posix/GetTime.c:45:12:
:info:build      warning: implicit declaration of function 'mach_timebase_info' is invalid in C99 [-Wimplicit-function-declaration]
:info:build         (void) mach_timebase_info(&info);
:info:build                ^
:info:build 
:info:build rts/posix/GetTime.c:45:32:
:info:build      error: use of undeclared identifier 'info'; did you mean 'sinf'?
:info:build         (void) mach_timebase_info(&info);
:info:build                                    ^~~~
:info:build                                    sinf
:info:build 
:info:build /usr/include/math.h:342:14:  note: 'sinf' declared here
:info:build extern float sinf(float);
:info:build              ^
:info:build 
:info:build rts/posix/GetTime.c:46:44:
:info:build      error: use of undeclared identifier 'info'; did you mean 'int'?
:info:build         timer_scaling_factor_numer = (uint64_t)info.numer;
:info:build                                                ^~~~
:info:build                                                int
:info:build 
:info:build rts/posix/GetTime.c:46:44:  error: expected expression
:info:build 
:info:build rts/posix/GetTime.c:47:44:
:info:build      error: use of undeclared identifier 'info'; did you mean 'int'?
:info:build         timer_scaling_factor_denom = (uint64_t)info.denom;
:info:build                                                ^~~~
:info:build                                                int
:info:build 
:info:build rts/posix/GetTime.c:47:44:  error: expected expression
:info:build 1 warning and 6 errors generated.
:info:build make[1]: *** [rts/dist/build/posix/GetTime.o] Error 1

[kencu (Ken)]

see also ​https://github.com/macports/macports-ports/pull/70

[markemer (Mark Anderson)]

I'd be willing to help, but I'm running in circles just trying to get ghc 8.4.3 to build. Are you still offering up those pointers? I think I brought this up a while back while trying to install pandoc.

[neverpanic (Clemens Lang)]

Absolutely. Here's my current WIP state that updates to 8.0.3: ​https://github.com/neverpanic/macports-ports/tree/haskell-platform-update. You may have to get a newer ghc-bootstrap compiler to compile 8.4.3, though.

[agm650]

Hello,

I'm willing to help on this topic. I've made the ghc-bootstrap to use the 8.6.2 version.

So after I guess I have to go on with the whole haskell-platform update (a bit scaring)?

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #57825.

[mouse07410 (Mouse)]

Guys, I know little about Haskell, but something about C and such. Can I help?

[kencu (Ken)]

Currently we are trying to manually replace the work of cabal to integrate it into the Macports way of doing things, and this is proving to be a huge, perhaps unsustainable, project.

Just calling cabal to install the packages is easy. But doesn't let Macports know what is going on.

We could find a way to drive and query cabal....(much like we might do with pip and cpan, perhaps.)

Or, we could parse the cabal package information or output to automate portfile generation and updates.

But doing it all manually is collapsing...

[mouse07410 (Mouse)]

Just calling cabal to install the packages is easy. But doesn't let Macports know what is going on

In Python, there are far more packages that are installable (and the user is likely to want to install) via pip than are supported by Macports. This already is causing me problems, because some packages are available in both Macports and pip, others - only in pip. And there are frequent collisions between the two.

Based on the above Python experience, I strongly suggest that we do as you said above:

Just calling cabal to install the packages is easy. But doesn't let Macports know what is going on

and keep Macports away from Hackage package management. You aren't going to be able to replicate in Macports even 10% of what's on Hackage - like the Python maintainers cannot replicate a significant part of PYPI. And that's fine. Just provide a working Python and a working pip - and the user will take care of the rest. I suggest using the same approach with Haskell and cabal.

[kencu (Ken)]

what have the other package management systems decided to do about this issue? I assume all of them are in exactly the same boat.

[mouse07410 (Mouse)]

what have the other package management systems decided to do about this issue? I assume all of them are in exactly the same boat.

Some of them probably do something reasonable, and some - don't (in my opinion). For example, IMHO what Macports Python does is crazy (IMHO), and causes unnecessary conflicts because, as I explained, there's no way for Macports to replicate he entire PYPI repo, so some packages get overridden when the user needs to install something, e.g., from PYPI and that thing pulls its own dependencies - again directly from PYPI. Leaving the packages to the corresponding infrastructure, and providing just the "main" binaries - ghc, cabal, and stack - would be best. And simplest too.

Then, BTW, you won't need to bother with the unnecessary dependency on LLVM. I just rebuilt GHC from the sources, and it was smooth as castor oil. Why Macport GHC port can't do the same? Don't overcomplicate things - KISS rules.

[neverpanic (Clemens Lang)]

Replying to mouse07410:

Some of them probably do something reasonable, and some - don't (in my opinion). For example, IMHO what Macports Python does is crazy (IMHO), and causes unnecessary conflicts because, as I explained, there's no way for Macports to replicate he entire PYPI repo, so some packages get overridden when the user needs to install something, e.g., from PYPI and that thing pulls its own dependencies - again directly from PYPI.

I think the current approach is supposed to be "package everything that needs native dependencies". The question of native dependencies is where the whole approach of "don't package any of the libraries" falls apart, because users use MacPorts precisely to avoid having to compile their own copy of $library_used_by_python_module or to figure out how to get the Python build system to find MacPorts' copy of said library.

Leaving the packages to the corresponding infrastructure, and providing just the "main" binaries - ghc, cabal, and stack - would be best. And simplest too.

Back when we originally packaged GHC, we were told by the Haskell community that we should really be providing GHC "batteries included" by packaging the haskell platform. Maybe it is time to revisit this decision.

However, keep in mind that many users will expect packages such as pandoc and shellcheck to be available from MacPorts. Approaching this as "if you want pandoc, install cabal/stack and run cabal/stack install pandoc" is not the best alternative from a UX perspective – especially since we don't have a good way to communicate this exception to the common usage of MacPorts.

Then, BTW, you won't need to bother with the unnecessary dependency on LLVM. I just rebuilt GHC from the sources, and it was smooth as castor oil. Why Macport GHC port can't do the same? Don't overcomplicate things - KISS rules.

The LLVM dependency isn't the issue here, I think. Building a newer version of GHC isn't very complicated. It's the platform vs. no platform and libraries vs. not packaging libraries discussion that makes things complicated.

Btw, stack has hundreds of dependencies – it's what kept me from finishing the packaging of a current version of haskell-platform. Even just building stack with the current approach probably needs some kind of automation.

If the maintainers feel married to the current approach - let's leave the current (ancient) port of ghc-7.8.3 alone, with all the goodies it offers, and add another "minimalistic" port of ghc-8.6.3 that provides only ghc (not dependent on LLVM!), cabal, and stack. Everything else users can install themselves just fine.

I'm not married to that approach. If you want to submit patches to update the GHC port and drop haskell-platform as well as the other Haskell ports, I'm fine with that. I don't have a strong opinion on LLVM or not (although I don't believe it's a big problem), however, I would expect commonly used ports such as pandoc and shellcheck to be available from MacPorts. If you can provide that plus a copy of stack, that'd be great.

Maybe a good approach would also be to build all tools (such as pandoc) using stack from a Portfile, so that they all ship their dependencies in a private folder. We've recently adopted a similar approach for ports using rust's cargo. See the cargo 1.0 PortGroup, for example.

[mouse07410 (Mouse)]

I think the current approach is supposed to be "package everything that needs native dependencies". The question of native dependencies is where the whole approach of "don't package any of the libraries" falls apart

Respectfully disagree. One reason - it proved to because infeasible to provide Macports clones of all the packages maintained by other env-specific ecosystems. Python is the best example of that.

For example, I use Macports not because I can't do pip install pycryptodome, but because it's convenient for me to, e.g., install a reasonable version of OpenSSL via sudo port install openssl (actually, a bad example - as one of the contributors, I do keep source tree of OpenSSL, and regularly rebuild master and stable branches ;).

I see absolutely nothing wrong for the user to do sudo port install gcc8 && pip-3.7 install numpy. It does not seem much harder than sudo port install py37-numpy.

Approaching this as "if you want pandoc, install cabal/stack and...

Why would a Haskell user want to install pandoc via Macports??? Any more than installing Haskell-libsodium via Macports? Or <put your favorite Haskell package here>?

Macports cannot and shouldn't become a substitute for ecosystem-specific package managers.

Even just building stack with the current approach probably needs some kind of automation...

Yes, non-trivial - which is why I'd be perfectly content packaging pre-built stack binary and keeping it. For example, my system uses pre-built stack - I did not rebuilt it, not do I want to. It works. I don't need Macports to provide it to me, nor do I feel like I'm obligated to rebuild it from sources.

It's the platform vs. no platform and libraries vs. not packaging libraries discussion that makes things complicated

There already is Haskell Platform. The main reason I don't use it in its entirety is because it's GHC conflicts with Macports libiconv. So I had to rebuild GHC myself. But stack from Haskell Platform is perfectly usable as is. Whatever Haskell libraries I need - stack or cabal will install for me, and would do that the right way. No reason for Macports to try emulating that.

Back when we originally packaged GHC, we were told by the Haskell community that we should really be providing GHC "batteries included" by packaging the haskell platform. Maybe it is time to revisit this decision.

Yes!! I strongly urge to revisit. Haskell ecosystem is very strong. It's neither needed nor possible to replicate it. The only reason I'm suggesting refreshing the port is because the default GHC provided by Haskel ecosystem conflicts with Macports. Everything else from Haskell eco coexists with Macports just fine, so no need to re-do it.

Maybe a good approach would also be to build all tools (such as pandoc) using stack from a Portfile, so that they all ship their dependencies in a private folder. We've recently adopted a similar approach for ports using rust's cargo

Now, that is an idea. But I'm getting concerned when I hear "to build *all tools" from you. My question is - since those tools are trivially installed by stack, what benefit is there in offering what essentially is a Macports wrapper for it? And is it worth the cost (time and efforts)? Is sudo port install pandoc really that much simpler than stack install pandoc, especially for somebody who presumably got GHC (with cabal and stack) because that somebody is developing within the Haskell ecosystem?

[neverpanic (Clemens Lang)]

Replying to mouse07410:

For example, I use Macports not because I can't do pip install pycryptodome, but because it's convenient for me to, e.g., install a reasonable version of OpenSSL via sudo port install openssl (actually, a bad example - as one of the contributors, I do keep source tree of OpenSSL, and regularly rebuild master and stable branches ;).

I see absolutely nothing wrong for the user to do sudo port install gcc8 && pip-3.7 install numpy. It does not seem much harder than sudo port install py37-numpy.

That's fine if (and only if) the pip-3.7 install numpy will work out of the box. It usually does with stuff in /usr/local, but making your pip find stuff in /opt/local commonly requires flags. Additionally, using MacPorts pip will put files in MacPorts prefix that MacPorts doesn't know about, which has previously led to problems. We want to avoid that.

Approaching this as "if you want pandoc, install cabal/stack and...

Why would a Haskell user want to install pandoc via Macports??? Any more than installing Haskell-libsodium via Macports? Or <put your favorite Haskell package here>?

Because people don't (and shouldn't) care that pandoc or shellcheck are written in Haskell. They just want the tool.

Yes, non-trivial - which is why I'd be perfectly content packaging pre-built stack binary and keeping it. For example, my system uses pre-built stack - I did not rebuilt it, not do I want to. It works. I don't need Macports to provide it to me, nor do I feel like I'm obligated to rebuild it from sources.

Sure, but given this argument, we might as well not package Haskell at all in MacPorts. We could just remove it and tell people to use the upstream binaries provided by the GHC developers.

There already is Haskell Platform. The main reason I don't use it in its entirety is because it's GHC conflicts with Macports libiconv. So I had to rebuild GHC myself.

Hm, that's weird. Binaries on macOS reference their libraries using absolute paths, and the upstream GHC binary shouldn't link against /opt/local/lib/libiconv.2.dylib, so unless you have DYLD_LIBRARY_PATH in your environment, the upstream Haskell Platform should just work. Have you looked into why that's not the case?

Maybe a good approach would also be to build all tools (such as pandoc) using stack from a Portfile, so that they all ship their dependencies in a private folder. We've recently adopted a similar approach for ports using rust's cargo

Now, that is an idea. But I'm getting concerned when I hear "to build *all tools" from you. My question is - since those tools are trivially installed by stack, what benefit is there in offering what essentially is a Macports wrapper for it? And is it worth the cost (time and efforts)? Is sudo port install pandoc really that much simpler than stack install pandoc, especially for somebody who presumably got GHC (with cabal and stack) because that somebody is developing within the Haskell ecosystem?

You're making a few assumptions here that I don't think are correct:

• I'm not trying to put an emphasis on all tools, but an emphasis on tools, i.e. it wouldn't make sense to package libraries in this way.
• The benefit of having those wrapped is that you have a single way to update them and a single interface to manage them. I don't want to have to care about five different language-specific package managers when all I want is a single end-user tool.
• Most of the Haskell stuff is distributable, so our build servers will provide precompiled binaries. This significantly speeds up installation.
• Given sufficient automation (and none of the dependency hell), the time required to do this would probably not be very high, i.e. the cost would be low. In my eyes, that would make it worth the effort.

I agree there's not a big difference for somebody who develops in the Haskell ecosystem, but those people really don't need our help in figuring this out anyway.

[mouse07410 (Mouse)]

Sure, but given this argument, we might as well not package Haskell at all in MacPorts. We could just remove it and tell people to use the upstream binaries provided by the GHC developers

That's what I wanted to do. It failed - see below.

There already is Haskell Platform. The main reason I don't use it in its entirety is because it's GHC conflicts with Macports libiconv. So I had to rebuild GHC myself.

Hm, that's weird. Binaries on macOS reference their libraries using absolute paths, and the upstream GHC binary shouldn't link against /opt/local/lib/libiconv.2.dylib, so unless you have DYLD_LIBRARY_PATH in your environment, the upstream Haskell Platform should just work. Have you looked into why that's not the case?

Of course I did (see #57821). There are two reasons for this problem:

• libHSbase exists only as libHSbase.a, so forget about absolute paths (the offending/offended library is static);
• libiconv.dylib from Macports mangles function names, but preserves the same library name.

The benefit of having those [(presumably executable) tools] wrapped is that you have a single way to update them and a single interface to manage them. I don't want to have to care about five different language-specific package managers when all I want is a single end-user tool

I don't see many Haskell-based tools that would qualify, and thus make what you describe worthwhile. But then, I cannot count myself as a significant contributor to Macports - so I defer to you here. Do as you see fit.

I agree there's not a big difference for somebody who develops in the Haskell ecosystem, but those people really don't need our help in figuring this out anyway

Ideally, a developer would download Haskell Platform and go from there. Because that platform conflicts with Macports (see above), the options for the developer are to

• replace Macports with Brew, which does not suffer from this problem;
• rebuild GHC from sources to make it Macports-compatible (that's what I finally did, exhausting other options);
• make Macports libiconv.dylib compatible - make a variant that doesn't mangle names (this request was shot down);
• get Macports GHC (considering how old it is, and the likelihood that Macports-installed Haskell packages would conflict with the newer ones makes this completely unattractive).

P.S. Factor in the fact that the current Haskell build tools (cabal and stack) are moving towards local snapshot-based approach - no global database of packages, registries at best per-user, or more likely - per-project. I don't see all the consequences of this, but it doesn't seem to play well with the Macports approach to global (per-machine) package repository.

[rseichter (Ralph Seichter)]

GHC build log

[rseichter (Ralph Seichter)]

Quoting my own post (​https://lists.macports.org/pipermail/macports-dev/2019-April/040509.html) because unfortunately there was no response:

I wanted to take a shot at updating the Pandoc port from version 1.12 to
2.7.1, but the GHC prerequisite build fails.

  :info:build deriveConstants: fd:31: hClose: invalid argument (Bad file descriptor)
  :info:build make[1]: *** [includes/dist-derivedconstants/header/DerivedConstants.h] Error 1

I found https://trac.macports.org/ticket/48899 but I am unsure if I
should add the build log file there or if a new ticket would be better?
I am also wondering about the chances of actually finding somebody
willing to work on Haskell, seeing that the existing ticket was filed
four years ago? Personally, I don't know enough about GHC.

-Ralph

P.S.: I know there is a Pandoc installer for macOS available, but it
looks like conversion to PDF does not work out of the box.

I have now attached the GHC build log to this ticket.

[mouse07410 (Mouse)]

My personal opinion - the goals that Macports GHC port maintainers set for it, are not attainable in practice - which is why the "upgrade" request is four years stale. Complete lack of API stability in Haskell ecosystem is another big factor why it's so difficult to maintain a decent port.

[mojca (Mojca Miklavec)]

Any pull requests or volunteers to bring Haskell up to date again are more than welcome.

I talked on their IRC channel a few days ago. What they would like to see is, say, the latest three versions available in parallel (similar to what we offer with python / perl / ruby). I did some testing, but there is quite a bunch of stuff that needs work in order to bring even a single one up to date, plus everything that follows with respect to individual packages that all need an update as well.

[rseichter (Ralph Seichter)]

The latest three versions? Seems like wishful thinking to me. I'd be glad having just the latest Haskell version available, but like I said, I don't know enough about it. Add to that a limited amount of time, and the fact that Haskell is certainly exotic, I'm afraid it would take a true enthusiast. Personally, I only care about Haskell because it is a requirement for Pandoc, and on an academic level because I dabbled with functional programming at the university, long time ago.

[mouse07410 (Mouse)]

One big difference between Python and Ruby on one hand, and Haskell on the other - is, as I already said more than once, the (almost) complete disregard for API stability by (too many of) Haskell package developers. This makes what you want to accomplish extremely difficult, IMHO - which probably is why you don't see a long line of volunteers to step in.

[neverpanic (Clemens Lang)]

An update to GHC 8.6 was merged in ​https://github.com/macports/macports-ports/pull/4794. The rest of the ports in MacPorts is now a bit broken, but it should be much easier to fix now, and there are already some further PRs open for review. I'm closing this for now.

I'm also dropping maintainership of ghc in ​https://github.com/macports/macports-ports/pull/5049.