[pandoc]: configure step fails with "missing dependencies..."

[KurtPfeifle (Kurt Pfeifle)]

I cannot upgrade pandoc on my Mavericks system. This is the relevant extract of "port log pandoc":

DEBUG: Executing command line:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.org_release_ports_textproc_pandoc/pandoc/work/pandoc-1.12.2.1" && runhaskell Setup configure --prefix=/opt/local --with-compiler=/opt/local/bin/ghc -v --enable-library-profiling --with-gcc=/usr/bin/clang 
Configuring pandoc-1.12.2.1...
Setup: At least the following dependencies are missing:
HTTP >=4000.0.5 && <4000.3,
data-default >=0.4 && <0.6,
highlighting-kate >=0.5.5 && <0.6,
network >=2 && <2.5,
random >=1 && <1.1,
temporary ==1.1.*,
zip-archive >=0.1.3.3 && <0.2
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_lil.fr.rsync.macports.org_release_ports_textproc_pandoc/pandoc/work/pandoc-1.12.2.1" && runhaskell Setup configure --prefix=/opt/local --with-compiler=/opt/local/bin/ghc -v --enable-library-profiling --with-gcc=/usr/bin/clang 
Exit code: 1
Error: org.macports.configure for port pandoc returned: configure failure: command execution failed
DEBUG: Error code: NONE
DEBUG: Backtrace: configure failure: command execution failed
Warning: targets not executed for pandoc: org.macports.install org.macports.configure org.macports.build org.macports.destroot

However:

kp@mb: > port installed hs-http hs-data-default hs-highlighting-kate hs-network hs-random hs-temporary hs-zip-archive
The following ports are currently installed:
  hs-data-default @0.5.3_1 (active)
  hs-highlighting-kate @0.5.5_1 (active)
  hs-http @4000.2.8_4 (active)
  hs-network @2.4.1.2_2 (active)
  hs-random @1.0.1.1_3 (active)
  hs-temporary @1.1.2.4_3 (active)
  hs-zip-archive @0.1.3.4_4 (active)

[mf2k (Frank Schima)]

Please attach the complete main.log file after cleaning the port. In the future, please fill in the Port field.

[KurtPfeifle (Kurt Pfeifle)]

output of "port log pandoc"

[KurtPfeifle (Kurt Pfeifle)]

Sorry -- had forgotten (again!) to fill in the Port field...

[KurtPfeifle (Kurt Pfeifle)]

main.log (after "port clean pandoc && port upgrade pandoc"

[neverpanic (Clemens Lang)]

Same here, please run sudo ghc-pkg recache and provide the output of ghc-pkg list and ghc-pkg check.

[KurtPfeifle (Kurt Pfeifle)]

Output of "ghc-pkg list"

[KurtPfeifle (Kurt Pfeifle)]

Output of "ghc-pkg check"

[KurtPfeifle (Kurt Pfeifle)]

Ok, these logs proof there's a problem with my installed packages. But they leave me clueless about how to fix it. (I have no idea about Haskell/ghc/cabal packages.)

[anddam (Andrea D'Amore)]

Cc Me!

[J.Gilbey@…]

Well, I'm still trying to build pandoc on OS X Yosemite, and it's proving to be very tricky indeed.

This is a log of how far I've got and where I've got stuck:

The current pandoc port no longer builds, as it depends on an out-of-date hs-text and other packages which have not been ported.

So I tried updating packages one at a time to get pandoc to build - until I hit a brick wall.

I have applied the patches described in tickets #46464, #46484, #46485, #46486, #46487 and #46488.

The following new packages are also required, and Portfiles are attached to this report:

hs-juicypixels ​http://hackage.haskell.org/package/JuicyPixels-3.2.2/JuicyPixels-3.2.2.tar.gz

hs-scientific ​http://hackage.haskell.org/package/scientific-0.3.3.5/scientific-0.3.3.5.tar.gz

hs-aeson: A tentative Portfile diff is

--- Portfile.orig	2015-01-08 14:16:08.000000000 +0000
+++ Portfile	2015-01-08 14:17:55.000000000 +0000
@@ -4,10 +4,9 @@
 PortSystem          1.0
 PortGroup           haskell 1.0
 
-haskell.setup       aeson 0.6.2.1
-revision            3
-checksums           rmd160  ac3d7bb17f8110b2deb2633896b5c00c3cfa4b73 \
-                    sha256  cbcee6956d0a3cfc45073b6c160e9c412c3dfad0f36ec71d6044a391f108ca01
+haskell.setup       aeson 0.8.0.2
+checksums           rmd160  31f600250578bc8ad022ff22a2723ef10337a3b4 \
+                    sha256  0707588dfb5fdfe787eba5b3d5a9950acb224a8dae9dcdcfc9c974ae2b6788d5
 
 maintainers         cal openmaintainer
 platforms           darwin

But then I got stuck, because the following packages included in haskell-platform are too old and need to be updated:

hs-attoparsec: The version in haskell-platform is too old for the current aeson (which is needed) to build :-( A tentative Portfile for a stand-alone port is attached.

hs-text: The version in haskell-platform is too old for the current hs-attoparsec. Portfile for stand-alone version attached.

But when I try building hs-attoparsec using the new version of hs-text, it dies because it's trying to look for the old version, or something like that - log files attached.

I don't know what to do now :-(

Julian

[J.Gilbey@…]

main.log from attempt to build attoparsec

[J.Gilbey@…]

Running the ghc command with -v on attoparsec build tree

[J.Gilbey@…]

Cc Me!

[neverpanic (Clemens Lang)]

The tickets you mentioned have been fixed. I've also committed hs-juicypixels in r131309.

Rather than updating packages in the platform, we can actually provide the same package in a different (pandoc-compatible) version. See for example the `hs-tagsoup-0.12`​ package that provides an older version of the hs-tagsoup port. Since haskell allows installing arbitrary versions of packages side-by-side this kind of thing is no problem and would be my preferred solution in this case.

I couldn't yet figure out why your compilation of hs-attoparsec uses the wrong version of hs-text. I will continue to look into this tomorrow.

[joelfred@…]

Replying to J.Gilbey@…:

But when I try building hs-attoparsec using the new version of hs-text, it dies because it's trying to look for the old version, or something like that - log files attached.

I don't know what to do now :-(

Julian

It looks like you need to recompile hs-scientific against text-1.2.0.3, and you also need something equivalent to hs-hashable, which may need to be upgraded. I uninstalled and reinstalled hs-scientific and I got one less error. I'm going to update hs-hashable...

[joelfred@…]

And hs-unordered-containers seems to have the same problem...

[joelfred@…]

Note to Cal: would it be better to put these things into haskell-platform???

My log of things I've made:

---

New packages required (Port files following):

hs-hashable ​https://hackage.haskell.org/package/hashable-1.2.3.1/hashable-1.2.3.1.tar.gz

hs-unordered-containers ​https://hackage.haskell.org/package/unordered-containers-0.2.5.1/unordered-containers-0.2.5.1.tar.gz

hs-haddock-library ​https://hackage.haskell.org/package/haddock-library-1.1.1/haddock-library-1.1.1.tar.gz

hs-parsec ​https://hackage.haskell.org/package/parsec-3.1.7/parsec-3.1.7.tar.gz

---

Updates required:

hs-conduit ​https://hackage.haskell.org/package/conduit-1.2.3.1/conduit-1.2.3.1.tar.gz

hs-parsec

hs-tagsoup

hs-texmath

hs-yaml

hs-zip-archive

---

Need to be recompiled against new packages (I may have missed a few):

hs-xml

[joelfred@…]

Okay, I'm attaching a tar of my local ports. Everything (and I mean EVERYTHING) needs to be checked. I don't know how to force things with dependencies to recompile when their dependencies are updated- that might have made my life a lot easier.

But pandoc appears to be working.

[mamoll (Mark Moll)]

Cc Me!

[mkae (Marko Käning)]

Cc Me!

[joelfred@…]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #46508.

[jowens@…]

Cc Me!

[neverpanic (Clemens Lang)]

I think pandoc should be fixed with r131411.

This was more complicated than I anticipated, and I chose not to update everything to the latest version due to the problem with the text package. While we could ship a separate, newer version of text, a package that depends on two different versions of text (e.g. along two separate paths in its dependency tree) is considered broken and triggers a warning that you shouldn't do that and it could lead to compile failures.

So there were basically two ways to tackle this problem:

• Ignore what the current Haskell Platform does, update hs-text and rebuild all its dependencies against the newer version (which would probably require updating them, in turn). That would make the haskell-platform port pointless.
• Package the latest versions that will still work with whatever is currently in the platform. I went with this route.

Please test whether this works for you. Please also comment on whether pandoc 1.12.4.2 is new enough for you, or whether you'd really, really want version 1.13.*.

If you encounter any problems with pandoc, please re-open this report.

[joelfred@…]

There's a third option- update Haskell-platform, right?

But I'm not that invested in all of this, I'm sure whatever solution you come up with will be just fine!

[J.Gilbey@…]

Replying to joelfred@…:

There's a third option- update Haskell-platform, right?

But I'm not that invested in all of this, I'm sure whatever solution you come up with will be just fine!

haskell-platform is already at its latest upstream version (2014.2), so it can't currently be "updated" against upstream.

Thanks for all your work on this! Really appreciated!

[J.Gilbey@…]

Replying to cal@…:

I think pandoc should be fixed with r131411.

This was more complicated than I anticipated, and I chose not to update everything to the latest version due to the problem with the text package. While we could ship a separate, newer version of text, a package that depends on two different versions of text (e.g. along two separate paths in its dependency tree) is considered broken and triggers a warning that you shouldn't do that and it could lead to compile failures.

So there were basically two ways to tackle this problem:

• Ignore what the current Haskell Platform does, update hs-text and rebuild all its dependencies against the newer version (which would probably require updating them, in turn). That would make the haskell-platform port pointless.
• Package the latest versions that will still work with whatever is currently in the platform. I went with this route.

Please test whether this works for you. Please also comment on whether pandoc 1.12.4.2 is new enough for you, or whether you'd really, really want version 1.13.*.

If you encounter any problems with pandoc, please re-open this report.

Wow, Cal, thank you!

I'm going to try a clean install of MacPorts and see whether this works - I'll let you know the results!

Julian

[J.Gilbey@…]

Oh, it didn't work. During the install, for some reason ghc 7.8.3_1 was built from source rather than being downloaded (took about an hour), and then various of the hs-* packages didn't install correctly. Attached is the trimmed output from restore_ports and the two relevant logs referred to. :-(

[J.Gilbey@…]

Installation failure on fresh MacPorts installation

[KurtPfeifle (Kurt Pfeifle)]

"Please also comment on whether pandoc 1.12.4.2 is new enough for you, or whether you'd really, really want version 1.13.*."

Ok, here goes...

The current version of Pandoc is 1.13.2. Since 12.2.1 (my currently installed version on Mac -- upgrade to 1.12.4.2_0 failed!) there have been numerous bug fixes in all reader and all writer modules as well as improvements to all templates, plus these very significant feature additions to Pandoc:

* Addition of InDesign ICML writer
* Addition of Emacs org-mode reader
* Addition of Emacs org-mode writer
* Addition of haddock writer
* Addition of DokuWiki writer
* Addition of Txt2Tags reader
* Addition of DOCX reader
* Addition of EPUB reader
* Addition of TWiki reader

The specific additional features why I really, really want version 1.13 are: Emacs org-mode reader+writer, the EPUB reader and the DOCX reader. (That I want the bugfixes too, plus the improvements to the templates, should be obvious.)

[jowens@…]

Unfortunately, I'm in the same sort of boat as J.Gilbey ... both pandoc and haskell-platform installs fail because the set of hs-* packages appears to be collectively incompatible. With no hs-* packages installed, but using the latest ghc:

haskell-platform fails on hs-random, which lacks dependencies "base-4.7.0.1-c64d224738ec7af4085e89ca9c12c37b" and "time-1.4.2-bf925e935c287d0b75398fe297453c28".

pandoc fails on hs-text, which lacks dependencies "base-4.7.0.1-c64d224738ec7af4085e89ca9c12c37b", "bytestring-0.10.4.0-18fe2f3ce284617c82da1702e16772cf", and "deepseq-1.3.0.2-733fe43e121f761739636bd6670bea77".

[anddam (Andrea D'Amore)]

Replying to jowens@…:

Unfortunately, I'm in the same sort of boat as J.Gilbey ... both pandoc and haskell-platform installs fail because the set of hs-* packages appears to be collectively incompatible. With no hs-* packages installed, but using the latest ghc:

I stumbled in this very same issue while upgrading pandoc, I eventually was able to install pandoc by removing all the installed hs- packages, deactivating the ghc port that was installed and active, removing ghc's packages.conf.d directory and then reactivating ghc. This should reinstall the correct package cache.

Here's what I did, if you want to try this as well be sure to understand the commands, you're the sole responsible for the commands your run:

• upgrade ghc
• uninstall pandoc and all hs-* ports using port uninstall pandoc name:hs- (if you have other ports depending on hs-* port you'll have to uninstall/deactivate those as well)
• deactivate ghc port deactivate ghc
• locate and delete ghc's package.conf.d, should be at $prefix/lib/ghc-7.8.3/package.conf.d, I deleted a stray ghc-7.6.3 folder there as well
• activate ghc port activate ghc
• install pandoc that should now install all required hs-* ports and upgraded pandoc

I ended up with a working pandoc @1.12.4.2_0 and all its dependencies, before this hs-sha and hs-zip-archive were choking on bad packages in cache.

YMMV.

[jowens@…]

Replying to and.damore@…:

Fantastic! and.damore, this totally solved my problem. Much appreciated.

[mkae (Marko Käning)]

I followed the above workaround procedure, but installing pandoc still fails on Mavericks due to a failing port 'hs-sha' caused by 'binary-0.7.1.0':

:info:build /opt/local/bin/ghc --make -fbuilding-cabal-package -O -static -dynamic-too -dynosuf dyn_o -dynhisuf dyn_hi -outputdir dist/build -odir dist/build -hidir dist/build -stubdir dist/build -i -idist/build -isrc -idist/build/autogen -Idist/build/autogen -Idist/build -optP-DINCLUDE_DECODER_INTERFACE -optP-include -optPdist/build/autogen/cabal_macros.h -package-name SHA-1.6.4.1 -hide-all-packages -no-user-package-db -package-db dist/package.conf.inplace -package-id array-0.5.0.0-98aa445e59f3eb0c886795ff07406d84 -package-id base-4.7.0.1-c64d224738ec7af4085e89ca9c12c37b -package-id binary-0.7.1.0-5b9c0c97f623eee41f8b0bd573b8186c -package-id bytestring-0.10.4.0-18fe2f3ce284617c82da1702e16772cf -XHaskell98 -XCPP -XBangPatterns Data.Digest.Pure.SHA -Wall -fno-ignore-asserts -fno-warn-orphans -funbox-strict-fields -fwarn-tabs
:info:build <command line>: cannot satisfy -package-id binary-0.7.1.0-5b9c0c97f623eee41f8b0bd573b8186c
:info:build     (use -v for more information)
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_hs-sha/hs-sha/work/SHA-1.6.4.1" && runhaskell Setup build -v
:info:build Exit code: 1
:error:build org.macports.build for port hs-sha returned: command execution failed
:debug:build Error code: CHILDSTATUS 32612 1
:debug:build Backtrace: command execution failed

Looks like hs-binary is a missing dependency here!

[anddam (Andrea D'Amore)]

Replying to jowens@…:

Fantastic! and.damore, this totally solved my problem. Much appreciated.

This actually was Clemens' (the owner of this ticket) suggestion.

Replying to mk@…:

I followed the above workaround procedure, but installing pandoc still fails on Mavericks due to a failing port 'hs-sha' caused by 'binary-0.7.1.0':

:info:build <command line>: cannot satisfy -package-id binary-0.7.1.0-5b9c0c97f623eee41f8b0bd573b8186c

Looks like hs-binary is a missing dependency here!

A few packages considered to be core now are builtin in the ghc port while earlier they were separated, binary is one of them.

Port ghc is provided as binary package and it contains binary-0.7.1.0-06fe2cfe2c00e6f7d43cdb53345c4945.conf, the fact that your setup sees the other hash checksum means it's not a clean install.

Obviously I cannot see what commands you ran and what output you get but I suggest trying again the whole shebang ensuring that you only have ghc-7.8.3 in $prefix/lib and that you erase directory packages.conf.d after deactivating the port. The point is to let the cache files provided by ghc port to be put in place.

FWIW I had the very same hash mismatch for hs-binary that you reported.

[mkae (Marko Käning)]

I see. OK, but I did follow your procedure step by step, exactly as outlined. Will try again tomorrow.

[neverpanic (Clemens Lang)]

It's entirely possible I still need to make some changes to some of the ports to force rebuilds where necessary. The hard part is figuring out which of those need the changes. For example, if a port previously built against the incorrect hs-binary (which is now gone) and is being downloaded as binary, it needs to be revbumped.

[akimd (Akim Demaille)]

Cc Me!

[gnw3]

The 6-step program worked for me on Mavericks and Snow Leopard systems, but only when I used "install -s pandoc", so there are still problems with the binary archives. I did try rebuilding only the "hs-XX" packages that configure reported as missing, but it went on and on, so it seems best just to rebuild everything pandoc needs.

Would it make sense to revbump all the packages needed by pandoc and then deal with remaining problems as they surface:

$ sudo port -s install pandoc
--->  Computing dependencies for pandoc
--->  Dependencies to be installed: hs-aeson hs-attoparsec hs-text hs-blaze-buil der hs-dlist hs-hashable hs-mtl hs-scientific 
hs-syb hs-unordered-containers hs- vector hs-primitive hs-alex hs-quickcheck hs-random hs-base64-bytestring hs-blaze-html 
hs-blaze-markup hs-data-default hs-data-default-class hs-data-default-instances-base hs-data-default-instances-containers 
hs-data-default-instances-dlist  hs-data-default-instances-old-locale hs-extensible-exceptions hs-happy hs-highlighting-kate 
hs-parsec hs-pcre-light hs-utf8-string hs-hslua hs-http hs-network  hs-pandoc-types hs-deepseq-generics hs-sha hs-tagsoup 
hs-temporary hs-exceptions  hs-texmath hs-xml hs-yaml hs-conduit hs-lifted-base hs-base-unicode-symbols hs-monad-control 
hs-transformers-base hs-mmorph hs-resourcet hs-void hs-semigroups  hs-nats hs-enclosed-exceptions hs-async hs-stm 
hs-zip-archive hs-digest hs-zlib

[manuel.deljesuspe@…]

Cc Me!

[neverpanic (Clemens Lang)]

Yes, that would make sense. Can you provide a patch?

[gnw3]

Replying to cal@…:

Yes, that would make sense. Can you provide a patch?

Not immediately -- now that pandoc is working for me I have more pressing problems (like broken 4-years-old MBP that I have been using for this sort of thing) . I'm not familiar with the subports stuff so it will take a while to get my head around it -- normally I'd just copy the required ports tree, run a script to massage the revision numbers, then test the changes locally on a non-critical box.

[neverpanic (Clemens Lang)]

I think r131981 should finally fix these issues. Let's see what the buildbots say…

Note that you will see warnings along the lines of "failed to execute Portfile from registry" during the upgrade when deactivating the old versions. These warnings are expected but harmless and could unfortunately not be avoided with this change.

I'm keeping this open to explore how pandoc could be updated without leaving haskell platform.

[mkae (Marko Käning)]

Yes, I can verify herewith that this fixed it. cal, thanks for your hard work on this major upgrade for all the haskell ports!!!