gcc5 @5.2.0: fix gcj

[howarth.at.macports@…]

The attached Portfile.diff, gcj-libiconv-linkage.patch and boehm-gc-darwin15-hack.patch corrects the build of gcj so that it is functional. The changes include...

1. Pass -with-libiconv-prefix=${prefix} to configure so that the proper libiconv is used (as the Portfile has depends_lib port:libiconv.
2. Reduce alignment in boehm-gc to 2 with boehm-gc-darwin15-hack.patch in order to suppress breakage from the recompilation of the system libunwind.dylib against Apple Clang 7.0 in darwin15.
3. Apply gcj-libiconv-linkage.patch to provide the appropriate linkage on libiconv automatically for executables created with gcj.
4. In order to create ecj1 during the build, ecj-4.9.jar is added to distfiles and copied into the source directory as ecj.jar per upstream's build instructions for gcj.

The functionality of the resulting gcj-mp-5 can be seen by compiling the testme.java source code....

public class testme { 
  public static void main(String args[]){ 
    System.out.println("Hello"); 
  } 
} 

using the command "gcj-mp-5 --main=testme -O testme.java" which will produce a functional a.out executable.

[howarth.at.macports@…]

patch to eliminate boehm-gc testsuite regressions against darwin15's system unwinder

[howarth.at.macports@…]

patch to provide missing linkage for libiconv on the gcj spec file

[howarth.at.macports@…]

Note that I have also updated #48528 to show that the proposed gcc5 update can be used to fix the build of the pdftk package.

[jeremyhu (Jeremy Huddleston Sequoia)]

We should also blacklist gcc49 and earlier on El Cap while we're at it.

[petrrr]

Cc Me!

[m7.thon@…]

Cc Me!

[howarth.at.macports@…]

Replying to jeremyhu@…:

We should also blacklist gcc49 and earlier on El Cap while we're at it.

Why bother? I don't see earlier gcc releases (gcc48, etc) backlisted in the existing Portfile. Anything else blocking this being committed?

[howarth.at.macports@…]

Ping. The gcc packages haven't have a functional gcj (due to the absence of building against ecj.jar to produce the missing ecj1 executable necessary to compile java files) since gcc46 or earlier.

[ryandesign (Ryan Carsten Schmidt)]

Can you post the link to the corresponding upstream gcc bug report? I'd like to verify that this is the correct fix, and that they plan to or have already committed similar.

[graziosi.angelo@…]

Replying to ryandesign@…:

Can you post the link to the corresponding upstream gcc bug report? I'd like to verify that this is the correct fix, and that they plan to or have already committed similar.

Since the upgrade to OSX 10.11, the users have lost PDFtk and the fix proposed by Howarth seems to fix this. I don't understand why we are still waiting. At worst we could always revert it, but I am sure that Howarth has done the right things and that sooner or later everyone has to go this route...

[jeremyhu (Jeremy Huddleston Sequoia)]

graziosi.angelo, you don't understand potential fallout to compiler changes.

At worst, the compiler produces bad code and anything that it compiled needs to be recompiled. We don't have a mechanism for triggering that, so we need to be extra careful taking toolchain changes.

If upstream doesn't approve it, we should be even more cautious.

[graziosi.angelo@…]

Replying to jeremyhu@…:

graziosi.angelo, you don't understand potential fallout to compiler changes.

At worst, the compiler produces bad code and anything that it compiled needs to be recompiled. We don't have a mechanism for triggering that, so we need to be extra careful taking toolchain changes.

Exactly what happens at (almost) every new compiler release. Recently, MSYS2/MINGW64, for having adopted GCC-5.2, had to recompile *all* its packages, and a few didn't rebuilt any more and are out of distribution.. On OSX and friends, things are no better..

If upstream doesn't approve it, we should be even more cautious.

[howarth.at.macports@…]

Replying to jeremyhu@…:

graziosi.angelo, you don't understand potential fallout to compiler changes.

At worst, the compiler produces bad code and anything that it compiled needs to be recompiled. We don't have a mechanism for triggering that, so we need to be extra careful taking toolchain changes.

If upstream doesn't approve it, we should be even more cautious.

I wouldn't hold my breath for a fix from the FSF gcc developers on this issue. The libjava development in FSF gcc (of which boehm-gc is part) is pretty much mothballed. There were vague preliminary discussions of upgrading the boehm-gc in FSF gcc to 7.2 from the current 6.6 back in 2011 which never went anywhere...

​http://www.hpl.hp.com/hosted/linux/mail-archives/gc/2011-April/004516.html

FYI, I don't expect upstream to accept this alignment hack as it is specific to darwin's use of a clang 3.7 or later compiled unwinder. Also note the comment in boehm-gc/include/private/gcconfig.h...

• ALIGNMENT is the largest N, such that
• all pointer are guaranteed to be aligned on N byte boundaries.
• defining it to be 1 will always work, but perform poorly.

Since the actual error seen in darwin15 is ....

* thread #1: tid = 0x20dbb8, 0x00007fff93f37148 libdyld.dylib`dyld_stub_binder, queue = 'com.apple.main-thread', stop reason = instruction step into
    frame #0: 0x00007fff93f37148 libdyld.dylib`dyld_stub_binder
libdyld.dylib`dyld_stub_binder:
->  0x7fff93f37148 <+0>:  pushq  %rbp
    0x7fff93f37149 <+1>:  testq  $0xf, %rsp
    0x7fff93f37150 <+8>:  jne    0x7fff93f372da            ; stack_not_16_byte_aligned_error

setting the ALIGNMENT to 2 simply forces everything on a 16-bit alignment and avoids the problem with a reduction in performance. Considering that we don't even have a usable gcj on *ANY* gcc past 4.6 and that the proposed changes produce *no* regressions in the boehm-gc and libjava test suites, I can't see any reason not to use these changes for now.

[howarth.at.macports@…]

Note that at the absolute very least you should be using the changes to download the ecj.jar file and place it in the source directory. Otherwise you build a crippled gcj which is incapable for processing java files and can only operate on precompiled class files. This isn't particularly well documented in FSF gcc. The changes were introduced in...

​https://gcc.gnu.org/ml/java-patches/2006-q4/msg00061.html

and require the build to either...

1. Place the ecj.jar in the source directory before starting the build
2. Pass the location of a preinstalled ecj.jar file to configure with the --with-ecj-jar= libjava configure option

The omission of building ecj1 (due to the absence of an ecj.jar) unfortunately isn't caught by the FSF gcc test suite due to their use of classes to test gcj rather than java source files.

[howarth.at.macports@…]

Also...

​https://gcc.gnu.org/ml/java/2014-08/msg00011.html

is the clearest description of the status of the libjava support posted in recent history.

[howarth.at.macports@…]

Long term, if you don't like the situation with gcj, you can simply stop building the java lang support in the FSF gcc packages. In general, packages that need gcj can be configured to use Oracle's java in general although quite a few configure scripts will need to be hacked as many of them don't honor changing the java compiler and will still try to use gcj.

[jeremyhu (Jeremy Huddleston Sequoia)]

Replying to graziosi.angelo@…:

Replying to jeremyhu@…:

graziosi.angelo, you don't understand potential fallout to compiler changes.

At worst, the compiler produces bad code and anything that it compiled needs to be recompiled. We don't have a mechanism for triggering that, so we need to be extra careful taking toolchain changes.

Exactly what happens at (almost) every new compiler release. Recently, MSYS2/MINGW64, for having adopted GCC-5.2, had to recompile *all* its packages, and a few didn't rebuilt any more and are out of distribution.. On OSX and friends, things are no better..

No, that's certainly not true. Some people may *choose* to recompile with newer compilers because they want to. That doesn't mean that the older compiled code was wrong.

We are careful here because we don't want to release compilers that produce incorrect binaries.

Replying to howarth.at.macports@…:

Also...

​https://gcc.gnu.org/ml/java/2014-08/msg00011.html

is the clearest description of the status of the libjava support posted in recent history.

Given that, it would seem best to just stop producing gcj.

[howarth.at.macports@…]

Replying to jeremyhu@…:

Replying to graziosi.angelo@…:

Replying to jeremyhu@…:

graziosi.angelo, you don't understand potential fallout to compiler changes.

At worst, the compiler produces bad code and anything that it compiled needs to be recompiled. We don't have a mechanism for triggering that, so we need to be extra careful taking toolchain changes.

Exactly what happens at (almost) every new compiler release. Recently, MSYS2/MINGW64, for having adopted GCC-5.2, had to recompile *all* its packages, and a few didn't rebuilt any more and are out of distribution.. On OSX and friends, things are no better..

No, that's certainly not true. Some people may *choose* to recompile with newer compilers because they want to. That doesn't mean that the older compiled code was wrong.

We are careful here because we don't want to release compilers that produce incorrect binaries.

Replying to howarth.at.macports@…:

Also...

​https://gcc.gnu.org/ml/java/2014-08/msg00011.html

is the clearest description of the status of the libjava support posted in recent history.

Given that, it would seem best to just stop producing gcj.

Dropping gcj entirely will break the build of any packages which require the ability to compile java into stand-alone executables (which is a functionality limited to gcj and not present in Oracle's java which can only compile to classes).

[ryandesign (Ryan Carsten Schmidt)]

Jack, thank you for the work you have done on this.

To explain why we have not committed this yet: MacPorts does not want to be in the business of forever maintaining patches to other people's software. Therefore, what we usually want is for patches to be submitted to the developers of the software. If they accept it, then we can update the MacPorts port to the fixed version, or apply the patch until they release a fixed version. If they do not accept the patch, then we would want to understand why they rejected it, and consider whether that is grounds for rejecting it in MacPorts as well.

If the developers no longer exist or do not care about the software enough to comment on the proposed changes, then that leaves us with having to evaluate the patches ourselves and making a decision. I am not an expert on the inner workings of the GCC package and am not sure I'm qualified to evaluate the correctness of your changes.

With regard to needing to download an install the ecj.jar, yes, I'm aware of that; we do it in the gcc45 and gcc47 ports already. Ideally, we would do it in all the gcc ports, in the same way (see #22066); there should not be unnecessary differences between the gcc ports.

[Ronny.Schuetz@…]

Cc Me!

[pmetzger (Perry E. Metzger)]

Ryan, is there any movement on this topic? I bring it up because pdftk is broken as a result...

[pmetzger (Perry E. Metzger)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

I am not working on this. I know pdftk is broken.

[pmetzger (Perry E. Metzger)]

Well, if you're not working on it, either this could get closed and the ports that require it to be present should be removed as unusable, or someone else could be recruited to work on it. I'd suggest that, given that someone has produced the patch, that it just be committed for now if we have no better solution?

[howarth.at.macports@…]

There was some vague talk upstream about updating the boehm-gc in FSF gcc but I suspect we won't see that before gcc 7.

[pmetzger (Perry E. Metzger)]

GCC 7 is some time away.

So again, it seems like the choices are:

1) Integrate the patch. ...or 2) Remove the ports that are broken because of the lack of it.

I don't think waiting another four or six months is going to produce better solutions.

[howarth.at.macports@…]

IMHO, I would integrate the patch but only for builds on darwin15 or later since only that platform has the problematic rebuild of libunwind.dylib with the newer llvm optimization that exposes the latent bug in the older boehm-gc.

[nortcele]

I support the latter proposal because I am eager to see pdftk work again.

[nortcele]

Cc Me!

[nortcele]

Replying to howarth.at.macports@…: …

4. In order to create ecj1 during the build, ecj-4.9.jar is added to distfiles and copied into the source directory as ecj.jar per upstream's build instructions for gcj.

…

According to the diff file, the ecj-4.9.jar file should be available in the same place as the gcc files but it is not. So the fetch phase fails.

--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.lip6.fr/pub/gnu/gcc/gcc-5.3.0   
--->  Attempting to fetch ecj-4.9.jar from http://lil.fr.distfiles.macports.org/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://nue.de.distfiles.macports.org/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://www.mirrorservice.org/sites/ftp.gnu.org/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from https://distfiles.macports.org/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://mse.uk.distfiles.macports.org/sites/distfiles.macports.org/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://fco.it.distfiles.macports.org/mirrors/macports-distfiles/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://osl.no.distfiles.macports.org/gcc5
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.funet.fi/pub/mirrors/sources.redhat.com/pub/gcc/releases/gcc-5.3.0/
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.funet.fi/pub/gnu/prep/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/gcc5
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.gnu.org/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from http://ftp.gnu.org/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.gnu.org/old-gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from http://mirrors.ibiblio.org/gnu/ftp/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from http://mirror.facebook.net/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from http://sea.us.distfiles.macports.org/macports/distfiles/gcc5
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.gwdg.de/pub/linux/gcc/releases/gcc-5.3.0/
--->  Attempting to fetch ecj-4.9.jar from ftp://gcc.gnu.org/pub/gcc/releases/gcc-5.3.0/
--->  Attempting to fetch ecj-4.9.jar from http://jnb.za.distfiles.macports.org/distfiles/gcc5
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.unicamp.br/pub/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from http://jog.id.distfiles.macports.org/macports/distfiles/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://cjj.kr.distfiles.macports.org/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://mirror.internode.on.net/pub/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from http://aarnet.au.distfiles.macports.org/pub/macports/mpdistfiles/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://nou.nc.distfiles.macports.org/pub/macports/distfiles.macports.org/gcc5
--->  Attempting to fetch ecj-4.9.jar from ftp://gcc.ftp.nluug.nl/mirror/languages/gcc/releases/gcc-5.3.0/
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.kddlabs.co.jp/pub/gnu/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.kddlabs.co.jp/pub/gnu/old-gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.dti.ad.jp/pub/GNU/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from ftp://ftp.informatik.hu-berlin.de/pub/gnu/gnu/gcc/gcc-5.3.0
--->  Attempting to fetch ecj-4.9.jar from http://her.gr.distfiles.macports.org/gcc5
--->  Attempting to fetch ecj-4.9.jar from http://svn.macports.org/repository/macports/distfiles/gcc5
Error: org.macports.fetch for port libgcc returned: fetch failed

[howarth.at.macports@…]

The ecj-4.9.jar file isn't kept on the ftp.gnu.org ftp server but at ​ftp://sourceware.org/pub/java/ecj-4.9.jar. Alternatively, you can get it from the gcc.gnu.org ftp server at ​ftp://gcc.gnu.org/pub/java/ecj-4.9.jar.

[nortcele]

I found that but I used the proposed patch as is. And that failed.

[howarth.at.macports@…]

Replying to Joel.Brogniart@…:

I found that but I used the proposed patch as is. And that failed.

Considering that the previous patch no longer applied to the current Portfile for gcc 5.3.0, I can only assume you incorrectly applied the sections manually. I've updated the Portfile.diff to apply cleanly to the current Portfile and it has no problems downloading the ecj-4.9.jar file.

[howarth.at.macports@…]

modify Portfile.diff to only apply boehm-gc-darwin15-hack.patch when building on darwin15 or later

[nortcele]

Thank you. With the new diff, ggc5 port could be installed and with it, pdftk could be installed too.

[mark.chilenski@…]

Cc Me!

[nortcele]

Could you please update your patch again for the new gcc5 portfile (5.3.0 revision 1)?

I don't have the skill to do it myself.

[harciga]

Cc Me!

[larryv (Lawrence Velázquez)]

Cc Me!

[nortcele]

Modify Portfile for GCC 5.4.0.

[ryandesign (Ryan Carsten Schmidt)]

FYI, gcj has been removed by the gcc developers as of gcc7 @7-20161002.

[pmetzger (Perry E. Metzger)]

Okay, so we know that this isn't going to be fixed "better" by a future gcc. So, either all ports dependent on gcj and java (like pdftk) should be removed and this should be closed, or the patch should be committed.

[jeremyhu (Jeremy Huddleston Sequoia)]

The former solution sounds more appropriate to me.

[mojca (Mojca Miklavec)]

See also #13553.

[nortcele]

Modify Portfile for GCC 5.4.0 with modification from Ticket #53726

[kencu (Ken)]

has duplicate #55033.

[nortcele]

Here is attachment:gcc5.5.0-java.diff​ a new patch to add java compilation to gcc 5.5.0. This patch should be used with attachment:boehm-gc-darwin15-hack.patch​.

[nortcele]

Patch to add java compilation to gcc5.5.0

[nortcele]

Oops, The attachment:gcc5.5.0-java.diff​ should be used with attachment:boehm-gc-darwin15-hack.patch​ and attachment:gcj-libiconv-linkage.patch​.

[kencu (Ken)]

Just so I'm clear I get it, you start with the current repo as it exists right now, with gcc5 at 5.4.0_2.

Then you upgrade to gcc 5.50 with the patch in #55068, then upgrade to gcc 5.5.0+ java with the gcc5.5.0-java.diff patch in this ticket, add the two extra files boehm-gc-darwin15-hack.patch and gcj-libiconv-linkage.patch from this ticket to the files directory, delete the old macosx-version-min.patch, and gcc5 is done. Once built, you now have gcc5.5.0 with java support.

Then you upgrade pdftk to support gcc5 with the patch from #48528.

There are a lot of tickets flying around, but I think that's it.

[kencu (Ken)]

I have made a PR <​https://github.com/macports/macports-ports/pull/915> with two commits to make this easier to follow and test, for anyone interested in doing so.

[ryandesign (Ryan Carsten Schmidt)]

Replying to ryandesign:

To explain why we have not committed this yet: MacPorts does not want to be in the business of forever maintaining patches to other people's software. Therefore, what we usually want is for patches to be submitted to the developers of the software. If they accept it, then we can update the MacPorts port to the fixed version, or apply the patch until they release a fixed version. If they do not accept the patch, then we would want to understand why they rejected it, and consider whether that is grounds for rejecting it in MacPorts as well.

If the developers no longer exist or do not care about the software enough to comment on the proposed changes, then that leaves us with having to evaluate the patches ourselves and making a decision. I am not an expert on the inner workings of the GCC package and am not sure I'm qualified to evaluate the correctness of your changes.

I think we should commit this after all. I would like to have a working pdftk in MacPorts on current macOS. The developer of pdftk has not responded to my inquiries into a way to build pdftk without gcj, so for now, gcj is required.

I still don't know much about the internals of gcc, but I feel OK about committing this small change, since it only changes alignment, and from what I understand about alignment, such a change should only affect performance, not correctness. So this will be a bit slower than ideal, but it should work, as opposed to the current situation, where the only gcj we have available (in gcc45/gcc47) does not work on El Capitan or later. If we restrict the patch only to where it's needed—on El Capitan and later, as the current proposal does—that will ensure that Yosemite and older still get full performance.

I had also been concerned that this change might affect, either in correctness or performance, aspects of gcc outside of gcj. But it seems the file being patched here, part of boehmgc, is only used for gcj, so it won't affect other parts of gcc. When gcj was removed from gcc in gcc7, the bundled copy of boehmgc was removed as well, so the patch is not something we'll have to maintain for new versions of gcc.

I had also wondered if this was still a bug in current boehmgc, since we have a standalone port for that. There's an ​upstream gcc bug report with much more discussion from Jack and others. It seems boehmgc fixed the problem a different way in an alpha version of boehmgc 7.2. (The version bundled with gcc5 is much older.) But the ​boehmgc commit Jack identified as possibly fixing the problem the "right" way is much larger, touching hundreds of lines, which I don't think I would dare to try to backport to the bundled copy in gcc, since the developers of gcc, who would be much more competent to do so, apparently didn't feel up to attempting that or updating to a newer boehmgc. But this answers my concern about whether we need a similar patch in the boehmgc port: we don't.

I still acknowledge Jeremy's concern that if the patch ends making a compiler that produces incorrect code, we have to rebuild everything that might have used the incorrect compiler. Even if that ends up being the case, it shouldn't be a problem to revbump affected ports, since the number of ports that use gcj can probably be counted on one hand.

The patch here doesn't add gcj support in the same way that is already present in the gcc47 and gcc45 ports. (For example, master_sites tags for the gcc source are missing.) I'll correct this before committing.

After committing this, hopefully we can do the same for gcc6, and maybe gcc49 and earlier as well. I tried backporting the patch to gcc47, but so far I get a build failure. We can look into that later if we feel like it. Or we could just turn off gcj on gcc49 and earlier on El Capitan and later, since gcc5 and gcc6 will be available.

[ryandesign (Ryan Carsten Schmidt)]

In e3b98f1eadf598515cc86648645256a03282705f/macports-ports:

gcc5: Update to 5.5.0 and fix gcj

Closes: #55068
Closes: #49227
See: #22066

Also use a common dist_subdir.

See: #50904