R: 4.1.2 fails to build on macOS 12.0.1 on M1 mac

[colmconn]

Hi R version 4.1.2 fails to build on a clean install of macports on macOS 12.0.1 on an M1 mac. The build log is attached. I'm not sure if the is a me issue or a port issue. Am happy to help in any way I can.

I ran sudo port selfupdate immediatly after installing macports to ensure everything was fresh. It reported no available upgrades.

Command to install R: sudo port install R

[colmconn]

R 4.1.2 build log

[reneeotten (Renee Otten)]

in the future, please make sure to add the port maintainer (port info --maintainer R)to Cc and add the port name to the "Port" field.

[ryandesign (Ryan Carsten Schmidt)]

Could you also attach /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_R/R/work/R-4.1.2/config.log?

We have successful builds on our buildbot on macOS 10.8.x through 12.x on x86_64 but failures on macOS 11 and 12 on arm64 so this may be an arm64-specific issue.

[colmconn]

config.log

[i0ntempest]

3368	configure:27349: checking whether mixed C/Fortran code can be run
3369	dyld[53169]: Library not loaded: @rpath/libgfortran.5.dylib
3370	  Referenced from: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_R/R/work/R-4.1.2/conftest
3371	  Reason: tried: '/libgfortran.5.dylib' (no such file)
3372	configure:27414: WARNING: cannot run mixed C/Fortran code
3373	configure:27416: error: Maybe check LDFLAGS for paths to Fortran libraries?

I think we've seen this somewhere?

[ryandesign (Ryan Carsten Schmidt)]

Yes, all over the place.

[tenomoto (Takeshi Enomoto)]

I was able to build R on my M1 MacBook by passing -Wl,-rpath ${prefix}/local/lib/libgcc if architecture is ARM. Created a pull request ​https://github.com/macports/macports-ports/pull/14926. tests/d-p-q-r-tst-2.R fails.

[kencu (Ken)]

gcc is supposed to be adding the rpath to it’s own lib folder to all executables by itself now.

Iain was recently quite certain he had it fixed to do so….

[kencu (Ken)]

Oh, I think I see. gcc11 on arm on MacPorts is not using Iain's gcc repo, it's using gcc-11.2 from FX's repo, which has not been updated with Iain's fixes.

And gcc-devel on arm is using Iain's gcc11 repo, but from a few months ago.

I'm going to build my self a current gcc11 using Iain's gcc11 repo on Intel and see if this rpath thing has been fixed there, before 1000 ports start adding the rpath manually.

[kencu (Ken)]

I built Iain's current gcc11 branch on 12.3.1 Intel. It uses @rpaths exclusively, and it works just fine.

example fortran program compiles with no errors:

 % cat hello.f90
program hello
    print *, "Hello World!"
end program

% gfortran-mp-11 hello.f90 

works

% ./a.out
 Hello World!

has @rpath linkages to gcc libraries as Iain wants and requires:

% otool -L a.out                    
a.out:
	@rpath/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	@rpath/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)

but automatically adds the needed rpath search path for gcc-built executables to find the libraries, as Iain said it did:

% otool -l a.out | grep LC_RPATH -A2
          cmd LC_RPATH
      cmdsize 32
         path @loader_path (offset 12)
--
          cmd LC_RPATH
      cmdsize 72
         path /opt/local/lib/gcc11/gcc/x86_64-apple-darwin21/11.3.0 (offset 12)
--
          cmd LC_RPATH
      cmdsize 80
         path /opt/local/lib/gcc11/gcc/x86_64-apple-darwin21/11.3.0/../../.. (offset 12)

% ls -la /opt/local/lib/gcc11/gcc/x86_64-apple-darwin21/11.3.0/../../..
total 40776
drwxr-xr-x    46 root  admin      1472 23 May 10:19 .
drwxr-xr-x  1438 root  wheel     46016 23 May 10:19 ..
drwxr-xr-x     3 root  admin        96 23 May 10:19 gcc
lrwxr-xr-x     1 root  admin        37 23 May 10:17 libasan.6.dylib -> /opt/local/lib/libgcc/libasan.6.dylib
lrwxr-xr-x     1 root  admin        15 23 May 10:17 libasan.dylib -> libasan.6.dylib
-rw-r--r--     1 root  admin      5600 23 May 10:17 libasan_preinit.o
lrwxr-xr-x     1 root  admin        39 23 May 10:17 libatomic.1.dylib -> /opt/local/lib/libgcc/libatomic.1.dylib
-rw-r--r--     1 root  admin    253552 23 May 10:17 libatomic.a
lrwxr-xr-x     1 root  admin        17 23 May 10:17 libatomic.dylib -> libatomic.1.dylib
-rwxr-xr-x     1 root  admin    155872 23 May 10:17 libcc1.0.so
lrwxr-xr-x     1 root  admin        11 23 May 10:17 libcc1.so -> libcc1.0.so
lrwxr-xr-x     1 root  admin        40 23 May 10:17 libgcc_s.1.1.dylib -> /opt/local/lib/libgcc/libgcc_s.1.1.dylib
lrwxr-xr-x     1 root  admin        38 23 May 10:17 libgcc_s.1.dylib -> /opt/local/lib/libgcc/libgcc_s.1.dylib
-rwxr-xr-x     1 root  admin  25405744 23 May 10:17 libgccjit.0.dylib
lrwxr-xr-x     1 root  admin        17 23 May 10:17 libgccjit.dylib -> libgccjit.0.dylib
lrwxr-xr-x     1 root  admin        41 23 May 10:17 libgfortran.5.dylib -> /opt/local/lib/libgcc/libgfortran.5.dylib
-rw-r--r--     1 root  admin  10333568 23 May 10:17 libgfortran.a
lrwxr-xr-x     1 root  admin        19 23 May 10:17 libgfortran.dylib -> libgfortran.5.dylib
-rw-r--r--     1 root  admin       195 23 May 10:17 libgfortran.spec
lrwxr-xr-x     1 root  admin        37 23 May 10:17 libgomp.1.dylib -> /opt/local/lib/libgcc/libgomp.1.dylib
-rw-r--r--     1 root  admin   1497672 23 May 10:17 libgomp.a
lrwxr-xr-x     1 root  admin        15 23 May 10:17 libgomp.dylib -> libgomp.1.dylib
-rw-r--r--     1 root  admin       169 23 May 10:17 libgomp.spec
lrwxr-xr-x     1 root  admin        36 23 May 10:17 libitm.1.dylib -> /opt/local/lib/libgcc/libitm.1.dylib
-rw-r--r--     1 root  admin   1379024 23 May 10:17 libitm.a
lrwxr-xr-x     1 root  admin        14 23 May 10:17 libitm.dylib -> libitm.1.dylib
-rw-r--r--     1 root  admin       162 23 May 10:17 libitm.spec
lrwxr-xr-x     1 root  admin        41 23 May 10:17 libobjc-gnu.4.dylib -> /opt/local/lib/libgcc/libobjc-gnu.4.dylib
-rw-r--r--     1 root  admin    395856 23 May 10:17 libobjc-gnu.a
lrwxr-xr-x     1 root  admin        19 23 May 10:17 libobjc-gnu.dylib -> libobjc-gnu.4.dylib
lrwxr-xr-x     1 root  admin        41 23 May 10:17 libquadmath.0.dylib -> /opt/local/lib/libgcc/libquadmath.0.dylib
-rw-r--r--     1 root  admin   1020704 23 May 10:17 libquadmath.a
lrwxr-xr-x     1 root  admin        19 23 May 10:17 libquadmath.dylib -> libquadmath.0.dylib
-rw-r--r--     1 root  admin       327 23 May 10:17 libsanitizer.spec
lrwxr-xr-x     1 root  admin        36 23 May 10:17 libssp.0.dylib -> /opt/local/lib/libgcc/libssp.0.dylib
-rw-r--r--     1 root  admin     62224 23 May 10:17 libssp.a
lrwxr-xr-x     1 root  admin        14 23 May 10:17 libssp.dylib -> libssp.0.dylib
-rw-r--r--     1 root  admin      2408 23 May 10:17 libssp_nonshared.a
lrwxr-xr-x     1 root  admin        39 23 May 10:17 libstdc++.6.dylib -> /opt/local/lib/libgcc/libstdc++.6.dylib
-rw-r--r--     1 root  admin      2402 23 May 10:17 libstdc++.6.dylib-gdb.py
-rw-r--r--     1 root  admin  18971840 23 May 10:17 libstdc++.a
lrwxr-xr-x     1 root  admin        17 23 May 10:17 libstdc++.dylib -> libstdc++.6.dylib
-rw-r--r--     1 root  admin   4051736 23 May 10:17 libstdc++fs.a
-rw-r--r--     1 root  admin    858224 23 May 10:17 libsupc++.a
lrwxr-xr-x     1 root  admin        38 23 May 10:17 libubsan.1.dylib -> /opt/local/lib/libgcc/libubsan.1.dylib
lrwxr-xr-x     1 root  admin        16 23 May 10:17 libubsan.dylib -> libubsan.1.dylib

So IMHO the solution to the @path issue for arm64 (now) for gcc11 would be to switch gcc11 on arm to Iain's gcc11 repo, current commit.

[kencu (Ken)]

Oop -- hold the phone. I see a problem with this plan.

Because Iain has set up gcc to add an rpath search directory like this:

/opt/local/lib/gcc11/gcc/x86_64-apple-darwin21/11.3.0/../../..

instead of like this:

/opt/local/lib/libgcc

all the executables are fragile to a version update or any other update that makes this directory disappear:

/opt/local/lib/gcc11/gcc/x86_64-apple-darwin21/11.3.0/../../..

and all the executables will break if gcc-11 becomes gcc-11.4.0, for example, and that 11.3.0 directory is no longer there.

SO this plan can't work for us. Yikers. Back to the drawing board.

[tenomoto (Takeshi Enomoto)]

Thanks for experimenting a general solution. This ticket is around for a several month. If we need some time for a more robust general solution, I suggest to commit the workaround before more R users stop using MacPorts. I was uncomfortable with the official binary: it assumes XQuartz and has its own Tcl/Tk.

[Schamschula (Marius Schamschula)]

I also ran into the same @rpath issue with octave, see #65040 and ​https://github.com/macports/macports-ports/pull/14925

Fortunately, only four instances, but I'd rather see a cleaner solution than using install_name_tool.

[kencu (Ken)]

Every single port that uses gcc for something (mostly fortran) is broken on arm - hundreds of them.

Iain was sure he had fixed it as above, but the fragile path is going to be an issue.

I asked him for a configure argument that we could use to force an rpath we wanted added to all executables, but Iain was not keen on what that might lead to... but we probably will need it.

[kencu (Ken)]

Replying to tenomoto:

Thanks for experimenting a general solution. This ticket is around for a several month. If we need some time for a more robust general solution, I suggest to commit the workaround before more R users stop using MacPorts. I was uncomfortable with the official binary: it assumes XQuartz and has its own Tcl/Tk.

Feel free to go ahead and manually fix ports one by one if you like. I get it -- this is a PITA.

I had hoped that a simple gcc update to the current gcc11 in Iain's repo would be the "one true fix" -- and it almost was.

Either way, once gcc is properly fixed up, the manual fixes will all have to be stripped out of the portfiles and then every single port that is built with gcc is going to have to be rebuilt with the proper rpath library directory added to it.

That's a yuck.

[kencu (Ken)]

I’d like to test to see if my versioned path concern is really true, or if the “…/…/…” stuff means the versioned part is actually skipped in the path search.

If that is how posix interprets that path, then one less issue.

The /opt/local/lib/gcc11 part would always have to exist, though. And we need that to be /opt/local/lib/libgcc for our plans, which could be a symlink, but which it can’t be if gcc11 is installed I think, so probably we’re still out of luck.

[tenomoto (Takeshi Enomoto)]

In add1dc36fa315b8b985c5e892df0e564e484b9a0/macports-ports (master):

R: enable build on M1

Closes: #64204

[kencu (Ken)]

good news, Iain has added the configuration option we needed, --with-darwin-add-path= that allows you to specify a path that the compiler will automatically add to the embedded run paths.

Once we get ready to move to a branch that has this added, we can add this to the configure args:

--with-darwin-add-path=${prefix}/lib/libgcc

and we should be in business, if keeping @rpaths is what we are going to do.

[kencu (Ken)]

Iain's latest fixes look good to me:

% gfortran-mp-11 hello.f90

% ./a.out
 Hello World!

% otool -L a.out
a.out:
	@rpath/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	@rpath/libgcc_s.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	@rpath/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)

% otool -l a.out | grep LC_RPATH -A 2
          cmd LC_RPATH
      cmdsize 40
         path /opt/local/lib/libgcc (offset 12)
--
          cmd LC_RPATH
      cmdsize 32
         path @loader_path (offset 12)
--
          cmd LC_RPATH
      cmdsize 72
         path /opt/local/lib/gcc11/gcc/x86_64-apple-darwin21/11.3.0 (offset 12)
--
          cmd LC_RPATH
      cmdsize 40
         path /opt/local/lib/gcc11 (offset 12)