Opened 4 years ago
Closed 4 years ago
#61256 closed enhancement (fixed)
Multicore support for math/R
Reported by: | webbp (Webb Phillips) | Owned by: | kjellpk (Kjell Konis) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | |
Keywords: | Cc: | i0ntempest, chrstphrchvz (Christopher Chavez) | |
Port: | R |
Description
Currently, math/R is built (by default) using Apple's clang. This causes both R itself and any packages installed by R to be single-core because Apple clang doesn't support OpenMP. This very much sucks and is ultimately Apple's fault. However, there is an easy solution, which is to build R with Macports clang. More specifically, when I install macports clang-10 llvm-10 and libomp, port select clang mp-clang-10 and llvm mp-llvm-10, and macports PATH is prefixed to my PATH, sudo port install R installs a multicore R capable of building multicore R packages. Anecdotally, this R on my 9-year-old MacBook Air is faster for many common operations than the default Macports R on my 2020 MacBook Air.
The vast majority of users will not know how to do this. Therefore, please update the default R build to build a multicore-capable R.
Change History (12)
comment:1 Changed 4 years ago by mf2k (Frank Schima)
comment:2 Changed 4 years ago by mf2k (Frank Schima)
Cc: | i0ntempest added |
---|---|
Owner: | set to kjellpk |
Port: | R added; math/R removed |
Status: | new → assigned |
Type: | request → enhancement |
comment:3 Changed 4 years ago by i0ntempest
How do I know if R on my system has multicore support? If it's that easy as you said, I might just blacklist apple clang.
comment:4 Changed 4 years ago by webbp (Webb Phillips)
My description of the solution isn't quite right. The way I solved this was with:
~/.R/Makevars: LLVM_LOC = /opt/local/libexec/llvm-10 CC=$(LLVM_LOC)/bin/clang -fopenmp CXX=$(LLVM_LOC)/bin/clang++ -fopenmp CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe LDFLAGS=-L/opt/local/lib -L$(LLVM_LOC)/lib -Wl,-rpath,$(LLVM_LOC)/lib CPPFLAGS=-I/opt/local/include -I$(LLVM_LOC)/include
That alone on my new machine breaks installing R packages, so there are some missing steps, and some of the other steps I took above may have been irrelevant.
comment:5 Changed 4 years ago by i0ntempest
Are those just for compiling R packages? I'm not very experienced in manipulating complex builds (like R) yet, so can you try building R with multicore support on your new machine and tell me exactly what change did you make? Then I could try putting them into the portfile.
comment:6 Changed 4 years ago by chrstphrchvz (Christopher Chavez)
compiler.openmp_version
should be used to specify OpenMP requirement; an OpenMP-capable compiler will be chosen and libomp
dependency added automatically. See CompilerSelection#GettingtheRightCompilerinthePortfile
comment:7 Changed 4 years ago by chrstphrchvz (Christopher Chavez)
Cc: | chrstphrchvz added |
---|
comment:8 Changed 4 years ago by webbp (Webb Phillips)
compiler.openmp_version 4.5
did not enable R to build packages with OpenMP, e.g., with compiler.openmp_version 4.5
:
clang: error: unknown argument: '-pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64' *** OpenMP not supported! data.table uses OpenMP to automatically *** parallelize operations like sorting, grouping, file reading, etc.
I've investigated the problem a bit more. The issue mostly or entirely affects packages installed by R rather than the R binary itself, that is, there are many R packages which benefit greatly from being built with openmp, but the R binary may not. The above .R/Makevars cause R to build packages with OpenMP in Mac OS 10.11, whereas these below work for 10.15. (Those are the only two Mac OS versions I have readily available.)
CC=/opt/local/bin/clang -fopenmp CXX=/opt/local/bin/clang++ -fopenmp # -O3 should be faster than -O2 (default) level optimisation .. CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe LDFLAGS=-L/opt/local/lib -Wl,-rpath,/opt/local/lib CPPFLAGS=-I/opt/local/include
Source: https://github.com/Rdatatable/data.table/wiki/Installation
Also, I believe that R uses the flags with which it was built as the default flags for building packages, so specifying these flags the Portfile should solve the problem. However, the specific flags that work differ for different Mac OS versions, and they are also known to be incompatible with some R packages. So maybe the best thing to do here is just to add some lines to the Portfile long_description, e.g.: to install multicore packages in R, do sudo port install clang-10 pkgconfig && sudo port select clang mp-clang-10
and create ~/.R/Makevars as above.
comment:9 Changed 4 years ago by i0ntempest
It's working for me:
* installing *source* package ‘data.table’ ... ** package ‘data.table’ successfully unpacked and MD5 sums checked ** using staged installation zlib 1.2.11 is available ok OpenMP supported ** libs /opt/local/bin/clang-mp-10 -fopenmp -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/local/include -fopenmp -fPIC -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c assign.c -o assign.o In file included from assign.c:1: In file included from ./data.table.h:13: In file included from ./myomp.h:2: /opt/local/include/libomp/omp.h:53:9: warning: ISO C restricts enumerator values to range of 'int' (2147483648 is too large) [-Wpedantic] omp_sched_monotonic = 0x80000000 ^ ~~~~~~~~~~ ...
My Makevar file is same as the one you have except I explicitly specified /opt/local/bin/clang-mp-10
and /opt/local/bin/clang++-mp-10
.
I've added openmp in as a variant, do you wanna try my portfile?
comment:10 Changed 4 years ago by i0ntempest
Also I just tried with the default macports R, it can build openmp packages despite itself is not built explicitly with openmp support (maybe it's automatically enabled). But it's still better to add a variant since it'll ensure one has the correct compiler and openmp library installed.
comment:11 Changed 4 years ago by i0ntempest
comment:12 Changed 4 years ago by i0ntempest
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
Note that a "request" ticket type is only for requesting a new port.
In the future, please add the port maintainer(s) to Cc (
port info --maintainers R
), if any.