Java version not detected on macOS Big Sur 11.0.1

[breun (Nils Breunese)]

When trying to upgrade a port on macOS Big Sur 11.0.1 which requires Java (via 'PortGroup java 1.0'), I get the following error:

% sudo port upgrade maven3
--->  Fetching archive for maven3
--->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://packages.macports.org/maven3
--->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://nue.de.packages.macports.org/maven3
--->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://mse.uk.packages.macports.org/maven3
--->  Computing dependencies for maven3
--->  Fetching distfiles for maven3
Error: maven3 requires Java 1.7+ but no such installation could be found.
Error: Failed to fetch maven3: missing required Java version
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_java_maven3/maven3/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.

However, OpenJDK 8 is installed via MacPorts just fine:

% /usr/libexec/java_home -f -v 1.7
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home
% port installed openjdk8
The following ports are currently installed:
  openjdk8 @8u272_0 (active)
% echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

The full log file:

version:1
:debug:sysinfo macOS 11.0 (darwin/20.1.0) arch i386
:debug:sysinfo MacPorts 2.6.4
:debug:sysinfo Xcode 12.2
:debug:sysinfo SDK 11.0
:debug:sysinfo MACOSX_DEPLOYMENT_TARGET: 11.0
:debug:main Executing org.macports.main (maven3)
:debug:main dropping privileges: euid changed to 502, egid changed to 501.
:debug:archivefetch archivefetch phase started at Sat Nov 14 01:55:29 CET 2020
:msg:archivefetch --->  Fetching archive for maven3
:debug:archivefetch Executing org.macports.archivefetch (maven3)
:debug:archivefetch euid/egid changed to: 0/0
:debug:archivefetch chowned /opt/local/var/macports/incoming to macports
:debug:archivefetch euid/egid changed to: 502/501
:info:archivefetch --->  maven3-3.6.3_0.darwin_20.noarch.tbz2 doesn't seem to exist in /opt/local/var/macports/incoming/verified
:msg:archivefetch --->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://packages.macports.org/maven3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404
:msg:archivefetch --->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://nue.de.packages.macports.org/maven3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404 Not Found
:msg:archivefetch --->  Attempting to fetch maven3-3.6.3_0.darwin_20.noarch.tbz2 from https://mse.uk.packages.macports.org/maven3
:debug:archivefetch Fetching archive failed: The requested URL returned error: 404 Not Found
version:1
:debug:sysinfo macOS 11.0 (darwin/20.1.0) arch i386
:debug:sysinfo MacPorts 2.6.4
:debug:sysinfo Xcode 12.2
:debug:sysinfo SDK 11.0
:debug:sysinfo MACOSX_DEPLOYMENT_TARGET: 11.0
:msg:archivefetch --->  Computing dependencies for maven3:info:archivefetch .:debug:archivefetch Searching for dependency: openjdk11
:debug:archivefetch Found Dependency: receipt exists for openjdk11
:debug:archivefetch Searching for dependency: maven_select
:debug:archivefetch Found Dependency: receipt exists for maven_select
:debug:main Executing org.macports.main (maven3)
:debug:main dropping privileges: euid changed to 502, egid changed to 501.
:debug:fetch fetch phase started at Sat Nov 14 01:55:33 CET 2020
:notice:fetch --->  Fetching distfiles for maven3
:debug:fetch Executing proc-pre-org.macports.fetch-fetch-0
:debug:fetch Discovered JAVA_HOME via /usr/libexec/java_home: /Library/Java/JavaVirtualMachines/openjdk15-openj9/Contents/Home
:debug:fetch Adding dependency on JDK fallback openjdk11
:error:fetch maven3 requires Java 1.7+ but no such installation could be found.
:error:fetch Failed to fetch maven3: missing required Java version
:debug:fetch Error code: NONE
:debug:fetch Backtrace: missing required Java version
:debug:fetch     while executing
:debug:fetch "$pre $targetname"
:error:fetch See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_java_maven3/maven3/main.log for details.

[jmroot (Joshua Root)]

What if you run the java_home command with 1.7+ rather than 1.7?

/usr/libexec/java_home -f -v 1.7+

[stanft (Stephan Anft)]

I have the same problem since Big Sur, i.e. with maven3 and gradle port.

This is the results of /usr/libexec/java_home on my system:

/usr/libexec/java_home -f -v 1.7: The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).

/usr/libexec/java_home -f -v 1.7+: The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).

/usr/libexec/java_home -f -v 1.8: /Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

It seems that the fail fast option might be the culprit. Following the result for version string 1.7+ with no fail fast:

/usr/libexec/java_home -v 1.7+: /Library/Java/JavaVirtualMachines/jdk-15.jdk/Contents/Home

... which is the newest Java SDK on my machine.

Regards, Stephan

[breun (Nils Breunese)]

That command works fine on my machine:

% /usr/libexec/java_home -f -v 1.7+
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

[stanft (Stephan Anft)]

Is this on Big Sur, too? And what JDKs are installed in total on your machine? I have too Big Sur machines here, and both have the same problem.

[breun (Nils Breunese)]

I don't know who you're asking, but I created this ticket and I'm indeed on Big Sur.

I have these JDKs installed:

% port installed | grep openjdk
  openjdk8 @8u272_0 (active)
  openjdk8-graalvm @20.2.0_0 (active)
  openjdk8-openj9 @8u272_0 (active)
  openjdk11 @11.0.9_0 (active)
  openjdk11-graalvm @20.2.0_0 (active)
  openjdk11-openj9 @11.0.9_0 (active)
  openjdk15 @15.0.1_0 (active)
  openjdk15-openj9 @15.0.1_0 (active)

Or from the perspective of java_home:

% /usr/libexec/java_home -V
Matching Java Virtual Machines (8):
    15.0.1 (x86_64) "AdoptOpenJDK" - "OpenJDK 15.0.1" /Library/Java/JavaVirtualMachines/openjdk15-openj9/Contents/Home
    15.0.1 (x86_64) "AdoptOpenJDK" - "OpenJDK 15.0.1" /Library/Java/JavaVirtualMachines/openjdk15/Contents/Home
    11.0.9 (x86_64) "AdoptOpenJDK" - "OpenJDK 11.0.9" /Library/Java/JavaVirtualMachines/openjdk11-openj9/Contents/Home
    11.0.9 (x86_64) "AdoptOpenJDK" - "OpenJDK 11.0.9" /Library/Java/JavaVirtualMachines/openjdk11/Contents/Home
    11.0.8 (x86_64) "GraalVM Community" - "GraalVM CE 20.2.0" /Library/Java/JavaVirtualMachines/openjdk11-graalvm/Contents/Home
    1.8.0_272 (x86_64) "AdoptOpenJDK" - "OpenJDK 8" /Library/Java/JavaVirtualMachines/openjdk8/Contents/Home
    1.8.0_272 (x86_64) "AdoptOpenJDK" - "OpenJDK 8" /Library/Java/JavaVirtualMachines/openjdk8-openj9/Contents/Home
    1.8.0_262+10 (x86_64) "GraalVM Community" - "GraalVM CE 20.2.0" /Library/Java/JavaVirtualMachines/openjdk8-graalvm/Contents/Home
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

[amake (Aaron Madlon-Kay)]

I'm also having this problem.

The issue seems to be that using -f -v with a non-literal version (e.g. 1.8+) fails when run as the root user.

$ sudo /usr/libexec/java_home -f -v 1.8+
The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
Please visit http://www.java.com for information on installing Java.

$ sudo /usr/libexec/java_home -f -v 1.8 
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

$ /usr/libexec/java_home -f -v 1.8+    
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

I tried wrapping find_java_home in the portgroup with exec_as_uid to run the checks as the (non-root) invoking user, but it still didn't work. It might have something to do with the env rather than the user itself.

[amake (Aaron Madlon-Kay)]

I have a temporary workaround ready here: ​https://github.com/macports/macports-ports/pull/9123

[Aaron Madlon-Kay <aaron@…>]

In a5935f1fd89e02a43832bd9914766729b76ca792/macports-ports (master):

java-1.0: temporarily skip version check on Big Sur

See #61445

[breun (Nils Breunese)]

Hm, that command doesn't fail for me and gives the same result, whether run via sudo or not:

~ % sudo /usr/libexec/java_home -f -v 1.8+
/Library/Java/JavaVirtualMachines/openjdk15-openj9/Contents/Home
~ % /usr/libexec/java_home -f -v 1.8+
/Library/Java/JavaVirtualMachines/openjdk15-openj9/Contents/Home

[JBYoshi (Jonathan Browne)]

On my system, I also get an error when running the command with sudo. However, manually setting the $JAVA_HOME environment variable to a valid installation will cause it to succeed:

jbyoshi@Yoshi ~ % sudo /usr/libexec/java_home -f -v 1.8+ 
The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
Please visit http://www.java.com for information on installing Java.

jbyoshi@Yoshi ~ % sudo JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home /usr/libexec/java_home -f -v 1.8+
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

In addition, unsetting the environment variable on a non-sudo environment reproduces the error:

jbyoshi@Yoshi ~ % unset JAVA_HOME
jbyoshi@Yoshi ~ % /usr/libexec/java_home -f -v 1.8+
The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
Please visit http://www.java.com for information on installing Java.

Strangely, the command always outputs the value provided for $JAVA_HOME when it's provided, even if the versions aren't compatible.

jbyoshi@Yoshi ~ % sudo JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home /usr/libexec/java_home -f -v 15+ 
/Library/Java/JavaVirtualMachines/openjdk8/Contents/Home

[catap (Kirill A. Korinsky)]

I've discovered that -f is a cause for strange behaviour:

catap@Kirills-mini-m1 ~ % env | grep JAVA
catap@Kirills-mini-m1 ~ % /usr/libexec/java_home -V       
Matching Java Virtual Machines (2):
    16 (arm64) "Azul Systems, Inc." - "Zulu 16.0.65-ea" /Library/Java/JavaVirtualMachines/zulu-jdk16/Contents/Home
    11.0.9.1 (arm64) "Azul Systems, Inc." - "Zulu 11.43.1007" /Library/Java/JavaVirtualMachines/zulu-jdk11/Contents/Home
/Library/Java/JavaVirtualMachines/zulu-jdk16/Contents/Home
catap@Kirills-mini-m1 ~ % /usr/libexec/java_home -v 1.8+
/Library/Java/JavaVirtualMachines/zulu-jdk16/Contents/Home
catap@Kirills-mini-m1 ~ % /usr/libexec/java_home -f -v 1.8+
The operation couldn’t be completed. Unable to locate a Java Runtime that supports (null).
Please visit http://www.java.com for information on installing Java.

catap@Kirills-mini-m1 ~ % 

I have no idea that -f means because I haven't found it inside man page for java_home for mac OS Catalina and Big Sur.

=> I've opened PR: ​https://github.com/macports/macports-ports/pull/9264 that solved issue on two machines with Big Sur.

[cjones051073 (Chris Jones)]

The fix you propose changes the behaviour of the command in ways we do not want. It means the command will, on failure to find a matching JDK return the default. e.g.

Oberon ~/Projects/MacPorts/ports > /usr/libexec/java_home -h
Usage: java_home [options...]
    Returns the path to a Java home directory from the current user's settings.

Options:
    [-v/--version   <version>]       Filter versions (as if JAVA_VERSION had been set in the environment).
    [-a/--arch      <architecture>]  Filter architecture (as if JAVA_ARCH had been set in the environment).
    [-F/--failfast]                  Fail when filters return no JVMs, do not continue with default.
    [   --exec      <command> ...]   Execute the $JAVA_HOME/bin/<command> with the remaining arguments.
    [-X/--xml]                       Print full JVM list and additional data as XML plist.
    [-V/--verbose]                   Print full JVM list with architectures.
    [-h/--help]                      This usage information.

Oberon ~/Projects/MacPorts/ports > /usr/libexec/java_home -v 16+    
/Library/Java/JavaVirtualMachines/openjdk15/Contents/Home

we do not want this, as it means an inappropriate JDK will be returned on macOS 11.

This is a I think clearly a bug on macOS 11, and until Apple fixes it the current workaround in the PG is the best option.

[breun (Nils Breunese)]

MacPorts uses -f, but /usr/libexec/java_home -h only shows -F/--failfast (at least on 10.15). On which versions of macOS does /usr/libexec/java_home -h list -f as a valid option?

[Aaron Madlon-Kay <amake@…>]

In c08c2d0d7053492a0782203b939754f2233f5638/macports-ports (master):

java-1.0 portgroup: better workaround for Big Sur issue

See #61445

[amake (Aaron Madlon-Kay)]

Replying to breun:

MacPorts uses -f, but /usr/libexec/java_home -h only shows -F/--failfast (at least on 10.15). On which versions of macOS does /usr/libexec/java_home -h list -f as a valid option?

It appears that we've been using -f since my commit here in October 2018: [4c0d76dc7a6bf8a319e3c23248abb15445f6dc16/macports-ports]

My OS X 10.8 VM also shows only -F/--failfast. I couldn't confirm 100% that -f and -F behave the same way because I don't have a JVM on the VM and it looks like a pain to get now (that Safari is too old for modern TLS, I guess).

We could change it to -F but that's clearly not the cause of this particular problem.

[Aaron Madlon-Kay <amake@…>]

In 292365f0d2ddc7969e5e91c5201e5b3264924685/macports-ports (master):

java-1.0 portgroup: wildcards not supported on Big Sur

See #61445

[Aaron Madlon-Kay <amake@…>]

In 79582c24624bdad52679a66a3872206962e319b9/macports-ports (master):

java-1.0 portgroup: restore skip if check fails on Big Sur

The last couple of changes reduced the chance of spurious warnings, but there
are still some failure cases: for example maven3 wants Java 1.7+, which is
handled as 1.7, but we don't have a port for Java 1.7.

See #61445

[breun (Nils Breunese)]

I could add a subport for Java 1.7 (aka Java 7), but Java 7 reached end of life in 2011, so that doesn't sound like great idea. We could bump that maven3 dependency to Java 1.8+, but technically Maven 3 requires "JDK 1.7 or above to execute" according to ​https://maven.apache.org/download.cgi so we could create an issue for anyone that really wants to run Maven 3 on JDK 1.7 for some obscure reason (I can't think of any good ones though).

[Janosch (Janosch Peters)]

As I am quiet annyoed by this bug I made an attempt to fix it. Here is the description from github:

Instead of using /usr/libexec/java_home -f -v this piece of code uses /usr/libexec/java_home -V and does the parsing of existing JVMs and the matching with ${java.version} on its own. The code only compares major versions (1.7, 1.8, 9, 15). Asterisk and plus notation is support, but only attached to the major version. 1.8+, 9* will work, 1.7.35+ won't. If the plus suffix is used and more than one candidate is a match, the highest JVM version wins. If there are multiple JVMs with the same major version, it just picks any one of those.

Please have a look at the PR ​https://github.com/macports/macports-ports/pull/9617

[telotortium (Robert Irelan)]

Replying to Janosch:

I patched ​​https://github.com/macports/macports-ports/pull/9617 into my installation, but I also needed to install openjdk8 before commons-lang3 (a dependency of pdftk-java) would build.

[Janosch (Janosch Peters)]

Replying to telotortium:

Replying to Janosch:

I patched ​​https://github.com/macports/macports-ports/pull/9617 into my installation, but I also needed to install openjdk8 before commons-lang3 (a dependency of pdftk-java) would build.

Hm, interesting. This does not happen in my environment for commons-lang3. Could you please post the output of /usr/libexec/java_home -V in the comment section of the PR? Thanks.

[saagarjha (Saagar Jha)]

In 861baccee2c945af991ffc3988257bc2be44bc26/macports-ports (master):

java portgroup: Continue to skip check

See: #61445

[Janosch (Janosch Peters)]

In 8d70ce38f4fde7edb6bba3dd21127fb3ec502fa5/macports-ports (master):

java-pg: Better workaround for Big Sur JVM detection

Trac: #61445

With this workaround, the check for ${java.version} works as
expected for values like 1.7, 1.8+ or 9 in Big Sur. If the "+"
selector is used and there are more than one candidate, the
highest JVM version wins.

"+" and "*" can only be used on the major version, e.g. 1.8+ or 11*.
Things like 1.8.56* are not supported, as they are not used in a
single port at the moment.

[Janosch (Janosch Peters)]

Replying to breun:

When trying to upgrade a port on macOS Big Sur 11.0.1 which requires Java (via 'PortGroup java 1.0'), I get the following error:

@Nils: My fix for this issue has been merged a few weeks ago (see comment above). Could you please check if the issue is resolved for you (and close the ticket if it is)?

[breun (Nils Breunese)]

@Janosch I wouldn't mind closing the ticket, but I don't see a button for doing that.