clang-11: set-xcode-analyzer does not work

[szhorvat (Szabolcs Horvát)]

Trying to run set-xcode-analyzer-mp-11 fails with an error. It appears that this tool is for Python 2.7 only, but it is set up to run with MacPorts's own python3.9, thus it fails due to the old print statement.

~ $ set-xcode-analyzer-mp-11
  File "/opt/local/libexec/llvm-11/bin/set-xcode-analyzer", line 9
    print "set-xcode-analyzer requires Python 2.7 or later"
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("set-xcode-analyzer requires Python 2.7 or later")?
~ [1] $ 

The comment inside the script says:

#!/opt/local/bin/python3.9

# [PR 11661] Note that we hardwire to /opt/local/bin/python3.9 because we
# want to the use the system version of Python on Mac OS X.
# This one has the scripting bridge enabled.

This does not make sense as it claims that the system Python is needed, yet it uses MacPorts's Python.

[szhorvat (Szabolcs Horvát)]

Note: I have not used set-xcode-analyzer before. I ended up here from ​https://clang-analyzer.llvm.org/xcode.html, which seems to be somewhat outdated.

[ryandesign (Ryan Carsten Schmidt)]

This does not make sense as it claims that the system Python is needed, yet it uses MacPorts's Python.

The clang-11 port replaces all occurrences of /usr/bin/python in set-xcode-analyzer with /opt/local/bin/python3.9, including the occurrence within the comment. Before the portfile changed it, the top of the file was:

#!/usr/bin/python

# [PR 11661] Note that we hardwire to /usr/bin/python because we
# want to the use the system version of Python on Mac OS X.
# This one has the scripting bridge enabled.

import sys
if sys.version_info < (2, 7):
    print "set-xcode-analyzer requires Python 2.7 or later"
    sys.exit(1)

However in clang 12 it looks like it has changed to:

#!/usr/bin/python

# [PR 11661] Note that we hardwire to /usr/bin/python because we
# want to the use the system version of Python on Mac OS X.
# This one has the scripting bridge enabled.

import sys
if sys.version_info < (3, 6):
    print "set-xcode-analyzer requires Python 3.6 or later"
    sys.exit(1)

...which doesn't make sense since /usr/bin/python isn't Python 3.6 or later and also since as you point out this isn't valid Python 3 print syntax.

I don't know what "the scripting bridge" is or whether we have it in the MacPorts versions of Python.

[szhorvat (Szabolcs Horvát)]

I don't know what "the scripting bridge" is or whether we have it in the MacPorts versions of Python.

It appears that the reason it needs to use /usr/bin/python is that otherwise this does not work:

from AppKit import *

It does not work with Python 2.7 from MacPorts.

[kencu (Ken)]

I have never tried to use this tool on any system; this is a holdover from some years back.

Could you give me a quick example of how it works / how to use it so I can test it once in a while?

[szhorvat (Szabolcs Horvát)]

This is my first time using it (successfully). I documented my personal use case here:

​https://github.com/igraph/igraph/wiki/Running-linters-and-analysers#run-within-xcode

Official (but somewhat outdated) docs are here:

​https://clang-analyzer.llvm.org/xcode.html

Beware that this tool patches the following file within Xcode on my machine, and causes Xcode to use the selected Clang version for static analysis:

/Applications/Xcode.app/Contents/PlugIns/Xcode3Core.ideplugin/Contents/SharedSupport/Developer/Library/Xcode/Plug-ins/Clang LLVM 1.0.xcplugin/Contents/Resources/Clang LLVM 1.0.xcspec

The way I used it was the following:

sudo set-xcode-analyzer-mp-11 --use-checker-build=/opt/local/libexec/llvm-11/

The change can be reverted like this:

sudo set-xcode-analyzer-mp-11 --use-xcode-clang

Both appeared to work, although I didn't do a direct diff to determine that the revert was accurate. I only inspected the result manually.

To use the static analysis feature from within Xcode, refer to my first two links.

[szhorvat (Szabolcs Horvát)]

You can run it without sudo first to verify that it at least starts up. The fix in my PR is necessary for it to even start.

But then there is also the other weirdness that @ryandesign mentioned about clang-12, i.e. that the script seems to require Python 3 but uses Python 2 syntax.

Note that I only tested on macOS 10.14. I don't know whether later macOS versions still ship with Python 2.7.

[kencu (Ken)]

I'll look into this and see how we might do something comprehensive across all our clang versions.

For now, you have your version working it seems.

I suspect I will just wind up removing the analyzer from all but the last couple of OS versions, simplify this all significantly, and leave the scripts alone as they come in the source tarball as upstream has them all working as they should on current systems, other packagers like homebrew don't touch llvm hardly at all, and nobody on anything older than about Mojave should need this feature anyway.

[szhorvat (Szabolcs Horvát)]

Sounds good, thanks for looking into it @kencu!

[szhorvat (Szabolcs Horvát)]

Ken, if you have any means to do so, can you please let the Clang people know that the set-xcode-analyzer script is broken in Clang 12, as pointed out by Ryan above? I.e., it requires Python 3.6 yet uses Python 2 syntax.

I cannot report this problem myself as I could not find any channels through which newcomers can report Clang bugs. Their bug tracker does not accept registrations: ​https://bugs.llvm.org/ The email address mentioned there rejects messages which do not come from already known addresses. In the end, it seems you either have ways to talk to them already, or you're locked out.

[szhorvat (Szabolcs Horvát)]

They broke it in this commit: ​https://github.com/llvm/llvm-project/commit/5e31e226b5b2b682607a6578ff5adb33daf4fe39#diff-08cd063d896300d91d46c15dda767e8ab8a1e4ad07fb346ed52fcd3cccd76e49

[kencu (Ken)]

That was exactly why I changed the recent clang* ports to require python3 a bit ago.

If you are going to take an issue like this upstream at LLVM, you have to have it fairly clearly thought through, as obviously all the shipping Apple systems work.

[szhorvat (Szabolcs Horvát)]

as obviously all the shipping Apple systems work.

I don't know if Apple even ships set-xcode-analyzer. I am still using 10.14 so I don't know what the situation is on newer macOS, but on my system there does not seem to be an Apple-provided set-xcode-analyzer. This is no surprise as Xcode is set up to use Apple's own analyzer out of the box. set-xcode-analyzer is used to change it to a different one. Perhaps Apple doesn't even support using an alternative Clang installation—I don't know.

[szhorvat (Szabolcs Horvát)]

That said, if you also don't have a convenient way to contact them, let's just forget about it. It's not worth the effort. I can patch up Xcode to do what I want manually, and apparently no one else is using this script (otherwise the breakage would've been noticed) ...

[kencu (Ken)]

If you have a fix, pop it up here, show us how it fixes the thing, and if it works, we'll use it.

We have 25 patches to clang so far. 26 is not a big deal.

[kencu (Ken)]

I do have a convenient way to contact the.

I also have about 2 kg of credibility in the bank, and have to be careful how I spend it :>

[kencu (Ken)]

BTW, perhaps you noticed that I already stopped reinplacing our python3 into the script you were pointing out that drove this issue.

Is that all the fix you needed?

[szhorvat (Szabolcs Horvát)]

For clang-12, a bit more is needed.

I'll let you know when I have a patch, but not today.