Opened 12 years ago

Closed 6 years ago

Last modified 4 years ago

#37846 closed defect (fixed)

cctools as does not support AVX, consider using llvm

Reported by: wd11@… Owned by: macports-tickets@…
Priority: Normal Milestone:
Component: ports Version: 2.1.2
Keywords: Cc: mww@…, ltalirz (Leopold Talirz), cooljeanius (Eric Gallager), mascguy (Christopher Nielsen)
Port: cctools gcc

Description

The gcc groups of compiler ports (gcc47 etc) depend on cctools' for the linker 'as'. However, that linker is deficient: it doesn't know about the AVX instruction set supported by Intel's sandy-bridge and ivy-bridge processors, which are used in the recent mac books. As a consequence, gcc cannot compile valid code, such as the attached c++ program (note: compile with the -march=native option on a intel corei7 chip)

Note that this is a known issue (http://stackoverflow.com/questions/9840207/how-to-use-avx-pclmulqdq-on-mac-os-x-lion), but the suggested solution is extremely inelegant and does not mesh well with macports updates.

Instead, this problem should solved at its heart, i.e. by shipping the gcc ports with a better linker (or by convincing cctools to ship a useful linker).

Attachments (1)

test1.cc (1.1 KB) - added by wd11@… 12 years ago.
valid source code that cannot be compiled.

Download all attachments as: .zip

Change History (28)

Changed 12 years ago by wd11@…

Attachment: test1.cc added

valid source code that cannot be compiled.

comment:1 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: jeremyhu@… mww@… added
Keywords: deficient dependency removed
Milestone: MacPorts 2.1.3

comment:2 Changed 12 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Cc: jeremyhu@… removed
Owner: changed from macports-tickets@… to jeremyhu@…
Port: cctools added; gcc47 removed

comment:3 Changed 12 years ago by jeremyhu (Jeremy Huddleston Sequoia)

as is not a linker. It's an assembler. ld is the linker and is provided by ld64.

comment:4 Changed 12 years ago by ltalirz (Leopold Talirz)

Cc: leopold.talirz@… added

Cc Me!

comment:5 Changed 11 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Summary: gcc47 cannot compile valid code because deficient linker cctools' ascctools as does not support AVX

comment:6 Changed 11 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Summary: cctools as does not support AVXcctools as does not support AVX, consider using llvm

We should consider using 'clang -c -x assembler' instead of as...

comment:7 Changed 11 years ago by cooljeanius (Eric Gallager)

Cc: egall@… added

Cc Me!

comment:8 Changed 10 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Owner: changed from jeremyhu@… to mww@…
Port: gcc added

comment:9 Changed 10 years ago by jeremyhu (Jeremy Huddleston Sequoia)

IMO, this is a gcc problem to work out. If the assembler doesn't support an instruction set, it shouldn't be used. gcc should either not use these instruction sets or should use llvm's assembler.

comment:10 in reply to:  9 Changed 10 years ago by cooljeanius (Eric Gallager)

Replying to jeremyhu@…:

IMO, this is a gcc problem to work out. If the assembler doesn't support an instruction set, it shouldn't be used. gcc should either not use these instruction sets or should use llvm's assembler.

You're referring to the gcc port(s) here, right? I see no reason why gcc upstream would want to use the assembler of a competing compiler; they would probably just recommend that we update the binutils port and us the as that it provides instead...

comment:11 Changed 8 years ago by kurthindenburg (Kurt Hindenburg)

Owner: changed from mww@… to macports-tickets@…
Status: newassigned

comment:12 Changed 7 years ago by mouse07410 (Mouse)

Any update on this issue? It's been over 5 years, and the problem remains.

comment:13 Changed 7 years ago by raimue (Rainer Müller)

These AVX instructions are not emitted by gcc; the example code uses intrinsics. gcc does not validate that, it just passes it through to the assembler like it always does for inline assembler code.

I do not know or check whether gcc on macOS would emit AVX code from C code (keyword: auto-vectorization).

If we want to support this, we need to modify the gcc ports in MacPorts to use the LLVM assembler. The simplest option seems to be to use --with-as=/usr/bin/as (maybe the same for the other target tools that are now provided by cctools?).

But this can only be done on macOS and Xcode version combinations that already use the LLVM assembler by default and not the old cctools. You need to find out which these are. Older versions should continue to use cctools, as that is still newer than what is provided in /usr/bin.

Then test this setup and report back.

comment:14 Changed 7 years ago by kencu (Ken)

gcc on macOS uses the ancient apple gcc assembler from about 1852. It is version 1.38, and hasn't been updated in years. The build of gcc-as in binutils is disabled as it is not apple-happy. I would love it if somebody would update that to something fresh.

However, gcc accepts -Wa,-q which causes it to use clang as the assembler. So you can add that to the cflags and cxxflags on a gcc build, and if clang calls to some newish version of clang, you're good to go.

There is a flag to spec the clang you want to sent the asm to, I believe. I just haven't found it / tested it / bothered to look for it.

When I want to build new asm with old gcc version (e.g. building TenFourFox for Intel on 10.6.8 using gcc-4.8 with modern libvpx instructions), I just add the mentioned flag and port select a newish clang.

Last edited 7 years ago by kencu (Ken) (previous) (diff)

comment:15 Changed 7 years ago by raimue (Rainer Müller)

MacPorts forces gcc to use ${prefix}/bin/as instead, which comes from cctools.

/usr/bin/as is already be the LLVM assembler on my system. There is no need to use as -q as far as I can see.

$ sw_vers -productVersion
10.12.6
$ xcodebuild -version
Xcode 9.2
Build version 9C40b
$ /usr/bin/as -v |& head -1
Apple LLVM version 9.0.0 (clang-900.0.39.2)

comment:16 Changed 7 years ago by kencu (Ken)

mmmmm. look at the same result on my system, however (10.6.8):

$ which as
/opt/local/bin/as
$ as -version
Apple Inc version cctools-895, GNU assembler version 1.38

looks like the situation depends on which OS you're on -- no surprise :>

Last edited 7 years ago by kencu (Ken) (previous) (diff)

comment:17 Changed 7 years ago by raimue (Rainer Müller)

Of course /opt/local/bin/as will always be cctools; it is provided by the MacPorts port. I was talking about /usr/bin/as, which is provided by the Xcode toolchain.

comment:18 Changed 7 years ago by kencu (Ken)

Oh, OK, you had noted ${prefix}/bin/as above.

Turns out /usr/bin/as is even worse on older systems, as it is just an even older version of cctools, although may well provide the exact same assembler:

$ /usr/bin/as -version
Apple Inc version cctools-809~2, GNU assembler version 1.38

Good to know there are different ways of doing things.

You have me thinking now. I'd like to somehow prove that -Wa,-q actually calls through to clang instead of /usr/bin/as or ${prefix}/bin/as just so I know for sure what I'm saying is actually fully correct. I'll see if I can figure out how to do that. -- K

comment:19 Changed 7 years ago by raimue (Rainer Müller)

On older systems, /usr/bin/as is an old cctools. That's why we started using ${prefix}/bin/as from the cctools port instead, to get the benefits of the newer version.

That's what I meant above: we should only use --with-as=/usr/bin/as on newer systems on which /usr/bin/as is the newer LLVM assembler. On older systems leave everything as it is.

Your theory on -Wa,-q is easy to prove:

$ /opt/local/bin/as -q
/opt/local/bin/as: assembler (/opt/local/bin/clang) not installed

comment:20 Changed 7 years ago by kencu (Ken)

Good! So on some vintage of OS, -Wa,-q is the proper way to go if you want gcc to use a current assembler. That is what I thought, as that is what worked for me.

Now we just need to figure out where the cutoff might be. Likely 10.6.8, I would bet.

comment:22 Changed 7 years ago by kencu (Ken)

The guts of this behaviour is controlled by cctools as driver.c.

For practical purposes, /usr/bin/as sends you to /usr/bin/clang for assember work on 10.10 or later, which on my system matched up to Xcode 7. On 10.9 and below, it appears you get the standard GNU assember 1.38 unless you send the -q flag, or have set the AS_INTEGRATED_ASSEMBER environment variable.

On older systems, if you have set AS_INTEGRATED_ASSEMBER, or if you have set the -q flag with -Wa,-q, then cctools as passes off the assembly to clang in the same directory as as is.

At this moment, you can't set a more specific name for clang, and you can't specify a full path to clang. It's just going to look for clang, next to as, wherever the running copy of as is.

So for a MacPorts install of cctools, /opt/local/bin/as which is the first one on the path usually will look for /opt/local/bin/clang. /usr/bin/as will look for /usr/bin/clang.

To make the MacPorts cctools as work to use clang as the assembler, there needs to be an /opt/local/bin/clang and the only way to get that is to sudo port select a clang manually, at present.

So on systems 10.9 or older, you need to select a clang, and then use -Wa,-q to make it work.

comment:23 Changed 7 years ago by mouse07410 (Mouse)

Last edited 7 years ago by mouse07410 (Mouse) (previous) (diff)

comment:24 Changed 7 years ago by raimue (Rainer Müller)

Has duplicate #56352.

comment:26 Changed 6 years ago by jmroot (Joshua Root)

Resolution: fixed
Status: assignedclosed

In 7aa2ff679e61f923475b6c2a56958da61205430b/macports-ports (master):

cctools: try to use clang in as

The clang integrated assembler is way more up-to-date than the ancient
pre-GPLv3 version of gas that is the alternative. The latter doesn't
know about newer instructions such as AVX. The as(1) shipped with
recent Xcode versions will just run clang behind the scenes under most
circumstances for this reason.

Our cctools can't actually depend on clang because that would create a
circular dependency. So we'll try to run the clang corresponding to the
LLVM being used by cctools, and if that isn't present, try the system
clang in /usr/bin if we're on an OS version that has a clang with a
working integrated assembler. (The ancient gas is still used as a final
fallback.)

Fixes: #37846

comment:27 Changed 4 years ago by mascguy (Christopher Nielsen)

Cc: mascguy added
Note: See TracTickets for help on using tickets.