Setting configure.compiler to a MacPorts-provided compiler doesn't add dependency automatically

[neverpanic (Clemens Lang)]

Setting configure.compiler to a compiler provided my MacPorts doesn't automatically add a dependency on this compiler.

For an example see the percona port, which sets configure.compiler to apple-gcc-4.2, but doesn't declare a dependency on it.

[ryandesign (Ryan Carsten Schmidt)]

I think that's intentional.

[raimue (Rainer Müller)]

I don't see the reason if this is intentional. If a compiler such as apple-gcc-4.2 is specified for configure.compiler, this explicitely requests the compiler installed by the MacPorts port. Thus, the apple-gcc-4.2 port should be in depends_build.

[raimue (Rainer Müller)]

Actually, they need to be depends_lib in case produced binaries are linked to a runtime library like libgcc. So this is more complicated than I thought initially. See also #30041 for some reference.

[jeremyhu (Jeremy Huddleston Sequoia)]

A build dependency should always be added for the port.

If it's gcc4X (not llvm-gcc42 and not apple-gcc4X), there should additionally be a lib dependency.

If the compiler is *clang* or apple-gcc42, it should skip_archcheck.

[jeremyhu (Jeremy Huddleston Sequoia)]

Cc Me!

[jeremyhu (Jeremy Huddleston Sequoia)]

This should really be done in 2.2

[raimue (Rainer Müller)]

I thought I could implement this because I already did something similar for the use_autoconf stuff in the configure phase. I do not find any time to look into the issue at the moment, so anyone else might take this ticket.

[skymoo (Adam Mercer)]

Cc Me!

[jeremyhu (Jeremy Huddleston Sequoia)]

I'm just doing this in individual ports until this is solved:

# TODO: base should do this: http://trac.macports.org/ticket/32542
if {[portconfigure::compiler_is_port ${configure.compiler}]} {
    depends_build-append port:$portconfigure::compiler_name_map(${configure.compiler})
    depends_skip_archcheck-append $portconfigure::compiler_name_map(${configure.compiler})
}

[larryv (Lawrence Velázquez)]

Cc Me!

[cooljeanius (Eric Gallager)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Replying to jeremyhu@…:

I'm just doing this in individual ports until this is solved:

That seems fine, except for the gcc4x ports which need to be a lib dependency as noted above.

[jeremyhu (Jeremy Huddleston Sequoia)]

Ah right. Crap. I'll need to update my block. Thanks for catching it.

But at least none of the default compilers provided in the default lists fall into that category.

[jeremyhu (Jeremy Huddleston Sequoia)]

This should handle it then:

# TODO: base should do this: http://trac.macports.org/ticket/32542
if {[portconfigure::compiler_is_port ${configure.compiler}]} {
    depends_build-append port:$portconfigure::compiler_name_map(${configure.compiler})

    # base 2.1.x ignores the argument and just use ${configure.compiler}
    if {[portconfigure::arch_flag_supported ${configure.compiler}]} {
        depends_skip_archcheck-append $portconfigure::compiler_name_map(${configure.compiler})
    }

    if {[string match macports-gcc* ${configure.compiler}]} {
        depends_lib-append port:$portconfigure::compiler_name_map(${configure.compiler})
    }
}

[jeremyhu (Jeremy Huddleston Sequoia)]

The workaround above causes issues with the generated PortIndex on the rsync servers, #37817

What is a viable workaround? As noted in the release notes for XCode 4.6, this is the last release to contain llvm-gcc, so we're quickly approaching the point where *any* backup compiler will be a port.

[larryv (Lawrence Velázquez)]

I’m working on it.

[jeremyhu (Jeremy Huddleston Sequoia)]

Awesome, thanks.

[larryv (Lawrence Velázquez)]

Cc Me!

[larryv (Lawrence Velázquez)]

Patch adding automatic dependency management for configure.compiler.

[larryv (Lawrence Velázquez)]

I’ve implemented this as an option_proc on configure.compiler. It should work for both the default value (generated from blacklists, whitelists, and the fallback list) and for explicit overrides. Just in case, it handles some bizarre cases like multiple overrides and unsetting the variable completely.

It’s worked well in my limited testing; let me know about any issues I overlooked.

[nonstop.server@…]

Cc Me!

[jeremyhu (Jeremy Huddleston Sequoia)]

Thanks, I'll give it a whirl this week.

The logic looks sound, but my tcl-foo is not quite there to know if this is the best approach, so I'll let someone else comment on that aspect.

[jeremyhu (Jeremy Huddleston Sequoia)]

That change works if I set configure.compiler, but not if I use the blacklist:

# This doesn't pull in the dependency on llvm-gcc42
compiler.blacklist  clang

# This pulls in the dependency
configure.compiler  macports-llvm-gcc-4.2

[larryv (Lawrence Velázquez)]

Replying to jeremyhu@…:

That change works if I set configure.compiler, but not if I use the blacklist:

Huh. I tested that on ld64 by changing the existing blacklist from “gcc-4.0” to “clang” and commenting out your workaround, but if I add a blacklist to zsh, nothing shows up. Will take a look shortly.

[larryv (Lawrence Velázquez)]

Ugh, looks like I can't even rely on configure.compiler being read once before dependencies are accessed. I’ll have to find a better place to put this.

[larryv (Lawrence Velázquez)]

part deux

[larryv (Lawrence Velázquez)]

Alright, let’s try this again. I make the same claims as last time, except this time they should be true!

[larryv (Lawrence Velázquez)]

third time’s the charm

[larryv (Lawrence Velázquez)]

Patch The Third has basically the same logic as the 2nd, but I rearranged things to clarify the control flow.

[jeremyhu (Jeremy Huddleston Sequoia)]

Awesome! I'll give it a whirl in a bit.

[jeremyhu (Jeremy Huddleston Sequoia)]

Awesome. I tried a small test case with each of:

configure.compiler  macports-llvm-gcc-4.2
compiler.whitelist  macports-llvm-gcc-4.2
compiler.blacklist  clang

and in each case, it pulled in llvm-gcc42 as a dependency! Good job. Please give some time for jmr or others to comment on the actual implementation (esp hijack_option), and let's get this in sometime this week =)

[neverpanic (Clemens Lang)]

hijack_option does change the semantics of depends_lib and depends_build. Although it probably doesn't make a difference in most cases we should consider doing this without this hack. If we had a callback that would be evaluated after the Portfile (and all variants) were completely sourced, we could add the correct compiler dependency there using a simple depends_build-append statement.

source:trunk/base/src/macports1.0/macports.tcl@102897:1608-1618#L1608​ is the relevant part of the code executing the Portfiles. Adding a callback there (e.g., $workername eval add_compiler_dependencies, which would in turn have to be created using $workername alias add_compiler_dependencies actually_implementing_procedure) seems like a cleaner solution to me and would avoid the change in semantics. We could also provide a generic callback mechanism there, just in case we (or a Portfile author) ever needs something similar.

[neverpanic (Clemens Lang)]

Working on a patch.

[larryv (Lawrence Velázquez)]

I got it, Cal. I was actually initially looking for a place to run this after the Portfile was done being sourced, but I gave up on mucking through macports1.0/ and settled on the variable trace. This is exactly what I wanted, though.

[neverpanic (Clemens Lang)]

4th attempt preventing the change of semantics of depends_lib and depends_build

[neverpanic (Clemens Lang)]

Attached a patch providing a generic callback mechanism (that might also be useful in other cases, I recall we also have an automatic dependcy on bsdmake if ${build.type} == bsd where this could be useful) and automatic compiler dependencies.

To prevent duplicate dependencies in ports that already correctly declare a dependency we might add depends_${type}-delete port:$compiler_port, but that's just a cosmetic change now.

Comments?

[larryv (Lawrence Velázquez)]

Looks good to me. Wish I’d known where mportopen was before I spent so much time making those traces work properly. C’est la vie.

The use_auto* options would probably benefit from using callbacks, too.

[jeremyhu (Jeremy Huddleston Sequoia)]

That looks great, thanks.

[neverpanic (Clemens Lang)]

#37817 reminds me this will not be working properly on the rsync server when generating the index in some cases: The portindex is generated on a linux box. portindex supports pretending to be on another platform (which is how we achieve platform-specific indices). However, on those machines, $xcodeversion is always "none", causing compiler.fallback to be {cc} by default.

So, e.g., for ports on 10.8 blacklisting all xcode compilers the fallback would be some macports compiler, causing a dependency to be added. On the server generating the portindex the fallback (and default anyway) would be cc. Thus, the dependency wouldn't get recorded in the PortIndex.

If we wanted to fix this correctly, we'd have to pretend having Xcode installed on the servers, too. This would also mean that our PortIndex would be Xcode-version-specific and we'd have to support multiple versions of Xcode per OS version. I'd say it's not worth the effort and we can live with somewhat off dependencies in the PortIndex (since building will still work correctly).

[neverpanic (Clemens Lang)]

Commited in r102932. Thanks to larryv and jeremyhu for the patches and testing.