py-netcdf4 @1.0.4_1: build failure when using ccache

[cooljeanius (Eric Gallager)]

I have configureccache set to yes in my macports.conf file and ran into this error when trying to build py26-netcdf:

running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'netCDF4' extension
compiling C sources
C compiler: {ccache /usr/bin/gcc-4.2} -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch x86_64 -arch i386

creating build/temp.macosx-10.6-x86_64-2.6
compile options: '-I/opt/local/include -I/opt/local/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c'
{ccache: netCDF4.c
sh: {ccache: command not found
sh: {ccache: command not found
error: Command "{ccache /usr/bin/gcc-4.2} -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch x86_64 -arch i386 -I/opt/local/include -I/opt/local/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c netCDF4.c -o build/temp.macosx-10.6-x86_64-2.6/netCDF4.o" failed with exit status 127
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_python_py-netcdf4/py26-netcdf4/work/netCDF4-1.0.4" && /opt/local/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 setup.py --no-user-cfg build 
Exit code: 1
Error: org.macports.build for port py26-netcdf4 returned: command execution failed
DEBUG: Error code: CHILDSTATUS 28040 1
DEBUG: Backtrace: command execution failed

[cooljeanius (Eric Gallager)]

main.log for py26-netcdf4

[cooljeanius (Eric Gallager)]

Edit: Oops, I forgot that this was a ticket that I had filed myself, feel free to remove my extraneous cc...

[cooljeanius (Eric Gallager)]

Anyways the reason I came back to this ticket was because this is an issue with the other py-netcdf4 sub-ports as well, and it is still an issue as of version @1.1.0...

[petrrr]

Cc Me!

[petrrr]

BTW: you should be able to remove you own CC address at any time.

[zzanderr]

I see that this is an old ticket, but I too am getting the same error when trying to upgrade py27-netcdf4. I have tried deactivating ccache, and even uninstalling ccache (of course along with appropriately modifying the settings on macports.conf and the PATH variable in my .bash_profile. All to no avail. The first--and the only previous--install of py27-netcdf4 worked okay, and I do not remember having to mess around with ccache, which was almost certainly active at the time. Now no matter what I have tried thus far gives me the same build error:

:info:build {ccache /mp2/local/bin/gcc-mp-4.7} -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch ppc -I/mp2/local/include -I/mp2/local/include -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c netCDF4/_netCDF4.c -o build/temp.macosx-10.5-ppc-2.7/netCDF4/_netCDF4.o
:info:build unable to execute '{ccache': No such file or directory
:info:build error: command '{ccache' failed with exit status 1

Note: I have also tried deactivating the old py27-netcdf4 and building from scratch. Still the same error. Weird! Any ideas? I am attaching the build log.

[zzanderr]

build log for py27-netcdf4 @1.1.8

[zzanderr]

P.S. Why does py27-netcdf4 apparently insist on using ccache? Could there be a permissions problem on the .ccache directory for this particular port?

[gnw3]

What is the output of nc-config -cc? If netcdf was built using ccache, you should see something like:

$ ccache /opt/local/bin/gcc-mp-4.8

If you edit $prefix/bin/nc-config to remove the "ccache " then the build should work, but someone needs to look at how the nc-config output is handled by the Python setup.py.

[mf2k (Frank Schima)]

Cc'ing current maintainers.

[seanfarley (Sean Farley)]

Thanks. Seems like gnwii is on this one. I'll be following to see if anything needs to be changed in the Portfile.

[zzanderr]

Sorry! I've been on vacation. Thanks for your prompt response!

As to gnwii's question, here is what I get (using the original py27-netcdf4, of course):

$ nc-config --cc
ccache /mp2/local/bin/gcc-mp-4.7

I will try building as gnwii suggests...

[zzanderr]

Oof! Now the build fails with the following:

running build_ext
building 'netCDF4._netCDF4' extension
creating build/temp.macosx-10.5-ppc-2.7
creating build/temp.macosx-10.5-ppc-2.7/netCDF4
/mp2/local/bin/gcc-mp-4.7 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch ppc -I/mp2/local/include -I/mp2/local/include -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c netCDF4/_netCDF4.c -o build/temp.macosx-10.5-ppc-2.7/netCDF4/_netCDF4.o
gcc-mp-4.7: error: unrecognized option '-arch'
error: command '/mp2/local/bin/gcc-mp-4.7' failed with exit status 1
Command failed:  cd "/mp2/local/var/macports/build/_mp2_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_python_py-netcdf4/py27-netcdf4/work/netCDF4-1.1.8" && /mp2/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg build 
Exit code: 1

I've dealt with this problem before. I think I need to change the flags which contain "-arch ppc" to "-mcpu 7450". However, I am not sure how to handle this in Python's build system.

I am attaching the most recent build log.

[zzanderr]

Should I open another ticket for this error?

[ryandesign (Ryan Carsten Schmidt)]

I fixed the immediate problem in r140527, but I'm not comfortable with the way this port works. It is not MacPorts custom for a port to build itself differently depending on how another port was built, but that's what py-netcdf4 does: it chooses which compiler to use based on the compiler that was used for netcdf. With netcdf it's possible to discover what compiler was used by checking which variants are selected in "port installed netcdf", but py-netcdf4 doesn't have compiler variants so that's not possible. That would also mean that if a user gets a binary from our packages server it might have been built with a different compiler than the netcdf on the user's system. If it's really so important that py-netcdf4 uses the same compiler as netcdf, then py-netcdf4 needs to get the same variants that netcdf has, and the require_active_variants procedure in the active_variants 1.1 portgroup needs to be used to ensure the variants the user has selected for the two ports match.

[ryandesign (Ryan Carsten Schmidt)]

Cc Me!

[seanfarley (Sean Farley)]

Replying to ryandesign@…:

I fixed the immediate problem in r140527, but I'm not comfortable with the way this port works. It is not MacPorts custom for a port to build itself differently depending on how another port was built, but that's what py-netcdf4 does: it chooses which compiler to use based on the compiler that was used for netcdf. With netcdf it's possible to discover what compiler was used by checking which variants are selected in "port installed netcdf", but py-netcdf4 doesn't have compiler variants so that's not possible. That would also mean that if a user gets a binary from our packages server it might have been built with a different compiler than the netcdf on the user's system. If it's really so important that py-netcdf4 uses the same compiler as netcdf, then py-netcdf4 needs to get the same variants that netcdf has, and the require_active_variants procedure in the active_variants 1.1 portgroup needs to be used to ensure the variants the user has selected for the two ports match.

Almost certainly, this (and any other port that depends on hdf5) needs to use the mpi portgroup so that the compilers line up. I truly hate compiler wrappers. I can try to look at revamping the py-netcdf portfile but don't know when I'll have time.

[mojca (Mojca Miklavec)]

Cc Me!

[seanfarley (Sean Farley)]

I just tried to recreate this locally but even with ccache everything seemed to work. I'll still work on the mpi stuff, just to get it inline with netcdf4.