Opened 11 years ago

Closed 9 years ago

#42220 closed defect (worksforme)

py-netcdf4 @1.0.4_1: build failure when using ccache

Reported by: cooljeanius (Eric Gallager) Owned by: seanfarley (Sean Farley)
Priority: Normal Milestone:
Component: ports Version: 2.2.1
Keywords: Cc: petrrr, jswhit (Jeff Whitaker), ryandesign (Ryan Carsten Schmidt), mojca (Mojca Miklavec)
Port: py-netcdf4

Description

I have configureccache set to yes in my macports.conf file and ran into this error when trying to build py26-netcdf:

running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'netCDF4' extension
compiling C sources
C compiler: {ccache /usr/bin/gcc-4.2} -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch x86_64 -arch i386

creating build/temp.macosx-10.6-x86_64-2.6
compile options: '-I/opt/local/include -I/opt/local/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c'
{ccache: netCDF4.c
sh: {ccache: command not found
sh: {ccache: command not found
error: Command "{ccache /usr/bin/gcc-4.2} -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch x86_64 -arch i386 -I/opt/local/include -I/opt/local/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c netCDF4.c -o build/temp.macosx-10.6-x86_64-2.6/netCDF4.o" failed with exit status 127
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_python_py-netcdf4/py26-netcdf4/work/netCDF4-1.0.4" && /opt/local/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 setup.py --no-user-cfg build 
Exit code: 1
Error: org.macports.build for port py26-netcdf4 returned: command execution failed
DEBUG: Error code: CHILDSTATUS 28040 1
DEBUG: Backtrace: command execution failed

Attachments (3)

main.log (10.1 KB) - added by cooljeanius (Eric Gallager) 11 years ago.
main.log for py26-netcdf4
main.2.log (107.2 KB) - added by zzanderr 9 years ago.
build log for py27-netcdf4 @1.1.8
main.3.log (9.2 KB) - added by zzanderr 9 years ago.

Download all attachments as: .zip

Change History (22)

Changed 11 years ago by cooljeanius (Eric Gallager)

Attachment: main.log added

main.log for py26-netcdf4

comment:1 Changed 10 years ago by cooljeanius (Eric Gallager)

Cc: egall@… added

Edit: Oops, I forgot that this was a ticket that I had filed myself, feel free to remove my extraneous cc...

Last edited 10 years ago by cooljeanius (Eric Gallager) (previous) (diff)

comment:2 Changed 10 years ago by cooljeanius (Eric Gallager)

Anyways the reason I came back to this ticket was because this is an issue with the other py-netcdf4 sub-ports as well, and it is still an issue as of version @1.1.0...

comment:3 Changed 10 years ago by petrrr

Cc: petr@… added

Cc Me!

comment:4 Changed 10 years ago by petrrr

Cc: egall@… removed
Summary: py26-netcdf4 @1.0.4_1: build failure when using ccachepy-netcdf4 @1.0.4_1: build failure when using ccache

comment:5 Changed 10 years ago by petrrr

Port: py-netcdf4 added; py26-netcdf4 removed

comment:6 Changed 10 years ago by petrrr

BTW: you should be able to remove you own CC address at any time.

comment:7 Changed 9 years ago by zzanderr

I see that this is an old ticket, but I too am getting the same error when trying to upgrade py27-netcdf4. I have tried deactivating ccache, and even uninstalling ccache (of course along with appropriately modifying the settings on macports.conf and the PATH variable in my .bash_profile. All to no avail. The first--and the only previous--install of py27-netcdf4 worked okay, and I do not remember having to mess around with ccache, which was almost certainly active at the time. Now no matter what I have tried thus far gives me the same build error:

:info:build {ccache /mp2/local/bin/gcc-mp-4.7} -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch ppc -I/mp2/local/include -I/mp2/local/include -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c netCDF4/_netCDF4.c -o build/temp.macosx-10.5-ppc-2.7/netCDF4/_netCDF4.o
:info:build unable to execute '{ccache': No such file or directory
:info:build error: command '{ccache' failed with exit status 1

Note: I have also tried deactivating the old py27-netcdf4 and building from scratch. Still the same error. Weird! Any ideas? I am attaching the build log.

Changed 9 years ago by zzanderr

Attachment: main.2.log added

build log for py27-netcdf4 @1.1.8

comment:8 Changed 9 years ago by zzanderr

P.S. Why does py27-netcdf4 apparently insist on using ccache? Could there be a permissions problem on the .ccache directory for this particular port?

comment:9 Changed 9 years ago by gnw3

What is the output of nc-config -cc? If netcdf was built using ccache, you should see something like:

$ ccache /opt/local/bin/gcc-mp-4.8

If you edit $prefix/bin/nc-config to remove the "ccache " then the build should work, but someone needs to look at how the nc-config output is handled by the Python setup.py.

comment:10 Changed 9 years ago by mf2k (Frank Schima)

Cc: jswhit@… added
Owner: changed from macports-tickets@… to sean@…

Cc'ing current maintainers.

comment:11 Changed 9 years ago by seanfarley (Sean Farley)

Thanks. Seems like gnwii is on this one. I'll be following to see if anything needs to be changed in the Portfile.

comment:12 Changed 9 years ago by zzanderr

Sorry! I've been on vacation. Thanks for your prompt response!

As to gnwii's question, here is what I get (using the original py27-netcdf4, of course):

$ nc-config --cc
ccache /mp2/local/bin/gcc-mp-4.7

I will try building as gnwii suggests...

comment:13 Changed 9 years ago by zzanderr

Oof! Now the build fails with the following:

running build_ext
building 'netCDF4._netCDF4' extension
creating build/temp.macosx-10.5-ppc-2.7
creating build/temp.macosx-10.5-ppc-2.7/netCDF4
/mp2/local/bin/gcc-mp-4.7 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch ppc -I/mp2/local/include -I/mp2/local/include -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c netCDF4/_netCDF4.c -o build/temp.macosx-10.5-ppc-2.7/netCDF4/_netCDF4.o
gcc-mp-4.7: error: unrecognized option '-arch'
error: command '/mp2/local/bin/gcc-mp-4.7' failed with exit status 1
Command failed:  cd "/mp2/local/var/macports/build/_mp2_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_python_py-netcdf4/py27-netcdf4/work/netCDF4-1.1.8" && /mp2/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg build 
Exit code: 1

I've dealt with this problem before. I think I need to change the flags which contain "-arch ppc" to "-mcpu 7450". However, I am not sure how to handle this in Python's build system.

I am attaching the most recent build log.

Changed 9 years ago by zzanderr

Attachment: main.3.log added

comment:14 Changed 9 years ago by zzanderr

Should I open another ticket for this error?

comment:15 Changed 9 years ago by ryandesign (Ryan Carsten Schmidt)

I fixed the immediate problem in r140527, but I'm not comfortable with the way this port works. It is not MacPorts custom for a port to build itself differently depending on how another port was built, but that's what py-netcdf4 does: it chooses which compiler to use based on the compiler that was used for netcdf. With netcdf it's possible to discover what compiler was used by checking which variants are selected in "port installed netcdf", but py-netcdf4 doesn't have compiler variants so that's not possible. That would also mean that if a user gets a binary from our packages server it might have been built with a different compiler than the netcdf on the user's system. If it's really so important that py-netcdf4 uses the same compiler as netcdf, then py-netcdf4 needs to get the same variants that netcdf has, and the require_active_variants procedure in the active_variants 1.1 portgroup needs to be used to ensure the variants the user has selected for the two ports match.

comment:16 Changed 9 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: ryandesign@… added

Cc Me!

comment:17 in reply to:  15 Changed 9 years ago by seanfarley (Sean Farley)

Replying to ryandesign@…:

I fixed the immediate problem in r140527, but I'm not comfortable with the way this port works. It is not MacPorts custom for a port to build itself differently depending on how another port was built, but that's what py-netcdf4 does: it chooses which compiler to use based on the compiler that was used for netcdf. With netcdf it's possible to discover what compiler was used by checking which variants are selected in "port installed netcdf", but py-netcdf4 doesn't have compiler variants so that's not possible. That would also mean that if a user gets a binary from our packages server it might have been built with a different compiler than the netcdf on the user's system. If it's really so important that py-netcdf4 uses the same compiler as netcdf, then py-netcdf4 needs to get the same variants that netcdf has, and the require_active_variants procedure in the active_variants 1.1 portgroup needs to be used to ensure the variants the user has selected for the two ports match.

Almost certainly, this (and any other port that depends on hdf5) needs to use the mpi portgroup so that the compilers line up. I truly hate compiler wrappers. I can try to look at revamping the py-netcdf portfile but don't know when I'll have time.

comment:18 Changed 9 years ago by mojca (Mojca Miklavec)

Cc: mojca@… added

Cc Me!

comment:19 Changed 9 years ago by seanfarley (Sean Farley)

Resolution: worksforme
Status: newclosed

I just tried to recreate this locally but even with ccache everything seemed to work. I'll still work on the mpi stuff, just to get it inline with netcdf4.

Note: See TracTickets for help on using tickets.