pypy: gcc uses too much memory

[macports.org@…]

Yesterday and today I tried updating my pypy 1.7 to 1.8, on a 2010 MBP with 8GB RAM.

In both cases, the machine got swapped to death. The second time around I looked into what was happening, and I observed the driver Python process using ~2.5GB RAM (real memory) and 4 CC processes using 1~1.5GB RAM each. While using multiple jobs is good when the translation is CPU bound, forcing the machine to do everything on-disk because everything gets swapped out is not.

I looked into it more, and apparently I'm hitting ​a wontfixed GCC 4.2 bug which makes GCC eat insane amounts of memory when -O1 or more is enabled. This may not be an issue in Lion depending on its GCC version.

As a result, I think forcing --make-jobs to 1 (or removing SMP compilation altogether) in 10.6 would be a good idea.

[ryandesign (Ryan Carsten Schmidt)]

Apple will never ship a version of GCC newer than 4.2.1 because GCC changed its license to GPL 3 and Apple does not like that license. As of Xcode 4.2, there is no GCC in Xcode; with Xcode 4.0 and 4.1 we use llvm-gcc-4.2 instead of GCC; in Xcode 4.2 we use clang.

In other ports that use lots of memory like this at compile time, I've simply disabled the parallel build. See #29547. We could potentially revisit whether we should only do that when the compiler is GCC.

[macports.org@…]

Replying to ryandesign@…:

Apple will never ship a version of GCC newer than 4.2.1 because GCC changed its license to GPL 3 and Apple does not like that license. As of Xcode 4.2, there is no GCC in Xcode; with Xcode 4.0 and 4.1 we use llvm-gcc-4.2 instead of GCC; in Xcode 4.2 we use clang.

In other ports that use lots of memory like this at compile time, I've simply disabled the parallel build. See #29547. We could potentially revisit whether we should only do that when the compiler is GCC.

An alternative could be to force gcc to 4.0 (in 10.6 cc is gcc 4.2, but 4.0 is also available, not sure about 10.7) for these problematic ports if it's compatible, that's what I ended up doing (manually) in order to compile pypy, via configure.cc=gcc-4.0.

Disabling parallel build work, although it's probably not as good since Pypy's translation already burns through a fuckton of RAM without taking gcc in account (hitting gcc's bug still results in the GCC process growing to the gigabyte range)

Side-note, I saw that translating pypy 1.8 via 1.7 was disabled in the portfile (while looking into knobs to get my translation working), I tried re-enabling it and it worked without any issue (using macport's 1.7, not a separate source checkout or anything).

[jmroot (Joshua Root)]

r90264