pypy: Buildbots failed to build PyPy for SnowLeopard, Lion and MountainLion because of a strict timeout

[someuser12]

During last update of the pypy port (r120871 at 2014-06-10 15:12:03Z), the automatically triggered builds by the buildbots failed for the platforms Snow Leopard, Lion and Moutain Lion because, at one stage during the build, there is no output for more than 1200 seconds. The build succeeded for Mavericks.

The logs:
​https://build.macports.org/builders/buildports-snowleopard-x86_64/builds/26910/steps/compile/logs/stdio
​https://build.macports.org/builders/buildports-lion-x86_64/builds/20726/steps/compile/logs/stdio
​https://build.macports.org/builders/buildports-mtln-x86_64/builds/14602/steps/compile/logs/stdio
​https://build.macports.org/builders/buildports-mavericks-x86_64/builds/4002/steps/compile/logs/stdio

Would it be possible to increase this timeout so that all buildbots succeed? (It would be nice to have a binary for this port since the build lasts for more than 3 hours and takes a lot of memory.) Alternatively, one could ask the developers to add progress messages during the "[rtyper]" steps.

Thanks!

[ryandesign (Ryan Carsten Schmidt)]

As I understand it, our 4 buildslaves are virtual machines running on a single physical Mac. And when a revision is committed, a build is kicked off on all buildslaves simultaneously, so they're all competing for the same physical resources. I'm not sure if the pypy build uses multiple processor cores, but if it does, only ¼ of them would actually be available when the 4 buildslaves are going at it at the same time.

I forced a build of pypy on the Mountain Lion buildslave only, and it succeeded after 2 hours 45 minutes. Same on the Snow Leopard buildslave. These binary packages are now available. I've tried the same on the Lion buildslave twice now, and it failed both times after over 2 hours. It could be that the older version of clang on the Lion server takes longer to compile, pushing this particular object over the 20-minute limit we've set.

It does seem pretty egregious that any build system would work for over 20 minutes without any output at all. Are the developers of pypy aware they are doing this?

We could increase this timeout on the buildbot, but what value should we set it to?

[ryandesign (Ryan Carsten Schmidt)]

The update to 2.4.0 built successfully on all 4 build servers.

[petrrr]

Cc Me!

[petrrr]

This might be a hosting ticket instead. Assigning maintainer for review.

[jmroot (Joshua Root)]

I patched the build system to print the output "live" a long time ago. (It was originally saved and only printed if there was an error.) Not sure if that was before or after this ticket was opened, but as mentioned it seems to build fine now.