#48025 closed defect (fixed)
Buildbot appears to be stuck
| Reported by: | ryandesign (Ryan Carsten Schmidt) | Owned by: | admin@… |
|---|---|---|---|
| Priority: | Normal | Milestone: | |
| Component: | server/hosting | Version: | |
| Keywords: | Cc: | mkae (Marko Käning) | |
| Port: |
Description
The main buildbot interface appears to be stuck. No builds are occurring on any builder.
Change History (9)
comment:1 follow-up: 2 Changed 9 years ago by ryandesign (Ryan Carsten Schmidt)
comment:2 Changed 9 years ago by dbevans (David B. Evans)
Replying to ryandesign@…:
To provide more information: you can see on the console page that all the boxes are white, which means the builds are pending. Looking at the builders page, you can see that all builders except the Yosemite builders are idle, but with pending builds; the builders should be performing those pending builds. For the Yosemite builders, it shows that they are building, but they aren't. For example the Yosemite ports builder shows it is compiling for build 4411. But looking at build 4411 in more detail, it shows it was interruped because Dave pressed the stop button on the web site, presumably because he thought the build was stuck. Normally, buildbot should then move on to the next build, but this has not happened.
In fact, it WAS stuck. If you look at the build that I tried to stop, the compile phase had previously timed out after not producing any output for 1200 seconds (20 minutes) and tried to kill the build process but without success. I thought that the stop button might succeed where the timeout had not but no luck.
comment:3 Changed 9 years ago by ryandesign (Ryan Carsten Schmidt)
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Keith resolved this issue by rebooting the build.macports.org server.
comment:4 follow-up: 7 Changed 9 years ago by mkae (Marko Käning)
| Resolution: | fixed |
|---|---|
| Status: | closed → reopened |
How can it be that it took more than a week to reboot the server?!
Does MacPorts get so little attention at Apple? (I am left somewhat surprised...)
I see there is a separate ticket (#48116) for the still stuck Yosemite server.
comment:6 Changed 9 years ago by mkae (Marko Käning)
| Resolution: | → fixed |
|---|---|
| Status: | reopened → closed |
comment:7 Changed 9 years ago by ryandesign (Ryan Carsten Schmidt)
Replying to mk@…:
How can it be that it took more than a week to reboot the server?!
Does MacPorts get so little attention at Apple? (I am left somewhat surprised...)
See comment:ticket:47976:8. The problem is that everyone who knew how MacPorts infrastructure at Mac OS Forge was set up has left, and our new admin contact is new to the team and is trying to figure out how everything works while also tasked with other duties. For the time being, when we file infrastructure issues, we should provide as much explicit detail as we can on how the problem might be resolved.
I see there is a separate ticket (#48116) for the still stuck Yosemite server.
Yes, and #47976 remains open for the Snow Leopard builder issue.
comment:8 follow-up: 9 Changed 9 years ago by mkae (Marko Käning)
OK, thanks for the hints. I've subscribed to the other tickets too. Wondering why the SL issue is still open, as the SL builder seems to be the first to be idle after catching up with building all those queued ports, no?!
comment:9 Changed 9 years ago by ryandesign (Ryan Carsten Schmidt)
Replying to mk@…:
Wondering why the SL issue is still open, as the SL builder seems to be the first to be idle after catching up with building all those queued ports, no?!
The Snow Leopard builder isn't actually building anything. It's all described in the other ticket.
To provide more information: you can see on the console page that all the boxes are white, which means the builds are pending. Looking at the builders page, you can see that all builders except the Yosemite builders are idle, but with pending builds; the builders should be performing those pending builds. For the Yosemite builders, it shows that they are building, but they aren't. For example the Yosemite ports builder shows it is compiling for build 4411. But looking at build 4411 in more detail, it shows it was interruped because Dave pressed the stop button on the web site, presumably because he thought the build was stuck. Normally, buildbot should then move on to the next build, but this has not happened.
I don't know why the buildbot is in this state; I don't recall this happening before. But it doesn't appear to be a problem specific to any of the 5 OS X builders, since they are all affected. So I would turn my attention to whatever machine is driving the builders. I am not familiar with buildbot infrastructure so I don't know if that's the machine where we see the web interface (build.macports.org) or yet another server.