Git + buildbot: post-commit-hook selecting which buildbots should be used for CI

[mkae (Marko Käning)]

On 01 Nov 2016, at 02:02 , Rainer Müller wrote:

Can we maybe decide on special commands in commit messages that cause the buildbot to completely ignore the commit? Such things should not limit us to what we can commit or not.

Perhaps a footer line (similar to Signed-off-by) at the end, such as:

buildbot: ignore

For reference, other services seem to use "[ci skip]" as a keyword that can appear anywhere in the commit message.

+1

I’d also suggest to use this also to specify which buildbots should be triggered for a commit along those lines:

buildbot: Mavericks Sierra

I think that can be very helpful in some cases.

[mkae (Marko Käning)]

Sorry, @raimue, I meant to enter your handle into the CC field, but I inadvertently made you owner of this ticket!

[neverpanic (Clemens Lang)]

Larry has worked on the buildbot lately and provided the GitHub webhook integration. He might now if there's an easy way to do this.

[larryv (Lawrence Velázquez)]

From a policy standpoint: I don’t like the idea of polluting our (permanent!) commit history with transient administrivia like this. Will continue discussing on macports-dev.

From a technical standpoint:

• It would be easy enough to filter out something simple like “buildbot: ignore”. Not so easy to parse OS names and match them to builders.
• Either Ryan or Rainer (can’t recall at the moment, would have to search email) has expressed a desire to remove builder-aware logic from master.cfg, which I agree with. This would restrict us to substring/regex-based line noise, like

  buildbot: ports-10.9_x86_64 ports-10.12_x86_64
  

• Accepted changes get sent to all builders. I don’t know off the top of my head whether there is a way for builders to filter requests.

[raimue (Rainer Müller)]

This Stackoverflow answer describes a possible implementation of what we want: ​http://stackoverflow.com/a/30670892/67498

[larryv (Lawrence Velázquez)]

This "modify a magic file" idea is intriguing.

[neverpanic (Clemens Lang)]

How is modifying a magic file (which is also in the history of the repository) different from putting the skip command in the commit message? Additionally, the magic file potentially increases the likelihood of merge conflicts and puts a useless file into the repository. I'd rather see the commit message approach than the file approach.

[mkae (Marko Käning)]

Replying to neverpanic:

... I'd rather see the commit message approach than the file approach.

+1

The KDE folks also control a variety of git hooks through their commit messages. See ​their commit template. I believe we should also have a common template for our devs which ​takes all the hints found in our "Commit messages" section of the "Working with git" wiki page on board.

[larryv (Lawrence Velázquez)]

I don’t think it’s necessary or desirable to bother with this. In any case, wholly off-topic.

[larryv (Lawrence Velázquez)]

Replying to neverpanic:

How is modifying a magic file (which is also in the history of the repository) different from putting the skip command in the commit message?

The difference is that one won’t see the magic file when one is looking through a list of commit messages.

If you want to argue that it pollutes the list of changes…I can’t really deny it. Different side of the same coin, I guess.

Additionally, the magic file potentially increases the likelihood of merge conflicts

That’s true, but I’m assuming that skipping out on the buildbot run would be a rare thing. If people start doing it whenever they don’t feel like getting a build failure email (which I’m frankly afraid they will), I’d rather we not implement this at all and just manually disable the webhook on the rare occasions that we need to do a mass commit.

and puts a useless file into the repository.

Got me there.

[ryandesign (Ryan Carsten Schmidt)]

The buildbot is supposed to automatically build all changes. Why do we want to give committers the ability to override that?

I feel there are real underlying problems that we should be fixing instead. For example, annoyance with failure emails due to known failures needs to be addressed by allowing a portfile syntax for indicating that a port is known to fail in certain configurations, not by making the committer exclude a particular builder every time they commit to that port by using commit message syntax.

[mkae (Marko Käning)]

Replying to ryandesign:

The buildbot is supposed to automatically build all changes. Why do we want to give committers the ability to override that?

Very simple: in case a committer knows in advance that only a specific platform is affected. Why to bug all other builders with Sierra-related build trials if you know that those won't give any valuable result on older platforms?!?

This potentially could lower the load on our buildbot system during the development of port (group) files which require testing on a buildbot.

P.S.: But, as always, this is only a suggestion (from someone not professionally involved in any kind of software development, so my ideas might be misleading/irrelevant).

[ryandesign (Ryan Carsten Schmidt)]

Replying to mkae:

Very simple: in case a committer knows in advance that only a specific platform is affected. Why to bug all other builders with Sierra-related build trials if you know that those won't give any valuable result on older platforms?!?

This potentially could lower the load on our buildbot system during the development of port (group) files which require testing on a buildbot.

The ideal solution in that case is to verify your changes on your local Sierra system before committing. If you have no local Sierra-capable system then committing hypothetical fixes and letting the buildbot test them is acceptable. We should implement the "successcache" to reduce the load on the buildbot.

[mkae (Marko Käning)]

Replying to ryandesign:

The ideal solution in that case is to verify your changes on your local Sierra system before committing.

Of course it is. That's what I always do. Still, sometimes things slip through, because it is sometimes not so easy to have everything identically set up like it is for the buildbots. I do try not to cause trouble, but I do sometimes and for that I'd like a feature like this which decreases the damage which I could do.

If you have no local Sierra-capable system then committing hypothetical fixes and letting the buildbot test them is acceptable.

I know.

We should implement the "successcache" to reduce the load on the buildbot.

I didn't know about this ticket.

[larryv (Lawrence Velázquez)]

Replying to ryandesign:

The buildbot is supposed to automatically build all changes. Why do we want to give committers the ability to override that?

I am not keen on "buildbot: ignore" and am strongly opposed to the individual builder selection.

[mojca (Mojca Miklavec)]

I would say that what we desperately need is:

• "success cache" (#52765)
• a way to disable builds for known-to-be-broken configurations (for example disable ports that require C++11 on certain platforms, disable software that fails to build on newer OSes, ...); not sure if there's a ticket, but Ryan posted an email to the list a while ago
• more efficient setup to avoid crashing the system after a "mass commit"
  • less important: being able to merge multiple commits into a single portwatcher job

Once we have that in place, I don't see that much need for hacking buildbot instructions into commit messages. If one port fails to build just on Sierra and builds successfully everywhere else, the next commit doesn't have to do a revbump and then the buildbot would only consume resources of Sierra anyway.

[mkae (Marko Käning)]

Fine, so, feel free to close this ticket as 'wontfix' then, I am not insisting on it. :-)

[neverpanic (Clemens Lang)]

We don't want to give developers a method to manually control this. Instead, we're going to implement #15712.