Opened 8 years ago
Last modified 2 years ago
#52742 assigned enhancement
buildbot: add support for building ports with variants
Reported by: | mojca (Mojca Miklavec) | Owned by: | neverpanic (Clemens Lang) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | buildbot/mpbb | Version: | |
Keywords: | buildbot | Cc: | larryv (Lawrence Velázquez), raimue (Rainer Müller), neverpanic (Clemens Lang), ryandesign (Ryan Carsten Schmidt), ctreleaven (Craig Treleaven), danchr (Dan Villiom Podlaski Christiansen), yan12125 (Chih-Hsuan Yen), l2dy (Zero King), dliessi (Davide Liessi) |
Port: |
Description
It would be very handy to have some limited support for building with variants.
Before trashing the subversion checkout on my buildbot, I wanted to attach partial patches.
See #51995.
Attachments (3)
Change History (23)
Changed 8 years ago by mojca (Mojca Miklavec)
Attachment: | variants-master.cfg.diff added |
---|
Changed 8 years ago by mojca (Mojca Miklavec)
Attachment: | variants-mpbb-fully-outdated.diff added |
---|
outdated patch for basic support of variants in mpbb
comment:1 Changed 8 years ago by mojca (Mojca Miklavec)
The patches are pretty outdated, but they should give the first impression. I wanted to fix the second one, but I should probably write some code to parse options between install-port
and <portname>
first.
Some hints by Clemens about how to proceed:
tools/dependencies.tcl
just opens all ports without any specified variants. Seetools/dependencies.tcl
line 66; the last parameter tomportopen
, which is currently{}
, is the list of variants. The format is easy to create:array set variations {} set variations(quartz) "+" set variations(x11) "-" mportopen ... [array get variations]gives you an mport reference as if the port was installed with
+quartz-x11
. You should be able to find some suitable parsing code for variants given on the command line insrc/port/port.tcl
, line 1695 and following.Looking at that file I have an impression that we forgot to consider global settings (
[array get global_variations]
):if {!$index_only} { # Add any global_variations to the variations # specified for the port (so we get e.g. dependencies right) array unset merged_variations array set merged_variations [array get variations] foreach { variation value } [array get global_variations] { if { ![info exists merged_variations($variation)] } { set merged_variations($variation) $value } }This means that even if someone would (theoretically) set up a buildbot with
+quartz -x11
in configuration file, those setting would probably be ignored. Or do I misunderstand the concept?Yes, I think the file doesn't handle global variants. I think it's also bad design that
port/port.tcl
has to do the merging of variant specs without at least some library support frommacports1.0
, but that's a story for a different refactoring.
An additional warning:
However, this output will only list positive variants, it will not list default variants that have been explicitly disabled. This might be a problem, because specifying
-x11 +quartz
will just give you a list of all dependencies with+quartz
, but not without+x11
, so if a port is written in a way that it does not automatically disablex11
if+quartz
is given (for example if both are supported simultaneously andx11
is the default), this might lead to unexpected results. I haven't tested any of this, but we probably should.
comment:2 Changed 8 years ago by mojca (Mojca Miklavec)
Summary: | buildbot: add support for building variants → buildbot: add support for building ports with variants |
---|
comment:3 Changed 8 years ago by mojca (Mojca Miklavec)
It looks like the core of the needed functionality was already added in r152489.
comment:4 Changed 8 years ago by jmroot (Joshua Root)
Some of the logging probably needs to be enhanced with variant info, but that's about it AFAICT.
comment:5 Changed 8 years ago by mojca (Mojca Miklavec)
I didn't check the logs, but among other things we also need:
- (straightforward/trivial) a revised version of my old attempt to add a field for variants to
master.cfg
- (lower priority) some basic logic to automatically build
+quartz
for certain ports
comment:6 Changed 8 years ago by mojca (Mojca Miklavec)
Keywords: | buildbot added |
---|---|
Port: | buildbot removed |
Changed 8 years ago by mojca (Mojca Miklavec)
Attachment: | variants-master.cfg.2.diff added |
---|
add a field to support variants
comment:7 Changed 8 years ago by mojca (Mojca Miklavec)
The new (trivial) patch for master.cfg
seems to work for me.
We need some basic safeguard (regexp?) though to make sure that only valid variant names are entered, otherwise people could end up writing port names or any other weird things. (I could imagine things equivalent to sql injections ending up here. Nothing to do with sql, but I hope you get the idea.)
This will be sufficient to get gkt3 +quartz
and other binaries on the buildbots. The next (independent) step would be to write the logic to build that automatically for some ports.
comment:8 Changed 8 years ago by ctreleaven (Craig Treleaven)
Cc: | ctreleaven added |
---|
comment:9 Changed 7 years ago by danchr (Dan Villiom Podlaski Christiansen)
Cc: | danchr added |
---|
comment:10 Changed 7 years ago by yan12125 (Chih-Hsuan Yen)
Cc: | yan12125 added |
---|
comment:11 Changed 7 years ago by l2dy (Zero King)
Cc: | l2dy added |
---|
comment:12 Changed 7 years ago by dliessi (Davide Liessi)
Cc: | dliessi added |
---|
comment:13 Changed 7 years ago by neverpanic (Clemens Lang)
Component: | server/hosting → buildbot/mpbb |
---|
comment:14 Changed 7 years ago by neverpanic (Clemens Lang)
I've been working on getting this working, ideally even without modifications to the master.cfg of the buildbot, but I've hit some problems. [f6e46815a5033358212ce383956fd567eb51e53c/mpbb] changed the variant handling code in mpbb in a way that will cause failures when requesting specific variants to be installed. See the commit on GitHub for a description of the problem. We would have to revert this commit for this effort to proceed, but I'd like to wait for feedback from Joshua before doing that.
comment:15 Changed 7 years ago by neverpanic (Clemens Lang)
comment:16 Changed 7 years ago by jmroot (Joshua Root)
I don't doubt the current code needs changes to work right when specifying arbitrary variants. Simply reverting that commit is not right either, though. The distinction between the canonical_active_variants and the variants that are being requested needs to be maintained.
comment:17 Changed 7 years ago by neverpanic (Clemens Lang)
You're right. I thought about this yesterday and came to the conclusion that this is really a bug in base, since base propagates these variants down the dependency tree and attempts to install a combination that will not work.
Either the glib2
Portfile should ignore the user's request to change the variant and forcefully set a variant, or base should not request this invalid combination.
comment:18 Changed 7 years ago by neverpanic (Clemens Lang)
comment:19 Changed 7 years ago by neverpanic (Clemens Lang)
Owner: | changed from admin@… to neverpanic |
---|---|
Status: | new → assigned |
patches for master.cfg that add support for building with arbitrary variants (no sanity checking yet!)