Continuous Integration of KDE software on MacPorts

Currently there is an ongoing discussion on KDE-DEVEL about this topic. MacPort's key developers' expertise regarding the setup of their buildbots could perhaps be helpful for the KDE folks. On the other hand one might also run Jenkins on OSX and adapt its configuration to the KDE software's requirements.

Requirements for the CI system

This is an excerpt from ​Ben Cooksley's post on KDE-DEVEL on thread "Running KDE apps on Apple OS X":

---

The ​infrastructure we have for our Linux builds should be nearly completely portable to OSX without too much trouble (in theory at least - i've never done any compilation on OSX).

The actual requirements aren't written anywhere, but you might find the ​documentation I've written for the CI system to be of some use.

A basic run down of what the CI system needs however:

1) Python 2.x, with json and lxml support (for the scripts which conduct builds)

2) Java (for the Jenkins node agent itself)

3) RSync and SSH (for the transferral of completed builds between nodes)

4) Git, Mercurial, Subversion, Bazaar, wget and GNU Tar (to access source code)

5) GNU Patch (for applying custom patches, used in certain builds)

6) A compiler, usable by CMake, QMake and autotools based build systems

7) GNU Make, Automake, Autoconf (for carrying out the build, and configuring it in certain rare cases)

We'll need to make adjustments to ensure the system doesn't attempt to launch Xvfb or a X11 Window Manager, which it currently will do when executing tests. These should be fairly easy to do however.

To build certain projects, the compiler will need to support C# and a certain level of C++11 as well. The Mono bindings will not be buildable if C# support is unavailable.

Under no circumstance should a CI node have Qt installed at the system level in any form, as this is provided directly by the CI system itself.

---

Checking on the above requirements in the current MacPorts tree shows that all necessary tools are available.
Which versions of these programs can be used with KDE's current CI system has to be checked step by step.

The only problem seems to be C# support. But those projects will probably only be running on Windows anyways.

Setting up the CI system (still hanging on OSX up to now)

This is what has to be done to get a KDE/CI system up and running on OSX (partially by also using MacPorts):

A) Start with fresh Mavericks install

B) Install MacPorts 2.2.1 using the official PKG

C) Install ports for bash use:

$ sudo port install bash bash-completion

D) Set up ​bash environment for MacPorts use regarding search paths (PATH, MANPATH)

E) Install Xcode from AppStore (and agree actively to the developer license agreement)

F) Install developer command line tools and install some of MacPorts' ports needed for later

$ xcode-select --install
$ sudo port install py27-lxml bazaar mercurial wget automake autoconf

Installing bazaar brings the also needed port gnutar into the system!

G) Since some tools are already installed on OSX due to Xcode (like git, svn, ssh, rsync, make, clang etc.) they don't need to be installed via MacPorts for now

H) Clone and update KDE's CI system:

$ git clone git://anongit.kde.org/websites/build-kde-org
$ cd build-kde-org
$ git checkout production
$ ./update-setup.sh   # Ignore 404 occurring due to error cloning ECMA262 by hg

I) Prepare environment for Qt5 build (set correct path to kde install directory in darwin.mavericks.cfg):

$ sudo port install dbus libmng libpng mysql56 pkgconfig sqlite2 tiff
$ cd ~/scripts
$ vi config/build/darwin-mavericks.cfg
$ python2.7 tools/prepare-environment.py --project qt5 --branchGroup kf5-qt5 --platform darwin-mavericks --sources /path/to/directory/where/qt5/will/be/built
...
$

J) Building Qt5 currently fails due to denied permissions:

$ python2.7 tools/perform-build.py --project qt5 --branchGroup kf5-qt5 --platform darwin-mavericks --sources /path/to/directory/where/qt5/will/be/built
.
.
.
== Syncing Dependencies from Master Server

Permission denied (publickey).
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [Receiver=3.0.9]
Syncing dependencies from master server for project qt5 failed.
$

OSX virtualisation

When setting up build-slaves it might be a good idea to not only have several Apple machines in real hardware, but perhaps also make use of some virtual OSX machines. This might speed up especially the test of various environments or give opportunity for an easy backup of the complete CI system.

See ​SOFTWARE LICENSE AGREEMENT FOR OS X MAVERICKS:

---

If OS X Mavericks is pre-installed “you are granted a limited, non-exclusive license to install, use and run one (1) copy of the Apple Software on a single Apple-branded computer at any one time”.

If OS X Mavericks is downloaded from the Mac App Store, “you are granted a limited, non-transferable, non-exclusive license (i) to download, install, use and run for personal, non-commercial use, one (1) copy of the Apple Software directly on each Apple-branded computer running OS X Mountain Lion, OS X Lion or OS X Snow Leopard (“Mac Computer”) that you own or control; … (iii) to install, use and run up to two (2) additional copies or instances of the Apple Software within virtual operating system environments on each Mac Computer you own or control that is already running the Apple Software, for purposes of: (a) software development; (b) testing during software development; (c) using OS X Server; or (d) personal, non-commercial use.”

---