[[PageOutline]] = Current status of setting up the CI system = This page shall describe what has to be done to get a KDE/CI system up and running on OSX. Partially this is achieved by using a few port from MacPorts. == Start with a fresh Mavericks/MacPorts install == Currently using the official MacPorts version 2.3.0 is recommended. To be on the safe side make sure your disk has about 60G space in total. However, currently a Mavericks 10.9.2 KDE/CI system on an i7-iMac uses something like this: {{{ $ df -h / Filesystem Size Used Avail Use% Mounted on /dev/disk0s2 64G 36G 29G 56% / }}} that is after all frameworks have been built and installed as shown further down. == Getting a MacPorts-compatible bash environment == Install ports for MacPorts' bash {{{ $ sudo port install bash bash-completion }}} and set up the [https://projects.kde.org/projects/playground/sdk/macports-kde/repository/revisions/master/show/contrib/user-setup bash environment for MacPorts use] regarding search paths (PATH, MANPATH) == Apple developer tools == Install * Xcode from AppStore (and agree actively to the developer license agreement) * developer command line tools {{{ $ xcode-select --install }}} == Tools already present on OSX == Some tools are already installed on OSX due to Xcode (like git, svn, ssh, rsync, make, clang etc.) they don't need to be installed via MacPorts for now. Should a more up-to-date version of any one of these be necessary for the CI system it could be added later. == Some more software supplied by MacPorts == Install ports needed for the CI scripts as well as KF5: {{{ $ sudo port install gnutar py27-lxml bazaar mercurial wget automake autoconf cppcheck shared-mime-info docbook docbook-xsl giflib boost }}} == Basic setup of CI system == Create some folders, clone and update CI system (assuming a user "kdeci" with its home directory "/Users/kdeci"): {{{ $ cd $ mkdir -p WC/KDECI-build; $ cd WC $ git clone git://anongit.kde.org/websites/build-kde-org $ cd build-kde-org $ git checkout production $ ./update-setup.sh # Ignore 404 occurring due to error cloning ECMA262 by hg }}} The update shell script created a folder {{{~/scripts}}} in which subsequent commands will have to be executed. == 1st step: Qt5 build == === Prepare environment for Qt5 build === Install additional ports specific for Qt5 as well as KF5, set correct path to KDE-install directory in config/build/darwin-mavericks.cfg: {{{ $ sudo port install zlib openssl dbus jpeg tiff libmng libpng mysql55 pkgconfig sqlite2 $ sudo port uninstall cmake # Remove MacPorts' cmake (which came into the system as build-dependency for mysql55) $ cd ~/scripts $ cat config/build/darwin-mavericks.cfg [DEFAULT] scriptsLocation=/Users/kdeci/scripts opSys=darwin architecture=mavericks compiler=clang [General] installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ $ python2.7 tools/prepare-environment.py --project qt5 --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/qt5 ... $ }}} === Set up build environment for Qt5 === Make sure all libs can be found by adding search paths and avoid inclusion of MacPorts' glib2: {{{ $ cd ~/scripts; cat config/build/qt5/darwin-mavericks.cfg [Build] configureCommand=%(configureExecutable)s -release -system-zlib -system-libpng -system-libjpeg -system-sqlite -dbus -plugin-sql-mysql -nomake examples -confirm-license -opensource -prefix {instPrefix} -no-framework -v -I/opt/local/include -L/opt/local/lib -no-glib }}} Consider here the use of options "{{{-debug -separate-debug-info}}}" instead of "{{{-release}}}". Include MySQL's binary path into PATH e.g. in {{{.macports/profile}}} (assuming /opt/local as MacPort's prefix): {{{ export PATH=/opt/local/lib/mysql55/bin:$PATH }}} Set up SSH for rsync access to build host, then start building Qt5: {{{ $ cat ~/.ssh/config Host build.kde.org Port 2022 $ ssh-keygen -C YOUR@EMAIL.ADDRESS # <-- send your public SSH key to Ben Cooksley in order to get access to the build server $ python2.7 tools/perform-build.py --project qt5 --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/qt5 ... }}} == Installation of cmake == Cmake needs a manual checkout of its sources: {{{ $ python2.7 tools/prepare-environment.py --project cmake --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/cmake ... $ git clone git://cmake.org/cmake.git ~/WC/KDECI-build/cmake $ python2.7 tools/perform-build.py --project cmake --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/cmake ... }}} == Keeping build environment up-to-date == For KF5 one has to modify the configuration settings by commenting out the kauth dependency on polkit-qt-1, since that dependency is only needed on Linux: {{{ $ cd ~/scripts $ ./update-setup.sh $ vi config/base/kf5-qt5 $ grep kauth config/base/kf5-qt5 #frameworks/kauth: kdesupport/polkit-qt-1 $ }}} Problems appeared for some frameworks since kconfig_compiler_kf5.app doesn't yet get installed on OSX (the build scripts error out with "/Applications/KDE/kconfig_compiler_kf5.app/Contents/MacOS/kconfig_compiler_kf5 as location is wrong"). This can be fixed for now introducing a temporary configuration file for kconfig: {{{ $ cat ~/scripts/config/build/kconfig/darwin-mavericks.cfg [DEFAULT] configureExtraArgs=-DCMAKE_INSTALL_BUNDLEDIR="lib/libexec/kf5" }}} which makes sure that the application bundle gets installed below PREFIX/lib/libexec/. '''{It seems like that this specific issue has been identified as a kconfig bug and got resolved in the meantime in revision 70193760afc52ebc1ab99c9e467b9df470d9e39a and the workaround is therefore not needed anymore!}''' In order to always have the latest KDE/CI running it makes sense to temporally stash away the changes made locally and upgrade via git {{{ $ # Check what has changed locally: $ git diff diff --git a/config/base/kf5-qt5 b/config/base/kf5-qt5 index 16abdf6..fff9647 100644 --- a/config/base/kf5-qt5 +++ b/config/base/kf5-qt5 @@ -18,7 +18,7 @@ general/vc: -qt5 general/shared-desktop-ontologies: -qt5 # KDE Frameworks -frameworks/kauth: kdesupport/polkit-qt-1 +#frameworks/kauth: kdesupport/polkit-qt-1 kde/*: general/kdesupport-svn kde/workspace/plasma-workspace: general/libdbusmenu-qt diff --git a/config/build/darwin-mavericks.cfg b/config/build/darwin-mavericks.cfg index cb62c49..0648464 100644 --- a/config/build/darwin-mavericks.cfg +++ b/config/build/darwin-mavericks.cfg @@ -1,7 +1,9 @@ [DEFAULT] +scriptsLocation=/Users/kdeci/scripts opSys=darwin architecture=mavericks compiler=clang +configurePlatformArgs=-DCMAKE_INSTALL_BUNDLEDIR="Applications/KF5" -DDATA_INSTALL_DIR="Library/Application Support" [General] -installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ \ No newline at end of file +installPrefix=/opt/kde/install/%(opSys)s/%(architecture)s/%(compiler)s/ $ $ git status On branch production Your branch is up-to-date with 'origin/production'. Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: config/base/kf5-qt5 modified: config/build/darwin-mavericks.cfg Untracked files: (use "git add ..." to include in what will be committed) build.sh dependencies/ kapidox/ kde_projects.xml poppler-test-data/ prepare.sh no changes added to commit (use "git add" and/or "git commit -a") $ $ # Updating CI scripts $ cd ~/scripts; git stash; git pull; git stash pop }}} == Installation of other projects and KDE frameworks == The various frameworks have to be handled by replacing "PROJECT" with the framework's project name: {{{ $ python2.7 tools/prepare-environment.py --project PROJECT --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/PROJECT ... $ python2.7 tools/perform-build.py --project PROJECT --branchGroup kf5-qt5 --platform darwin-mavericks --sources ~/WC/KDECI-build/PROJECT ... }}} or alternatively by using [https://projects.kde.org/projects/playground/sdk/macports-kde/repository/revisions/master/show/contrib/scripts/KDECI scripts from our MacPorts/KDE git repository] {{{ $ ./prepare.sh PROJECT ... $ ./build.sh PROJECT ... }}} where {{{prepare.sh}}} takes care of creating the build directory if it is not yet existing. This is only needed when building a project manually, otherwise done by the jenkins slave. Alternatively one can combine both of these steps using: {{{ $ ./install.sh PROJECT }}} All frameworks of a specific tier can be build as a whole using {{{ $ # This builds all frameworks of tier 1 $ ./tier-install.sh 1 ... $ # This builds all frameworks of tier 2, but forces rebuilding every project regardless of $ # whether git pulled in changes or not (useful when e.g. qt5 or cmake was updated) $ ./tier-install.sh 2 rebuild ... }}} In order to be able to build KF5 frameworks of tier 3 it is - '''as a temporary workaround''' - necessary to copy kdoctools' files from its install directory to where framework builds can find them (i.e. below "/Library/Application Support"): {{{ $ sudo rm -rf /Library/Application\ Support/kf5/kdoctools/ ; sudo cp -Rp /opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdoctools/inst/Library/Application\ Support/kf5 /Library/Application\ Support }}} '''{It seems like that this can be avoided by patching Qt5's QStandardPaths in an appropriate manner. Details concerning this will be updated later.}''' KF5 builds have been successfully done so far for: {{{ Additional dependencies: - extra-cmake-modules - phonon Tier1 frameworks (COMPLETE): - attica - kapidox - karchive - kcodecs - kconfig - kcoreaddons - kdbusaddons - kdnssd - { kglobalaccel } - kguiaddons - ki18n - kidletime - kimageformats - kitemmodels - kitemviews - kplotting - kwidgetsaddons - { kwindowsystem } - solid - sonnet - threadweaver Tier2 frameworks (COMPLETE): - kauth - kcompletion - kcrash - kdoctools - kjobwidgets - kpty - kunitconversion Tier3 frameworks incl. phonon (ALMOST COMPLETE): [ATTENTION: (1) above workaround needed, (2) the build order matters here] - kconfigwidgets - kservice - kiconthemes - ktextwidgets - kxmlgui - kcmutils - kbookmarks - kdesu - kemoticons - knotifications - kwallet - kio - kactivities - kdeclarative - kparts - kdewebkit - kinit - kmediaplayer - knewstuff - knotifyconfig - ktexteditor - kdesignerplugin - kded Tier 4: - { frameworkintegration } Porting aids (ALMOST COMPLETE): - kjs - kdelibs4support - kjsembed - khtml - kross }}} Still to be done (but actually not needed/useful on OSX): {{{ Tier 3: - { plasma-framework } Porting aids: - krunner (needs plasma-framework) Other projects: - kf5umbrella (needs plasma-framework) - okteta (needs kf5umbrella) }}} Curly brackets in the above lists mark frameworks not needed on MacOSX, since plasma won't be running for now. Having the very compact [http://wiki.gentoo.org/wiki/Project:KDE/Frameworks#kdelibs Gentoo's Project:KDE/Frameworks wiki page] around turned out to be very helpful while building the above KDE frameworks manually. More info is found on [http://api.kde.org/frameworks-api/frameworks5-apidocs/ KDE's official API page] and the above lists have been aligned with it. Nice dependency graphs for every framework can be found there by clicking the "Dependencies" link on each frameworks page, some more (overview) graphs are [http://agateau.com/2013/12/05/kf5-diagrams/ here]. == Installs into "/Library/Application Support" == These folders should be copied into the MacPorts/KDE-CI system's "/Library/Application Support" directory (as it was done for as a temporary work-around for kdoctools above): {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/Library/Application Support/*" | egrep "/kf5$" ./frameworkintegration/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/frameworkintegration/inst/Library/Application Support/kf5 ./kauth/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kauth/inst/Library/Application Support/kf5 ./kconfigwidgets/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kconfigwidgets/inst/Library/Application Support/kf5 ./kdelibs4support/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdelibs4support/inst/Library/Application Support/kf5 ./kdesignerplugin/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdesignerplugin/inst/Library/Application Support/kf5 ./kdoctools/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdoctools/inst/Library/Application Support/kf5 ./khtml/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/khtml/inst/Library/Application Support/kf5 ./kio/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kio/inst/Library/Application Support/kf5 ./kjs/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjs/inst/Library/Application Support/kf5 ./knewstuff/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/knewstuff/inst/Library/Application Support/kf5 ./kwidgetsaddons/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kwidgetsaddons/inst/Library/Application Support/kf5 ./kxmlgui/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kxmlgui/inst/Library/Application Support/kf5 ./sonnet/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/sonnet/inst/Library/Application Support/kf5 }}} == Problematic installs into "/Library/Application Support" == The following folders do not reside below a kf5 sub-directory of "/Library/Application Support" which seems to indicate that there is a problem in the corresponding CMake files of {{{khtml}}} and {{{ktexteditor}}} frameworks: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/Library/Application Support/*" | grep -v "/kf5/" | egrep -v "/kf5$" ./khtml/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/khtml/inst/Library/Application Support/khtml ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5 ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/commands ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files/quickcoding ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/files/quickcoding/cpp ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/indentation ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/libraries ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/script/libraries/emmet ./ktexteditor/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/ktexteditor/inst/Library/Application Support/katepart5/syntax }}} == Installs into "/Applications/KF5" == These are all application packages which might have to be copied into {{{/Applications/KF5}}} on the KDE/CI system: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/frameworks/*/Applications/KF5/*.app" ./kactivities/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kactivities/inst/Applications/KF5/kactivitymanagerd.app ./kded/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kded/inst/Applications/KF5/kded5.app ./kdelibs4support/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kdelibs4support/inst/Applications/KF5/kdebugdialog5.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/Applications/KF5/kdeinit5.app ./kjsembed/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjsembed/inst/Applications/KF5/kjscmd5.app ./kross/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kross/inst/Applications/KF5/kf5kross.app ./kwallet/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kwallet/inst/Applications/KF5/kwalletd5.app }}} == Installs NOT into "/Applications/KF5" == These are all application packages which MIGHT NOT have to be copied into {{{/Applications/KF5}}} on the KDE/CI system: {{{ $ cd ~/WC/KDECI-builds; find . -type d -wholename "./*/local-inst/opt/kde/*/frameworks/*.app" | grep -v "Applications/KF5" ./kauth/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kauth/inst/lib/libexec/kauth/kauth-policy-gen.app ./kconfig/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kconfig/inst/lib/libexec/kf5/kconfig_compiler_kf5.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/lib/libexec/kf5/start_kdeinit.app ./kinit/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kinit/inst/lib/libexec/kf5/start_kdeinit_wrapper.app ./kio/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kio/inst/lib/libexec/kf5/kio_http_cache_cleaner.app ./kjsembed/local-inst/opt/kde/install/darwin/mavericks/clang/kf5-qt5/frameworks/kjsembed/inst/bin/kjsconsole.app }}}