Context Navigation

Changes between Version 334 and Version 335 of SummerOfCode

Timestamp:: Oct 15, 2019, 9:03:27 AM (5 years ago)
Author:: umeshksingla (Umesh Singla)
Comment:: update ideas for gsoc 2020

Legend:

: Unmodified
: Added
: Removed
: Modified

SummerOfCode

-                      v334
+                      v335
  - Subscribe to the [MailingLists mailing list] [https://lists.macports.org/mailman/listinfo/macports-dev macports-dev] if you do not already read it. Don't be too shy to post.
 '''[#contact Get in contact!]''' Most important is to discuss your ideas with potential mentors via private email, the MacPorts development list, or the IRC channel before applying.
+'''[#contact Get in contact!]''' Most important is to discuss your ideas with potential mentors via the MacPorts development list, or the IRC channel before applying.
 == Students ==
 …
   macports_base::
     You would be expected to extend and improve functionality of the core of the package manager, mainly written Tcl (and C). It's OK if you have no prior experience with Tcl though. See below for various ideas.
+    You would be expected to extend and improve the functionality of the core of the package manager, mainly written in Tcl (and C). It's OK if you have no prior experience with Tcl though. See below for various ideas.
   ports::
     With tens of thousands of available ports probably overlapping with your daily needs of software. Portfiles are written in simple declarative Tcl, but you would probably spend more time figuring out how to (better) package the software which might be written in almost any programming language. A knowledge and passion of any language like C++, Perl, Ruby, Python, Rust, Go, Haskell, JavaScript, ... or technology like Qt, KDE, ... could greatly improve support of certain software in our ecosystem.
+    With tens of thousands of available ports probably overlapping with your daily needs of the software. Portfiles are written in simple declarative Tcl, but you would probably spend more time figuring out how to (better) package the software which might be written in almost any programming language. A knowledge and passion of any language like C++, Perl, Ruby, Python, Rust, Go, Haskell, JavaScript, ... or technology like Qt, KDE, ... could greatly improve the support of certain software in our ecosystem.
   infrastructure::
     Several independent projects to support development of MacPorts could be undertaken in any language of your choice (gravitating towards python & web), like writing a Django app in Python, improving frontend or backend of Buildbot continuous integration (JavaScript or Python), implementing a way to fire up virtual machines with macOS for continuous integration builds, ...
 …
 This does not have to be an entire GSOC project, but it could be used as part of the application process and/or a supplement to the main project.
 Most version increases in a port require only minor changes.
-A small number of updates, such as to the [https://trac.macports.org/ticket/48899 Glasgow Haskell Compiler], would be much more of a challenge.
 * Difficulty: Easy to Hard
+* Languages: Tcl, language of the port
+* Potential mentors: mcalhoun
+* Languages: Tcl, the language of the port
+* Potential mentors: mcalhoun
+* Importance: Low
 ==== Managing and Fixing Qt Versions ==== #qt
 Fix issues in [query:status=assigned|new|reopened&port~=qt3|qt4|qt5&group=type&order=priority open tickets for Qt 3, Qt 4, and Qt 5], in particular allowing for concurrent installation of the various Qt versions. There are around 50 outstanding tickets for the various Qt versions. Some involve patching and testing on multiple OSs. Some are probably invalid but need to be tested to determine validity. The end goal for this project is to allowing for concurrent installation of the various Qt versions, and then verifying and resolving as many issues as possible that weren't resolved by the concurrent changes. Requires knowledge of Portfile programming (Tcl), Qt programming (primarily C++), and Qt makefile programming (QMake); other programming knowledge that would be useful include shell (e.g., bash) and C.
+Fix issues in [query:status=assigned|new|reopened&port~=qt3|qt4|qt5 open tickets for Qt 3, Qt 4, and Qt 5], in particular allowing for concurrent installation of the various Qt versions. There are around 50 outstanding tickets for the various Qt versions. Some involve patching and testing on multiple OSs. Some are probably invalid but need to be tested to determine validity. The end goal for this project is to allow for concurrent installation of the various Qt versions, and then verifying and resolving as many issues as possible that weren't resolved by the concurrent changes. Requires knowledge of Portfile programming (Tcl), Qt programming (primarily C++), and Qt makefile programming (QMake); other programming knowledge that would be useful include shell (e.g., bash) and C.
 * Difficulty: Medium to Hard
 * Languages: Tcl, C++, QMake
 * Potential mentors: mcalhoun, michaelld
+* Importance: Low
 ==== Make Blacklisting MacPorts Compilers Easier ==== #blacklisting
 Not all compilers work with all ports.
-For example, a port may require [https://en.wikipedia.org/wiki/C%2B%2B11 C++11], which is not supported by GCC 4.2, which is the default compilers on older macOS systems.
 MacPorts has a mechanism to blacklist compilers that do not meet the port's requirements.
 Currently, blacklisting a range of compilers is very easy (e.g. `compiler.blacklist {clang < 500}` ensures that the Clang compiler supports C++11).
 …
 * Languages: Tcl
 * Potential mentors: mcalhoun
+* Importance: Low
 ==== Add Support for x86_64h Architecture ==== #x86_64h
 …
 * Languages: Tcl
 * Potential mentors: mcalhoun
+* Importance: Low
 ==== Allow for Multiple Runs of Each Phase ==== #multipleruns
 …
 * Languages: Tcl
 * Potential mentors: mcalhoun
+* Importance: Low
 ==== Allow a Variant to More Elegantly Become “undefaulted” ==== #undefaulted
 …
 * Languages: Tcl
 * Potential mentors: mcalhoun
+* Importance: High
 ==== Prevent `port reclaim` from Removing Build Dependencies ==== #reclaim
 …
 * Languages: Tcl
 * Potential mentors: mcalhoun
+* Importance: Medium
 ==== Give Portfile Better Access to CFLAGS, CXXFLAGS, etc. ==== #accessflags
 …
 * Languages: Tcl
 * Potential mentors: mcalhoun
+* Importance: Low
 ==== Simplify Compilers and MPI PortGroups with Recent Base Changes ==== #accessflags
 …
 * Languages: Tcl
 * Potential mentors: mcalhoun
+==== Phase out dependency on Xcode ==== #xcode
+MacPorts currently requires a full Xcode installation, even though a lot of ports will install just fine with the Command Line Tools package only. Since we also have a number of ports that need Xcode to build, we cannot completely remove the Xcode dependency. Your task would be to provide a way for maintainers to easily identify ports that depend on Xcode and mark them as such, so MacPorts can warn users without Xcode installed that a port they want to install needs the full Xcode package.
+To achieve this, you can modify "trace mode", a `DYLD_INSERT_LIBRARIES`-based sandbox to track whether a port has accessed files belonging to the Xcode package. If it does, your modifications should cause a warning to be printed suggesting the port maintainers to add `use_xcode yes` to the Portfile (unless of course, it is already there). You should also implement an error message if a user without Xcode installed tries to install a port that has `use_xcode yes` set.
+* Difficulty: Medium
+* Languages: Tcl, C
+* Potential mentors: mcalhoun, cal
+* Importance: Low
 ==== Announcements distributed over the ports tree ==== #news
 Some announcements about configuration changes could be presented to users when they update their ports tree. That involves a mechanism to add news items to the ports tree and a new `port news` command that allows the user to read them (and also mark them as read). Each news items could define conditions when they should be shown to the user, for example depending on OS version or if a specific port is installed. It will need a new API in macports1.0 to check for unread items, so the port client can poll it, for example when using a `port news` command. Additionally, these news items should also be available on the web.
+Some announcements about configuration changes could be presented to users when they update their ports tree. That involves a mechanism to add news items to the ports tree and a new `port news` command that allows the user to read them (and also mark them as read). Each news item could define conditions when they should be shown to the user, for example depending on the OS version or if a specific port is installed. It will need a new API in macports1.0 to check for unread items, so the port client can poll it, for example when using a `port news` command. Additionally, these news items should also be available on the web.
 As an example, there is a similar feature in [https://www.gentoo.org/support/news-items/ Gentoo Portage]. You can refer to these emails as well - [https://lists.macports.org/pipermail/macports-dev/2018-March/037801.html March 21] and [https://lists.macports.org/pipermail/macports-dev/2018-March/037865.html March 24, 2018].
 …
 * Difficulty: Easy
 * Languages: Tcl, HTML
+* Potential mentors: g5pw, raimue
+==== Collect build statistics ==== #build-stats
+Go through the [https://build.macports.org history of builds] and collect per-port statistics & success matrix:
+* whether the current version of port built on each particular OS/arch
+* when was the last time the port built on that OS/arch
+* links to all builds
+* list of installed files, differences in installed files on different OS versions
+* perhaps include some basic functionality to allow checking for build reproducibility
+* what is the latest version of port (in case it's already outdated), ...
+It would be nice to get a decent website for each port to get a quick overview of the port status & health across different OSes. (Part of statistics collection could also be implemented as an extension to Buildbot or Buildbot setup.)
+The project has had a previous attempt by a previous student, the details of which can be found here: https://github.com/macports/macports-webapp. You'll find a lot of documentation in docs/ directory to serve as an initial idea but it's open to suggestions for any or complete change.
+* Difficulty: Easy to medium
+* Languages: JSON, HTML, Python or any scripting language
+* Potential mentors: mojca, umeshksingla
+* Potential mentors: ?
+* Importance: High
 ==== Improve startupitem code ==== #startupitem
 MacPorts has the ability to automatically generate startup items for the current platform. For OS X, these are plist files for launchd which will be installed as `/Library/LaunchDaemons/org.macports.*.plist`. The current code would need a little care and could make use of options which have been added in recent releases of launchd.
+MacPorts has the ability to automatically generate startup items for the current platform. For OS X, these are plist files for launchd which will be installed as `/Library/LaunchDaemons/org.macports.*.plist`. The current code would need a little care and could make use of options that have been added in recent releases of launchd.
 Features that could be useful include (but are not limited to):
 …
 * Languages: Tcl, C
 * Potential mentors: larryv, pixilla
+* Importance: Low
 ==== Implement fakeroot functionality ==== #fakeroot
 …
 * Difficulty: Medium
 * Languages: Tcl, C
+* Potential mentors: jeremyhu
+==== Generating Portfiles ====
+There are multiple tasks related to the generation of Portfiles (see below). Some of these may not be enough work for a full summer project, so they could be combined while writing proposals freely when the applying student wants to.
+===== i. Perl modules integration from CPAN ===== #cpan2port
+There has been [[browser:contrib/cpan2port|an attempt]] to write a script for automatic generation of Portfiles from CPAN. This would simplify the maintenance of Perl modules in MacPorts. Revive this project and finish the script or rewrite it.
+Resources:
+* http://www.gentoo.org/proj/en/perl/g-cpan.xml
+* http://search.cpan.org/~bingos/CPANPLUS-0.9001/bin/cpan2dist
+* http://packages.debian.org/stable/dh-make-perl
+* Difficulty: Easy to medium
+* Languages: Perl, probably Tcl
+* Potential mentors: pixilla
+===== ii. Read packages from various package managers ===== #foo2port
+As with the cpan2port proposal above, and with the previous [wiki:pypi2port pypi2port] GSoC entry, except with other various package managers, such as [http://opam.ocamlpro.com/ opam] for ocaml packages, [http://www.haskell.org/cabal/ cabal] for Haskell, [http://luarocks.org/ luarocks] for Lua, [https://npmjs.org/ npm] for node.js, and so on.
+* Classification: Medium
+* Languages: Tcl, C, OCaml, Haskell, Lua, Node.js, etc.
+* Potential mentors: pixilla
+* Potential mentors: ?
+* Importance: Medium
 ==== Speed up trace mode ==== #tracemode
 …
 * Programming languages: Tcl, C
 * Potential mentors: cal
+* Importance: Medium
 ==== Auto-detection of build dependencies ==== #dependencies-gen
 …
 When creating a new portfile one of the problems is always the specification of the complete (and preferably minimal) list of build dependencies, especially when one starts with a complete install where most dependencies are already available.
 It is possible to invert the trace mode logic so that it detects all files a configure and/or build process accesses, in ${prefix} but outside of the port's build directory. This information can then be used to generate a dependency tree and information from the registry can then be used to simplify that tree so that it only lists direct dependencies. Can be combined with the above project. Consult mentor.
 * Difficulty: Medium to Easy
+It is possible to invert the trace mode logic so that it detects all files a configure and/or build process accesses, in ${prefix} but outside of the port's build directory. This information can then be used to generate a dependency tree and information from the registry can then be used to simplify that tree so that it only lists direct dependencies. It can be combined with the above project. Consult mentor.
+* Difficulty: Easy to Medium
 * Programming languages: Tcl, C
 * Potential mentors: cal
+* Importance: Low
 === Buildbot ideas ===
+While other projects mostly require access to macOS, frontend javascript and/or python developers are welcome to work in close collaboration with buildbot mentors on any OS.
+==== Implement better views in JavaScript ====
+Buildbot 1.0 lost some advantages of waterfall view from version 0.8, but it supports writing custom views in Javascript. Extend the buildbot core and/or write some views that would make Buildbot 1.0 a lot more useful for MacPorts.
+Buildbot is currently written in AngularJS 1.5, but could be extended in other more modern frameworks like vue or react.
+More info about how the output could look like: #55978
+* Difficulty: Medium
+* Programming languages: JavaScript, Python, json, HTML, AngularJS, Vue.js, React
+* Potential mentors: from Buildbot community
+NA
 === More Ideas/Hints for your own ideas ===
 …
 * Difficulty: Easy
 * Potential mentors: raimue
+==== Bump version and checksum of existing port ==== #bump
+* Importance: Low
+==== Improve `port bump` ==== #bump
 Homebrew has brew bump-formula-pr. Such functionality saves a lot of work when updating many ports. Bumping ports using PortGroup like GitHub and bitbucket should also be supported. Existing revision should be reset or removed interactively. A suggested commit message for the update should be shown.
 …
 * Languages: Tcl, C
 * Potential mentors: TBD
+==== MacPorts port for self-management ==== #self-management
+The MacPorts port should be the source for updating a user’s MacPorts installation.
+Currently, the MacPorts port is used to build the .pkg installer for MacPorts that is used for the initial installation of MacPorts, and port uses the “selfupdate” mechanism for maintaining the MacPorts installation. The selfupdate mechanism is (at least not documented as such) not accessible through the MacPorts API and does not use the MacPorts mechanisms for maintaining ports.
+* Importance: Medium
+==== Parallel execution ==== #parallel
+When an action will run targets on multiple ports, run them in parallel when possible and sensible (requires tracking dependencies between both targets and ports and figuring out the maximum reasonable parallelism, e.g. several ports can fetch at once on a fast connection but you only want one 'make -j8' at a time).
 * Difficulty: Challenging
 * Languages: Tcl, C
+* Potential mentors: ?
+* Importance: Medium
+==== Migrate muniversal into base (lipo merging) ==== #muniversal
+Integrate the [[source:trunk/dports/_resources/port1.0/group/muniversal-1.0.tcl|muniversal portgroup]] into the base. Not just a direct copy-and-paste, but in a way that makes sense and preserves the way portfiles are expected to behave (which the current portgroup doesn't).
+* Difficulty: Medium
+* Languages: Tcl, C
 * Potential mentors: TBD
+==== Parallel execution ==== #parallel
+When an action will run targets on multiple ports, run them in parallel when possible and sensible (requires tracking dependencies between both targets and ports and figuring out the maximum reasonable parallelism, e.g. several ports can fetch at once on a fast connection but you only want one 'make -j8' at a time).
+* Difficulty: Challenging
+* Languages: Tcl, C
+* Potential mentors: TBD
+==== Migrate muniversal into base (lipo merging) ==== #muniversal
+Integrate the [[source:trunk/dports/_resources/port1.0/group/muniversal-1.0.tcl|muniversal portgroup]] into base. Not just a direct copy-and-paste, but in a way that makes sense and preserves the way portfiles are expected to behave (which the current portgroup doesn't).
+* Difficulty: Medium
+* Languages: Tcl, C
+* Potential mentors: TBD
+==== App portgoup ==== #app
+Enhance the launching of GUI apps packaged by MacPorts:
+ - Fix app icon bouncing on Dock after app launched (#40110)
+ - Support multiple apps per port (#41681)
+* Difficulty: Easy
+* Languages: TCL, XML
+* Potential mentors: TBD
+==== Portfiles ==== #portfiles
+Sweep through all Portfiles and look for useful opportunities to add more built-in Tcl functions that make Portfiles more (usefully) terse, powerful, flexible or easier to write. I'm sure there is an entirely family of helper functions yet to be written here. This might also include porting additional packages to MacPorts and cleaning up or removing obsolete ports.
+* Classification: Medium
+* Language: Tcl
+* Potential mentors: larryv
+* Importance: Medium
 ==== Documentation and website ==== #docs
 …
 * Potential mentors: larryv
+==== MacPorts statistics ==== #mpstats
+Enhance collection and reporting of inventory of ports installed by participating users: StatisticsIdeas
+* Difficulty: Medium
+* Language: TBD
+* Potential mentors: TBD
+{{{
+#!comment
+==== Improve fetching from version control ==== #fetchtypes
+Make cvs/svn/git/hg/bzr fetch types checkout into the distfiles dir and then export into the work dir, to [[ticket:16373|avoid having to re-fetch]] after cleaning the work directory.
+"`fetch.type svn`" is inefficient in that it checks out a new working copy every time, directly to the work area. That would be like a normal port downloading the distfile every time. Instead, we should check out a working copy to that port's distpath, and then in the extract phase we should `svn export` it to the work area.
+Some checks will be needed in the fetch phase to ensure that an existing working copy:
+* has no modifications: check `svn status`. Ideally, we would try to clean up the working copy, for example by `svn revert`-ing modified or added or deleted files, and then in a second `svn status` run, delete any unversioned files. But it's already an improvement if we just discard the working copy if `svn status --ignore-externals` produces any output.
+* is from the right URL: check `svn info`: check if the "URL" is the one we want. If not, check that the "Repository Root" is a substring of the repository we want. If yes, try to `svn switch` to the URL and revision we want; if not, discard the working copy.
+* Difficulty: Easy
+* Languages: Tcl, C, bash
+* Contact: larryv
+}}}
+{{{
+#!comment
+# This was just a wild idea by me. After reading it again, I am no longer sure if this is suitable for an idea. The compiler binary checks would be covered by a functioning trace mode already and environment variable checks are hard to implement (if possible at all).
+# However, I already typed it out now, so I leave it here for discussion with other mentors. --raimue@
+==== Run basic checks on build systems ==== #buildcheck
+Some mistakes are very common on newly written ports, mostly because build systems do not always respect the usual conventions. While the port works for the initial port author, it may fail for others due to these mistakes. These could be checked for automatically to catch them before adding the port to the ports tree.
+First, a set of tests could be run on the extracted, patched (and configured) sources. For example, a possible check could include whether the given Makefile respects the CC/CPPFLAGS/CFLAGS/LDFLAGS environment variables, which is one of the most common mistakes. Often smaller projects just [UsingTheRightCompiler hardcode the compiler] to `cc` or even `gcc`. This could be checked for in various ways. One option would be to overwrite Makefiles rules to verify the passed parameters. Another option would be to use a custom compiler script as `CC` that checks the flags in question are always passed to the compiler by the build system and match those given in the Portfile (or the defaults). Other binaries such as `cc`/`gcc` need to be shadowed and invocation must raise an error.
+These checks would be included as a new option in existing commands, for example `port build --check`, or a new phase `prebuildcheck` to be run before the `build` phase. This mode could be enabled automatically with a flag in `macports.conf` for MacPorts developers.
+You will definitely need to come up with more ideas to fill the whole summer.
+* Classification: Easy to Hard
+* Languages: Tcl
+* Potential mentors: (raimue)
+}}}
+{{{
+#!comment
+# Listing already done tasks here, maybe parts of this could spin-off a new project idea
+# Done 2009
+==== Logging ==== #logging
+Currently, MacPorts has no notion of logging of build activities of a given port or sets of ports. When a build is attempted but an error keeps it from completing, there's no way to track the problem other than the build progress that was output to the terminal, if the verbose mode was requested in the first place. Otherwise, the build environment has to be pruned and the build attempted once again to even get a look at the precise error message. This is particularly problematic when automated builds are attempted since there's usually no one around to have a look at the failure spew. An infrastructure to remedy this situation and endow MacPorts with a rich set of logging capabilities has to be developed to open up the door to truly automated build runs of large sets of ports and thus to packaging of binaries, since with logging we'd have a fully reliable way of catching, reporting and processing of all sorts of fetch/configure/build/destroot/install/etc errors.
+This could be extended with the interaction with a server-side application like MPWA that could consume these logs (read MPWA proposal). A more detailed draft of this task can be found on the LoggingProposal page.
+Classification: medium task to relatively challenging[[BR]]
+Programming languages: Tcl and C[[BR]]
+Potential mentor: blb
+}}}
+=== New Ideas ===
+. Add sanity tests
+. Improve UX for macports command line and colors
+. Improve ports web application - medium
+. pypi2port upt
+. buildbot user interface improvements - medium
+. coloring - easy
 == Contacting us == #contact
 …
 == Mentors ==
 The following committers have agreed to be mentors for GSoC 2019 (append `@macports.org` for email if it is missing)
+The following committers have agreed to be mentors for GSoC 2019 (append `@macports.org` for the email if it is missing)
 ||= Name =||= Email =||= Area =||