103 | | === Ports === |
104 | | |
105 | | ==== Qt 5 ==== #qt |
106 | | |
107 | | Fix issues in [query:status=assigned|new|reopened&port~=qt3|qt4|qt5 open tickets for Qt 3, Qt 4, and Qt 5], in particular allowing for concurrent installation of the various Qt versions (Note: co-installable ports for Qt 4.8.6, 5.3.2 and 5.4.1 have already been submitted to Trac). |
108 | | |
109 | | * Difficulty: Medium |
110 | | * Language: Tcl, C++ |
111 | | * Potential mentors: michaelld, pixilla |
112 | | |
113 | | === Core tasks === |
114 | | |
115 | | |
116 | | ==== Phase out dependency on Xcode ==== #xcode |
117 | | |
118 | | MacPorts currently requires a full Xcode installation, even though a lot of ports will install just fine with the Command Line Tools package only. Since we also have a number of ports that need Xcode to build, we cannot completely remove the Xcode dependency. Your task would be to provide a way for maintainers to easily identify ports that depend on Xcode and mark them as such, so MacPorts can warn users without Xcode installed that a port they want to install needs the full Xcode package. |
119 | | |
120 | | To achieve this, you can modify "trace mode", a `DYLD_INSERT_LIBRARIES`-based sandbox to track whether a port has accessed files belonging to the Xcode package. If it does, your modifications should cause a warning to be printed suggesting the port maintainers to add `use_xcode yes` to the Portfile (unless of course, it is already there). You should also implement an error message if a user without Xcode installed tries to install a port that has `use_xcode yes` set. |
121 | | |
122 | | * Difficulty: Medium |
123 | | * Languages: Tcl, C |
124 | | * Potential mentors: cal |
125 | | |
126 | | ==== MacPorts port for self-management ==== #self-management |
127 | | |
128 | | The MacPorts port should be the source for updating a user’s MacPorts installation. |
129 | | |
130 | | Currently the MacPorts port is used to build the .pkg installer for MacPorts that is used for the initial installation of MacPorts, and port uses the “selfupdate” mechanism for maintaining the MacPorts installation. The selfupdate mechanism is (at least not documented as such) not accessible through the MacPorts API and does not use the MacPorts mechanisms for maintaining ports. |
131 | | |
132 | | * Difficulty: Challenging |
133 | | * Languages: Tcl, C |
134 | | * Potential mentors: TBD |
135 | | |
136 | | ==== Implement fakeroot functionality for destroot phase ==== #fakeroot |
137 | | |
138 | | Currently MacPorts uses root privileges in the destroot phase. That should be replaced by a system that runs as the macports user, but intercepts all operations that would require root privileges (chown/chmod/etc.) and record the resulting permissions in a database. |
139 | | |
140 | | The existing functionality of trace mode in darwintracelib1.0 could be leveraged for this task. |
141 | | |
142 | | * Difficulty: Medium |
143 | | * Languages: Tcl, C |
144 | | * Potential mentors: jeremyhu |
145 | | |
146 | | ==== Generating Portfiles ==== |
147 | | |
148 | | There are multiple tasks related to the generation of Portfiles. Some of these may not be enough work for a full summer project, so they could be combined for proposals freely when the applying student wants to. |
149 | | |
150 | | ===== Perl modules integration from CPAN ===== #cpan2port |
151 | | |
152 | | There has been [[browser:contrib/cpan2port|an attempt]] to write a script for automatic generation of Portfiles from CPAN. This would simplify the maintenance of Perl modules in MacPorts. Revive this project and finish the script or rewrite it. |
153 | | |
154 | | Resources: |
155 | | * http://www.gentoo.org/proj/en/perl/g-cpan.xml |
156 | | * http://search.cpan.org/~bingos/CPANPLUS-0.9001/bin/cpan2dist |
157 | | * http://packages.debian.org/stable/dh-make-perl |
158 | | |
159 | | * Difficulty: Easy to medium |
160 | | * Languages: Perl, probably Tcl |
161 | | * Potential mentors: pixilla |
162 | | |
163 | | ===== Read packages from other various package managers ===== #foo2port |
164 | | |
165 | | As with the cpan2port proposal above, and with the previous [wiki:pypi2port pypi2port] GSoC entry, except with other package managers, such as [http://opam.ocamlpro.com/ opam] for ocaml packages, [http://www.haskell.org/cabal/ cabal] for haskell, [http://luarocks.org/ luarocks] for lua, [https://npmjs.org/ npm] for node.js, and so on. |
166 | | |
167 | | * Classification: Medium |
168 | | * Languages: Tcl, C, OCaml, Haskell, Lua, Node.js, etc. |
169 | | * Potential mentors: pixilla |
170 | | |
171 | | ==== Speed up trace mode ==== #tracemode |
172 | | |
173 | | Trace mode is a library preloading-based sandbox used to hide files that a port does not depend on or that are not part of a standard system's installation (such as `/usr/local`). This can avoid problems due to incompatible user-installed software and avoid "automagic" dependencies and increase the reproducibility of builds. |
174 | | |
175 | | Unfortunately, enabling trace mode adds a significant performance penalty to the build process. However, the trace mode code can certainly be optimized using appropriate cache data structures, such as a modified [http://en.wikipedia.org/wiki/Trie Trie]. Your task would be to identify the performance bottle necks, draft appropriate caching data structures and implement them. |
176 | | |
177 | | * Difficulty: Medium to Hard |
178 | | * Programming languages: Tcl, C |
179 | | * Potential mentors: cal |
180 | | |
181 | | |
182 | | ===== Auto-detection of build dependencies ===== #dependencies-gen |
183 | | |
184 | | When creating a new portfile one of the problems is always the specification of the complete (and preferably minimal) list of build dependencies, especially when one starts with a rather complete install where most dependencies are already available. |
185 | | |
186 | | It is possible to invert the trace mode logic so that it detects all files a configure and/or build process accesses, in ${prefix} but outside of the port's build directory. This information can then be used to generate a dependency tree and information from the registry can then be used to simplify that tree so that it only lists direct dependencies. |
187 | | |
188 | | * Difficulty: Medium to Easy |
189 | | * Programming languages: Tcl, C |
190 | | * Potential mentors: cal |
191 | | |
192 | | ==== Improve startupitem code ==== #startupitem |
193 | | |
194 | | MacPorts has the ability to automatically generate startup items for the current platform. For OS X, these are plist files for launchd which will be installed as `/Library/LaunchDaemons/org.macports.*.plist`. The current code would need a little care and could make use of options which have been added in recent releases of launchd. |
195 | | |
196 | | Features that could be useful include (but are not limited to): |
197 | | - Not using daemondo if the daemon works fine under launchd without it |
198 | | - Ability to install multiple plists |
199 | | - Support for LaunchAgents as well as LaunchDaemons |
200 | | - Installing plists in ~/Library for non-root installs if the user wants |
201 | | - only modify specific XML tags to avoid clobbering additions by user |
202 | | - Support startupitems in standalone binary packages (currently a brutal hack is used to include daemondo in such packages, see #43648) |
203 | | |
204 | | * Difficulty: Easy |
205 | | * Languages: Tcl, C |
206 | | * Potential mentors: larryv, pixilla |
207 | | |
208 | | ==== Parallel execution ==== #parallel |
209 | | |
210 | | When an action will run targets on multiple ports, run them in parallel when possible and sensible (requires tracking dependencies between both targets and ports and figuring out the maximum reasonable parallelism, e.g. several ports can fetch at once on a fast connection but you only want one 'make -j8' at a time). |
211 | | |
212 | | * Difficulty: Challenging |
213 | | * Languages: Tcl, C |
214 | | * Potential mentors: TBD |
215 | | |
216 | | ==== Migrate muniversal into base (lipo merging) ==== #muniversal |
217 | | |
218 | | Integrate the [[source:trunk/dports/_resources/port1.0/group/muniversal-1.0.tcl|muniversal portgroup]] into base. Not just a direct copy-and-paste, but in a way that makes sense and preserves the way portfiles are expected to behave (which the current portgroup doesn't). |
219 | | |
220 | | * Difficulty: Medium |
221 | | * Languages: Tcl, C |
222 | | * Potential mentors: TBD |
223 | | |
224 | | ==== Improve fetching from version control ==== #fetchtypes |
225 | | |
226 | | Make cvs/svn/git/hg/bzr fetch types checkout into the distfiles dir and then export into the work dir, to [[ticket:16373|avoid having to re-fetch]] |
227 | | after cleaning the work directory. |
228 | | |
229 | | This task alone is most probably not enough for the whole Summer Of Code. |
230 | | |
231 | | * Difficulty: Easy |
232 | | * Languages: Tcl, C, bash |
233 | | * Potential mentors: larryv |
234 | | |
235 | | ==== App portgoup ==== #app |
236 | | Enhance the launching of GUI apps packaged by MacPorts: |
237 | | |
238 | | - Fix app icon bouncing on Dock after app launched (#40110) |
239 | | - Support multiple apps per port (#41681) |
240 | | |
241 | | * Difficulty: Easy |
242 | | * Languages: TCL, XML |
243 | | * Potential mentors: TBD |
244 | | |
245 | | |
246 | | {{{ |
247 | | #!comment |
248 | | # This was just a wild idea by me. After reading it again, I am no longer sure if this is suitable as an idea. The compiler binary checks would be covered by a functioning trace mode already and environment variable checks are hard to implement (if possible at all). |
249 | | # However, I already typed it out now, so I leave it here for discussion with other mentors. --raimue@ |
250 | | |
251 | | ==== Run basic checks on build systems ==== #buildcheck |
252 | | |
253 | | Some mistakes are very common on newly written ports, mostly because build systems do not always respect the usual conventions. While the port works for the initial port author, it may fail for others due to these mistakes. These could be checked for automatically to catch them before adding the port to the ports tree. |
254 | | |
255 | | First, a set of tests could be run on the extracted, patched (and configured) sources. For example, a possible check could include whether the given Makefile respects the CC/CPPFLAGS/CFLAGS/LDFLAGS environment variables, which is one of the most common mistakes. Often smaller projects just [UsingTheRightCompiler hardcode the compiler] to `cc` or even `gcc`. This could be checked for in various ways. One option would be to overwrite Makefiles rules to verify the passed parameters. Another option would be to use a custom compiler script as `CC` that checks the flags in question are always passed to the compiler by the build system and match those given in the Portfile (or the defaults). Other binaries such as `cc`/`gcc` need to be shadowed and invocation must raise an error. |
256 | | |
257 | | These checks would be included as a new option in existing commands, for example `port build --check`, or a new phase `prebuildcheck` to be run before the `build` phase. This mode could be enabled automatically with a flag in `macports.conf` for MacPorts developers. |
258 | | |
259 | | You will definitely need to come up with more ideas to fill the whole summer. |
260 | | |
261 | | * Classification: Easy to Hard |
262 | | * Languages: Tcl |
263 | | * Potential mentors: (raimue) |
264 | | }}} |
265 | | |
266 | | |
267 | | === Secondary tasks === |
268 | | |
269 | | ==== Portfiles ==== #portfiles |
270 | | |
271 | | Sweep through all Portfiles and look for useful opportunities to add more built-in Tcl functions that make Portfiles more (usefully) terse, powerful, flexible or easier to write. I'm sure there is an entirely family of helper functions yet to be written here. This might also include porting additional packages to MacPorts and cleaning up or removing obsolete ports. |
272 | | |
273 | | * Classification: Medium |
274 | | * Language: Tcl |
275 | | * Potential mentors: larryv |
| 103 | === Ideas === |
322 | | * Potential mentors: TBD |
323 | | |
| 151 | * Contact: raimue, l2dy |
| 152 | |
| 153 | ==== Improve startupitem code ==== #startupitem |
| 154 | |
| 155 | MacPorts has the ability to automatically generate startup items for the current platform. For OS X, these are plist files for launchd which will be installed as `/Library/LaunchDaemons/org.macports.*.plist`. The current code would need a little care and could make use of options which have been added in recent releases of launchd. |
| 156 | |
| 157 | Features that could be useful include (but are not limited to): |
| 158 | - Not using daemondo if the daemon works fine under launchd without it |
| 159 | - Ability to install multiple plists |
| 160 | - Support for LaunchAgents as well as LaunchDaemons |
| 161 | - Installing plists in ~/Library for non-root installs if the user wants |
| 162 | - only modify specific XML tags to avoid clobbering additions by user |
| 163 | - Support startupitems in standalone binary packages (currently a brutal hack is used to include daemondo in such packages, see #43648) |
| 164 | |
| 165 | * Difficulty: Easy |
| 166 | * Languages: Tcl, C |
| 167 | * Potential mentors: larryv, pixilla |
| 168 | |
| 169 | ==== Speed up trace mode ==== #tracemode |
| 170 | |
| 171 | Trace mode is a library preloading-based sandbox used to hide files that a port does not depend on or that are not part of a standard system's installation (such as `/usr/local`). This can avoid problems due to incompatible user-installed software and avoid "automagic" dependencies and increase the reproducibility of builds. |
| 172 | |
| 173 | Unfortunately, enabling trace mode adds a significant performance penalty to the build process. However, the trace mode code can certainly be optimized using appropriate cache data structures, such as a modified [http://en.wikipedia.org/wiki/Trie Trie]. Your task would be to identify the performance bottlenecks, draft appropriate caching data structures and implement them. |
| 174 | |
| 175 | * Difficulty: Medium to Hard |
| 176 | * Programming languages: Tcl, C |
| 177 | * Potential mentors: cal |
| 178 | |
| 179 | ==== Auto-detection of build dependencies ==== #dependencies-gen |
| 180 | |
| 181 | When creating a new portfile one of the problems is always the specification of the complete (and preferably minimal) list of build dependencies, especially when one starts with a complete install where most dependencies are already available. |
| 182 | |
| 183 | It is possible to invert the trace mode logic so that it detects all files a configure and/or build process accesses, in ${prefix} but outside of the port's build directory. This information can then be used to generate a dependency tree and information from the registry can then be used to simplify that tree so that it only lists direct dependencies. Can be combined with the above project. Consult mentor. |
| 184 | |
| 185 | * Difficulty: Medium to Easy |
| 186 | * Programming languages: Tcl, C |
| 187 | * Potential mentors: cal |
| 188 | |
| 189 | ==== Improve fetching from version control ==== #fetchtypes |
| 190 | |
| 191 | Make cvs/svn/git/hg/bzr fetch types checkout into the distfiles dir and then export into the work dir, to [[ticket:16373|avoid having to re-fetch]] after cleaning the work directory. |
| 192 | "`fetch.type svn`" is inefficient in that it checks out a new working copy every time, directly to the work area. That would be like a normal port downloading the distfile every time. Instead, we should check out a working copy to that port's distpath, and then in the extract phase we should `svn export` it to the work area. |
| 193 | |
| 194 | Some checks will be needed in the fetch phase to ensure that an existing working copy: |
| 195 | |
| 196 | * has no modifications: check `svn status`. Ideally, we would try to clean up the working copy, for example by `svn revert`-ing modified or added or deleted files, and then in a second `svn status` run, delete any unversioned files. But it's already an improvement if we just discard the working copy if `svn status --ignore-externals` produces any output. |
| 197 | * is from the right URL: check `svn info`: check if the "URL" is the one we want. If not, check that the "Repository Root" is a substring of the repository we want. If yes, try to `svn switch` to the URL and revision we want; if not, discard the working copy. |
| 198 | |
| 199 | * Difficulty: Easy |
| 200 | * Languages: Tcl, C, bash |
| 201 | * Contact: larryv |
| 202 | |
| 203 | ==== Generating Portfiles ==== |
| 204 | |
| 205 | There are multiple tasks related to the generation of Portfiles. Some of these may not be enough work for a full summer project, so they could be combined while writing proposals freely when the applying student wants to. |
| 206 | |
| 207 | ===== Perl modules integration from CPAN ===== #cpan2port |
| 208 | |
| 209 | There has been [[browser:contrib/cpan2port|an attempt]] to write a script for automatic generation of Portfiles from CPAN. This would simplify the maintenance of Perl modules in MacPorts. Revive this project and finish the script or rewrite it. |
| 210 | |
| 211 | Resources: |
| 212 | * http://www.gentoo.org/proj/en/perl/g-cpan.xml |
| 213 | * http://search.cpan.org/~bingos/CPANPLUS-0.9001/bin/cpan2dist |
| 214 | * http://packages.debian.org/stable/dh-make-perl |
| 215 | |
| 216 | * Difficulty: Easy to medium |
| 217 | * Languages: Perl, probably Tcl |
| 218 | * Potential mentors: pixilla |
| 219 | |
| 220 | ===== Read packages from other various package managers ===== #foo2port |
| 221 | |
| 222 | As with the cpan2port proposal above, and with the previous [wiki:pypi2port pypi2port] GSoC entry, except with other package managers, such as [http://opam.ocamlpro.com/ opam] for ocaml packages, [http://www.haskell.org/cabal/ cabal] for Haskell, [http://luarocks.org/ luarocks] for Lua, [https://npmjs.org/ npm] for node.js, and so on. |
| 223 | |
| 224 | * Classification: Medium |
| 225 | * Languages: Tcl, C, OCaml, Haskell, Lua, Node.js, etc. |
| 226 | * Potential mentors: pixilla |
| 227 | |
| 228 | === More Ideas/Hints for your own ideas === |
| 229 | |
| 230 | ==== Shell environment ==== #shell-environment |
| 231 | |
| 232 | Add support for providing basic and port-provided environmental services to users in the `~/.profile`, `~/.cshrc`, and `~/.xinitrc` files, so that instead of manipulating the user's .profile to modify certain paths, the installer could append "`source /opt/local/etc/bash.rc`" to the end of a user's .profile file and that bash.rc would source all the files in `/opt/local/etc/bash.d`. |
| 233 | |
| 234 | * Difficulty: Easy |
| 235 | * Potential mentors: raimue |
| 236 | |
| 237 | ==== Implement fakeroot functionality for destroot phase ==== #fakeroot |
| 238 | |
| 239 | Currently, MacPorts uses root privileges in the destroot phase. That should be replaced by a system that runs as the macports user but intercepts all operations that would require root privileges (chown/chmod/etc.) and record the resulting permissions in a database. |
| 240 | |
| 241 | The existing functionality of trace mode in darwintracelib1.0 could be leveraged for this task. |
| 242 | |
| 243 | * Difficulty: Medium |
| 244 | * Languages: Tcl, C |
| 245 | * Potential mentors: jeremyhu |
| 246 | |
| 247 | ==== MacPorts port for self-management ==== #self-management |
| 248 | |
| 249 | The MacPorts port should be the source for updating a user’s MacPorts installation. |
| 250 | |
| 251 | Currently, the MacPorts port is used to build the .pkg installer for MacPorts that is used for the initial installation of MacPorts, and port uses the “selfupdate” mechanism for maintaining the MacPorts installation. The selfupdate mechanism is (at least not documented as such) not accessible through the MacPorts API and does not use the MacPorts mechanisms for maintaining ports. |
| 252 | |
| 253 | * Difficulty: Challenging |
| 254 | * Languages: Tcl, C |
| 255 | * Potential mentors: TBD |
| 256 | |
| 257 | ==== Parallel execution ==== #parallel |
| 258 | |
| 259 | When an action will run targets on multiple ports, run them in parallel when possible and sensible (requires tracking dependencies between both targets and ports and figuring out the maximum reasonable parallelism, e.g. several ports can fetch at once on a fast connection but you only want one 'make -j8' at a time). |
| 260 | |
| 261 | * Difficulty: Challenging |
| 262 | * Languages: Tcl, C |
| 263 | * Potential mentors: TBD |
| 264 | |
| 265 | ==== Migrate muniversal into base (lipo merging) ==== #muniversal |
| 266 | |
| 267 | Integrate the [[source:trunk/dports/_resources/port1.0/group/muniversal-1.0.tcl|muniversal portgroup]] into base. Not just a direct copy-and-paste, but in a way that makes sense and preserves the way portfiles are expected to behave (which the current portgroup doesn't). |
| 268 | |
| 269 | * Difficulty: Medium |
| 270 | * Languages: Tcl, C |
| 271 | * Potential mentors: TBD |
| 272 | |
| 273 | ==== App portgoup ==== #app |
| 274 | Enhance the launching of GUI apps packaged by MacPorts: |
| 275 | |
| 276 | - Fix app icon bouncing on Dock after app launched (#40110) |
| 277 | - Support multiple apps per port (#41681) |
| 278 | |
| 279 | * Difficulty: Easy |
| 280 | * Languages: TCL, XML |
| 281 | * Potential mentors: TBD |
| 282 | |
| 283 | ==== Portfiles ==== #portfiles |
| 284 | |
| 285 | Sweep through all Portfiles and look for useful opportunities to add more built-in Tcl functions that make Portfiles more (usefully) terse, powerful, flexible or easier to write. I'm sure there is an entirely family of helper functions yet to be written here. This might also include porting additional packages to MacPorts and cleaning up or removing obsolete ports. |
| 286 | |
| 287 | * Classification: Medium |
| 288 | * Language: Tcl |
| 289 | * Potential mentors: larryv |
| 290 | |
| 291 | ==== Documentation and website ==== #docs |
| 292 | |
| 293 | Improve MacPorts [query:status!=closed&component=guide|server/hosting|website|wiki documentation, website and Trac system]. Note that pure documentation proposals are not allowed by Google. |
| 294 | |
| 295 | * Difficulty: Easy to difficult |
| 296 | * Languages: PHP, Python |
| 297 | * Potential mentors: larryv |
| 298 | |
| 299 | ==== MacPorts statistics ==== #mpstats |
| 300 | |
| 301 | Enhance collection and reporting of inventory of ports installed by participating users: StatisticsIdeas |
| 302 | |
| 303 | * Difficulty: Medium |
| 304 | * Language: TBD |
| 305 | * Potential mentors: TBD |
| 306 | |
| 307 | {{{ |
| 308 | #!comment |
| 309 | # This was just a wild idea by me. After reading it again, I am no longer sure if this is suitable for an idea. The compiler binary checks would be covered by a functioning trace mode already and environment variable checks are hard to implement (if possible at all). |
| 310 | # However, I already typed it out now, so I leave it here for discussion with other mentors. --raimue@ |
| 311 | |
| 312 | ==== Run basic checks on build systems ==== #buildcheck |
| 313 | |
| 314 | Some mistakes are very common on newly written ports, mostly because build systems do not always respect the usual conventions. While the port works for the initial port author, it may fail for others due to these mistakes. These could be checked for automatically to catch them before adding the port to the ports tree. |
| 315 | |
| 316 | First, a set of tests could be run on the extracted, patched (and configured) sources. For example, a possible check could include whether the given Makefile respects the CC/CPPFLAGS/CFLAGS/LDFLAGS environment variables, which is one of the most common mistakes. Often smaller projects just [UsingTheRightCompiler hardcode the compiler] to `cc` or even `gcc`. This could be checked for in various ways. One option would be to overwrite Makefiles rules to verify the passed parameters. Another option would be to use a custom compiler script as `CC` that checks the flags in question are always passed to the compiler by the build system and match those given in the Portfile (or the defaults). Other binaries such as `cc`/`gcc` need to be shadowed and invocation must raise an error. |
| 317 | |
| 318 | These checks would be included as a new option in existing commands, for example `port build --check`, or a new phase `prebuildcheck` to be run before the `build` phase. This mode could be enabled automatically with a flag in `macports.conf` for MacPorts developers. |
| 319 | |
| 320 | You will definitely need to come up with more ideas to fill the whole summer. |
| 321 | |
| 322 | * Classification: Easy to Hard |
| 323 | * Languages: Tcl |
| 324 | * Potential mentors: (raimue) |
| 325 | }}} |