Installing many ports causes "too many open files" error

[ryandesign (Ryan Carsten Schmidt)]

On a fresh MacPorts 1.9.0 beta 1 installation, installed from the disk image, on Mac OS X 10.6.3 with Xcode 3.2.2, with no ports installed, I tried to install wine-devel +universal. This has lots of dependencies, 47 of which installed fine before xorg-libXcursor failed during installation saying

--->  Installing xorg-libXcursor @1.1.10_0+universal
Error: Target org.macports.install returned: couldn't read file "/System/Library/Frameworks/Tcl.framework/Versions/8.5/Resources/Scripts/clock.tcl": too many open files
Log for xorg-libXcursor is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_x11_xorg-libXcursor/main.log
Error: The following dependencies failed to build: xorg-libXcursor xorg-libXinerama xorg-xineramaproto xorg-libXrandr xorg-randrproto xorg-libXxf86vm xorg-xf86vidmodeproto
Error: Status 1 encountered during processing.

Simply repeating the port install command allowed the installation to proceed, which says to me that MacPorts is holding some files open even after it's done with a port.

[ryandesign (Ryan Carsten Schmidt)]

terminal log

[jmroot (Joshua Root)]

Need to see which files are open when it fails.

[ryandesign (Ryan Carsten Schmidt)]

Ok, how do we get that information?

[jmroot (Joshua Root)]

Replying to ryandesign@…:

Ok, how do we get that information?

With lsof or similar.

[jmroot (Joshua Root)]

Like this for example:

sudo lsof -c tclsh

My testing shows that we seem to be accumulating sqlite temp files.

[jmroot (Joshua Root)]

Hackish fix committed in r67838, which deletes the interpreter for each dependency's mport right after running the install target (instead of waiting until mportclose).

[jmroot (Joshua Root)]

The correct fix, BTW, would be either to use sqlite's temp_store pragma to make it store temporary data in memory instead of files (I couldn't get this to work), or refactor things so that a single database object is shared by the main interpreter and the portfile subinterpreters.

[jmroot (Joshua Root)]

In 5def6d355b055c32deda01ee443885b4dcd00931/macports-base (master):

Only access the registry from the main interpreter

Fixes: #24857

[jmroot (Joshua Root)]

In c48853e678871e580dca89232e52a6d965a31812/mpbb (master):

dependencies.tcl: activate using already open mport

Doing this instead of using registry::run_target saves running another
mportopen for each dependency, so should perform better. This wasn't
originally possible due to the workaround for
#24857.