macOS 10.4.11: failed to build py27-psutil due to lack of libproc support

[dzhub]

Hi,

while I tried to install certbot, I got this error recently:

:info:build In file included from psutil/_psutil_osx.c:18:
:info:build /opt/local/include/libproc.h:35:27: error: sys/proc_info.h: No such file or directory
:info:build /opt/local/include/libproc.h:37:26: error: Availability.h: No such file or directory
:info:build In file included from psutil/_psutil_osx.c:18:
:info:build /opt/local/include/libproc.h: In function 'proc_listpidspath':
[...]

Where does sys/proc_info.h belong to? Is there a way for me to install it somehow?

Thank you for your support.

[mf2k (Frank Schima)]

Please attach the complete main.log file.

In the future, please add the port maintainer(s) to Cc (port info --maintainers py27-psutil), if any.

[jmroot (Joshua Root)]

Please attach the complete main.log. The py27-psutil port doesn't declare any dependencies other than python; which port provides /opt/local/include/libproc.h?

[dzhub]

main.log of py27-psutil

[dzhub]

I dont know, which port provides libproc.h, but it is referenced by

:info:build In file included from psutil/_psutil_osx.c:18:

I have attached the complete main.log now.

[stromnov (Andrey Stromnov)]

Please provide the output of

port provides /opt/local/include/libproc.h

[dzhub]

Here you go:

port provides /opt/local/include/libproc.h
/opt/local/include/libproc.h is not provided by a MacPorts port.

But it is definitly there:

locate libproc.h
/opt/local/include/libproc.h

[stromnov (Andrey Stromnov)]

The $prefix namespace (usually/opt/local) is not intended to be manually modified. Your MacPorts installation seems to have been contaminated by external intervention.

[dzhub]

I dont remember I ever touched this. How can I fix it?

[stromnov (Andrey Stromnov)]

mv /opt/local/include/libproc.h /opt/local/include/libproc.h_
port install py27-psutil
mv /opt/local/include/libproc.h_ /opt/local/include/libproc.h

[dzhub]

I tried that. It doesnt compile:

:info:build psutil/_psutil_osx.c:18:21: error: libproc.h: No such file or directory

[kencu (Ken)]

On 10.4, there is no libproc.h in /usr/include, nor is there a sys/proc_info.h

You pretty much must have added this header yourself somewhere along the line; over at TigerBrew Misty has come up with a modified header that contains the libproc routines that Tiger does support (which is not quite all of the ones we need, so I'm still working on filling in the missing functions or two in some future supplement to our legacysupport PortGroup).

I'm afraid this one is not a trivial fix -- there are two or three of us around here who might be able to find some time to see if this port can be fixed to work on Tiger (or better described as "see if Tiger can be fixed to support this port").

But -- it won't likely be today, and perhaps may not be fixable. But you are in the right place!

[kencu (Ken)]

I looked at this. It also needs the missing libproc function proc_pidpath that doen't appear to have support in Tiger's kernel.

There are userland examples of software that does this kind of thing on Tiger, namely lsof and ps. Both are available on Apple Open Source.

ps compiles easily. lsof looks better, but is not so easy to compile as some AOS headers have been modified and damaged so as not to build.

In other words, it's a project, similar to adding adding posix_spawn to Tiger.

[dzhub]

Did I get you right: py27-psutil will not come to Tiger any soon, right?

[kencu (Ken)]

Do you have any programming skills? The basic issue is that proc_pidpath was added in 10.5 Leopard and it requires kernel support to implement easily.

I have been looking at how to add that function to Tiger. The most common use for it is to find the calling processes' full path, and we can possibly work around that use case with an NS function that does something similar, if the calling process and the requested proc_pidpath are the same.

To implement the function in general is turning out to be much harder than you would think. That information (full path for a PID) is just not easily there on Tiger.

There are two good places to look -- both ps and lsof implement something similar without use proc_pidpath to do it. ps is approachable enough -- builds easily for me on Tiger with the existing sources on Apple's site. lsof does a better job, but is harder to build.

SO -- that's where we are. Can you help?

[dzhub]

Yes, I understand the problem, and the path ps and lsof have paved so far. But my programming skills dont go beyond bash and a little pearl :-/

[kencu (Ken)]

I installed initial libproc support for Tiger here <​https://github.com/macports/macports-legacy-support/commit/5607e8902a9569b27ed4917bc0aeac55c10796b5> and it is presently in legacy-support-devel. Assuming these changes turn out as well as expected, I'll roll them out for general use soon.

It still doesn't have the missing proc_pidpath function, but it's a step in that direction.

[kencu (Ken)]

No good news -- the proc_info kernel system call that backstops these functions appears to have made it into the last Tiger i386 kernel, but not the last Tiger PPC kernel.

So the libproc headers might be able to be used on Tiger i386 -- but not PPC, at present. Most of us using Tiger use Tiger PPC for the MacOS9 layer, and AFAIK there's no reason to stay on Tiger i386.

Building a new kernel for Tiger PPC that has these calls in it is way beyond my skill set. Likewise, I don't know how to write a kernel extension (kext) that would offer this either. So it's possible Tiger PPC will permanently be out-of-luck for this.

[kencu (Ken)]

It turns out that building a new kernel for Tiger is actually not very hard after all, much to my surprise, given that there is a very straightforward recipe to follow here <​https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/build/build.html#//apple_ref/doc/uid/TP30000905-CH221-CIHFEIFI>.

Other than having to copy one missing header <mach-o/kld.h> from xnu-792.24.17/EXTERNAL_HEADERS/mach-o/kld.h into /usr/include/mach-o, to build kextsymboltool, it built through just by following the directions in about 10 minutes total.

So one never knows...

[dzhub]

Sounds promising, gonna hold my breath :-) Thank you so much for your engagement.