#69327 closed defect (fixed)
Symlinks in prefix make destroot layout unpredictable
Reported by: | Dave-Allured (Dave Allured) | Owned by: | SoapZA |
---|---|---|---|
Priority: | Normal | Milestone: | MacPorts 2.9.2 |
Component: | base | Version: | |
Keywords: | Cc: | reneeotten (Renee Otten) | |
Port: | py312-meson |
Description
port install meson
fails to install a working executable for meson
command, causing build failures in other ports. I am building from sources into a custom prefix, on Monterey 12.7.2.
rackmac1:~/port/291/bin 45> where ninja /Users/dallured/port/291/bin/ninja rackmac1:~/port/291/bin 46> where meson rackmac1:~/port/291/bin 47> ls -go | grep -e ninja -e meson lrwxr-xr-x 1 39 Feb 12 06:52 meson -> /Users/dallured/port/291/bin/meson-3.12 -rwxr-xr-x 1 410168 Feb 11 18:56 ninja rackmac1:~/port/291/bin 48> ./meson ./meson: Command not found.
I am not sure whether this problem is in port meson, or port py312-meson. It could be either. The build log for py312-meson shows many suspicious warnings for py312-meson. "Operation not permitted", etc. See attached build log.
Attachments (4)
Change History (30)
Changed 9 months ago by Dave-Allured (Dave Allured)
Attachment: | meson.install.0212c.reinstall.log added |
---|
comment:1 Changed 9 months ago by Dave-Allured (Dave Allured)
On closer examination:
- Most, if not all, of "Operation not permitted" warnings are normal for installing without root privileges. Therefore those warnings are not relevant, and can be ignored.
- This looks like the real problem. Port py312-meson is not correctly installing into the custom prefix. How to fix?
Warning: violation by /Volumes Warning: py312-meson violates the layout of the ports-filesystems!
comment:2 Changed 9 months ago by kencu (Ken)
just to be clear here.
you have this file:
/Users/dallured/port/291/bin/meson
it is a symlink to this:
/Users/dallured/port/291/bin/meson -> /Users/dallured/port/291/bin/meson-3.12
yet when you do this:
cd /Users/dallured/port/291/bin ./meson
you get this:
./meson: Command not found
is that all correct?
If it is, does this exist?
/Users/dallured/port/291/bin/meson-3.12
comment:3 Changed 9 months ago by Dave-Allured (Dave Allured)
Yes, that is all correct. That ls -go
followed by grep was supposed to show that, but that was a bit obscure, sorry.
The meson sym link was put in by the meson port. It was expecting that py312-meson already existed in the same directory, but it is not there.
Also to clarify further, my custom prefix is /Users/dallured/port/291
. Everything is supposed to be installing under here. This has worked out quite well for hundreds of ports, but it hit this meson thing today.
comment:4 follow-up: 6 Changed 9 months ago by ryandesign (Ryan Carsten Schmidt)
The meson port installs the $prefix/bin/meson symlink that points to $prefix/bin/meson-3.12
The py312-meson port installs $prefix/bin/meson-3.12
So if you're missing meson-3.12, then the py312-meson port's build/installation log is the one we need to be examining.
comment:5 Changed 9 months ago by kencu (Ken)
you might also do a
port contents py312-meson
to see where bin/meson-3.12 actually is
that should be fun ;>
comment:6 Changed 9 months ago by ryandesign (Ryan Carsten Schmidt)
Replying to ryandesign:
the py312-meson port's build/installation log is the one we need to be examining.
And I see it is included in the file you attached, but you appear to have provided the output of running port
with the -v
flag, so it's missing all the additional detail that we would have gotten from a main.log file or running port
with the -d
flag.
py-meson, like all py-* ports, uses the python portgroup, which is where the code that creates the bin symlinks is (in a post-destroot block). If you're missing this symlink for the py312-meson port I would expect you to be missing the symlinks for all other py-* ports too.
Changed 9 months ago by Dave-Allured (Dave Allured)
Attachment: | py312-meson.0212c.port-contents.txt added |
---|
Output from port contents py312-meson
Changed 9 months ago by Dave-Allured (Dave Allured)
Attachment: | meson.0212c.installed.txt added |
---|
List of all my installed ports
comment:7 Changed 9 months ago by kencu (Ken)
looks like your home folder is symlink to something else?
comment:8 follow-up: 10 Changed 9 months ago by Dave-Allured (Dave Allured)
symlink to something else?
Yes. I had never noticed that. It is a shared mac within a server room. in /Users, there are a bunch of sym links to physical user home directories.
Changed 9 months ago by Dave-Allured (Dave Allured)
Attachment: | py312-meson.0212e.reinstall.log added |
---|
I uninstalled and re-ran port -d install py312-meson
only, to get the requested extra debug info. Here is the console output. There was no main.log because, I guess, the build thought it was successful.
comment:9 Changed 9 months ago by ryandesign (Ryan Carsten Schmidt)
Yes, main.log files are deleted after successful builds unless you set keeplogs yes
in macports.conf or use the -k
flag on the command line.
comment:10 Changed 9 months ago by ryandesign (Ryan Carsten Schmidt)
From your re-run log, this line is where the portgroup would create the symlink that you're missing:
DEBUG: Executing proc-post-org.macports.destroot-destroot-0
Unfortunately the portgroup doesn't log anything when it does or does not create the symlink. You could try adding some logging code there to see what's happening.
Replying to Dave-Allured:
symlink to something else?
Yes. I had never noticed that. It is a shared mac within a server room. in /Users, there are a bunch of sym links to physical user home directories.
From your port contents we can see that some parts of MacPorts are evidently resolving that symlink:
Port py312-meson contains: /Users/dallured/port/291/share/py312-meson/bash-completion/completions/meson
while other parts aren't:
/Volumes/RM1_Homes/dallured/port/291/Library/Frameworks/Python.framework/Versions/3.12/bin/meson
I don't know if that's the source of the problem but you could try reinstalling MacPorts base with --prefix=/Volumes/RM1_Homes/dallured/port/291
instead of --prefix=/Users/dallured/port/291
, and similarly changing applications_dir
and frameworks_dir
if you've customized them. If reinstalling py312-meson after doing that fixes the problem, then you could use port contents
on your installed ports to locate those that mention paths in in /Users/dallured/port/291 and then rebuild those ports.
comment:11 Changed 9 months ago by ryandesign (Ryan Carsten Schmidt)
Cc: | reneeotten added; reneeotten@… removed |
---|---|
Port: | meson removed |
Summary: | meson @1.3.1: Missing primary meson command → py312-meson @1.3.1: Missing meson-3.12 command |
comment:12 Changed 9 months ago by reneeotten (Renee Otten)
the intention of the meson
port is to install the pyXY-meson
port, where XY is MacPorts’ current default Python version. Additionally, it will create the symlink(s) so that you and other ports (including the meson PortGroup) should/will JustWork. That does work where I have tested it and there have not been any other reports of issues, so it appears this is something unusual on your specific setup. Not sure if I (or anyone else here) can fix this for you..
comment:13 follow-up: 16 Changed 9 months ago by Dave-Allured (Dave Allured)
it appears this is something unusual on your specific setup
Apparently so. Ken and Ryan have correctly identified that user home directories on this machine go through sym links. My guess is that this really is the source of this particular problem. For myself, I can look at a workaround.
The lingering question is, should support for this rare special case (sym linked home directory) be added to Macports for general application?
comment:14 Changed 9 months ago by jmroot (Joshua Root)
Cf. #21082, #39850. In short, things get really confusing when the prefix is not a fully resolved and normalised path. Whether the normalised or un-normalised version is created in the destroot, anything not checking for both will potentially get it wrong. I would guess the glob in the python portgroup is just not matching anything.
You probably don't need to reinstall base since you're not actually moving any files, just update macports.conf to use the resolved path for everything.
comment:15 Changed 9 months ago by reneeotten (Renee Otten)
Resolution: | → invalid |
---|---|
Status: | assigned → closed |
okay, so this has nothing to do with the py-meson
or meson
ports.
comment:16 Changed 9 months ago by kencu (Ken)
Replying to Dave-Allured:
The lingering question is, should support for this rare special case (sym linked home directory) be added to Macports for general application?
Ideally, sure, but in real life -- probably too many pitfalls to reliably do it.
If configuring macports to install into --prefix=/Volumes/RM1_Homes/dallured/port/291
fixes things up for you (it should) then someone could add that to the documentation as a workaround.
comment:17 Changed 9 months ago by jmroot (Joshua Root)
It would likely require patching a lot of build systems to handle such paths consistently in the destroot phase. It might be python that is normalising the path in this case; you could check with python3.12 -c "import sysconfig; print(sysconfig.get_config_var('base'))"
.
comment:18 Changed 9 months ago by ryandesign (Ryan Carsten Schmidt)
Resolution: | invalid |
---|---|
Status: | closed → reopened |
Let's leave it open for a minute to confirm whether fully resolving the MacPorts prefix fixes the problem. If it does, then the next question is if we can do anything about it. It may not be a bug in a specific python port or in the python portgroup but we might consider it to be a bug in base.
If it is, mentioning it in the documentation is a good start. Another good step could be for port diagnose
to print an error if the prefix is not fully resolved. And finally, could the configure script do the hard work of determining if the path is fully resolved, and either error out if it is not or even better just fix it internally and use the fully resolved path instead?
comment:19 Changed 9 months ago by Dave-Allured (Dave Allured)
Yes, leave this open for a bit, for the conversation. Thank you. I am in a crunch this week, but will try the prefix test in a few days. I think Renee has a point, that this may not have much to do specifically with the meson ports.
comment:20 Changed 9 months ago by Dave-Allured (Dave Allured)
As suggested, I ran port contents
to locate ports trying to use the fully resolved prefix that includes /Volumes
. Only the py-* ports did this. Note that base ports python310
, python311
, python312
respected the original prefix and did not try to use the fully resolved prefix. It seems that this problem may be isolated to py-* ports only, so a portgroup solution might be feasible.
rackmac1:~/port 56> foreach p ( `cat logfiles/py312-meson.0212e.installed.list ` ) foreach? echo `port contents $p | grep -c /Volumes` " $p" |& tee -a counts.volumes-str.0212e foreach? end 0 aom 0 autoconf 0 automake 0 bash 0 bison 0 bison-runtime 0 brotli 0 bzip2 0 cctools 0 clang-16 0 clang_select 0 cmake 0 cmake-bootstrap 0 curl 0 curl-ca-bundle 0 db48 0 djvulibre 0 expat 0 fftw-3 0 flex 0 fontconfig 0 freetype 0 gawk 0 gcc13 0 gcc13-libcxx 0 gcc_select 0 gdbm 0 gettext 0 gettext-runtime 0 gettext-tools-libs 0 giflib 0 git 0 gmake 0 gmp 0 gperf 0 grep 0 gsed 0 hatchling_select 0 help2man 0 icu 0 imath 0 isl 0 jasper 0 jbig2dec 0 jbigkit 0 kerberos5 0 lapack 0 lcms2 0 ld64 0 ld64-xcode 0 lerc 0 libarchive 0 libb2 0 libcomerr 0 libcxx 0 libde265 0 libdeflate 0 libedit 0 libffi 0 libgcc 0 libgcc13 0 libiconv 0 libidn 0 libidn2 0 libjpeg-turbo 0 libmpc 0 libomp 0 libpaper 0 libpng 0 libpsl 0 libraw 0 libtextstyle 0 libtool 0 libunistring 0 libuv 0 libxml2 0 libyaml 0 llvm-16 0 llvm_select 0 lmdb 0 lz4 0 lzip 0 lzo2 0 m4 0 mpfr 0 nasm 0 ncurses 0 nghttp2 0 ninja 0 openexr 0 openjpeg 0 openssl 0 openssl3 0 ossp-uuid 0 p5.34-authen-sasl 0 p5.34-b-cow 0 p5.34-canary-stability 0 p5.34-capture-tiny 0 p5.34-cgi 0 p5.34-clone 0 p5.34-common-sense 0 p5.34-compress-raw-bzip2 0 p5.34-compress-raw-zlib 0 p5.34-config-autoconf 0 p5.34-cpan-meta 0 p5.34-cpan-meta-check 0 p5.34-cpan-meta-requirements 0 p5.34-cpan-meta-yaml 0 p5.34-data-optlist 0 p5.34-data-section 0 p5.34-devel-checkbin 0 p5.34-devel-symdump 0 p5.34-digest-hmac 0 p5.34-digest-sha1 0 p5.34-encode 0 p5.34-encode-locale 0 p5.34-error 0 p5.34-extutils-cbuilder 0 p5.34-extutils-makemaker 0 p5.34-extutils-manifest 0 p5.34-file-slurper 0 p5.34-getopt-long 0 p5.34-gssapi 0 p5.34-html-parser 0 p5.34-html-tagset 0 p5.34-http-date 0 p5.34-http-message 0 p5.34-inc-latest 0 p5.34-io-compress 0 p5.34-io-compress-brotli 0 p5.34-io-html 0 p5.34-io-socket-ip 0 p5.34-io-socket-ssl 0 p5.34-ipc-cmd 0 p5.34-json 0 p5.34-json-xs 0 p5.34-locale-gettext 0 p5.34-locale-maketext-simple 0 p5.34-lwp-mediatypes 0 p5.34-mixin-linewise 0 p5.34-module-build 0 p5.34-module-load 0 p5.34-module-load-conditional 0 p5.34-module-metadata 0 p5.34-mozilla-ca 0 p5.34-mro-compat 0 p5.34-net-libidn 0 p5.34-net-smtp-ssl 0 p5.34-net-ssleay 0 p5.34-params-check 0 p5.34-params-util 0 p5.34-perl-ostype 0 p5.34-perlio-utf8_strict 0 p5.34-pod-coverage 0 p5.34-pod-coverage-trustpod 0 p5.34-pod-escapes 0 p5.34-pod-eventual 0 p5.34-pod-parser 0 p5.34-pod-simple 0 p5.34-podlators 0 p5.34-regexp-ipv6 0 p5.34-scalar-list-utils 0 p5.34-socket 0 p5.34-software-license 0 p5.34-sub-exporter 0 p5.34-sub-install 0 p5.34-sub-name 0 p5.34-sub-uplevel 0 p5.34-term-readkey 0 p5.34-test-cpan-meta 0 p5.34-test-cpan-meta-json 0 p5.34-test-deep 0 p5.34-test-exception 0 p5.34-test-failwarnings 0 p5.34-test-fatal 0 p5.34-test-leaktrace 0 p5.34-test-more-utf8 0 p5.34-test-needs 0 p5.34-test-nowarnings 0 p5.34-test-pod 0 p5.34-test-pod-coverage 0 p5.34-test-simple 0 p5.34-test-warn 0 p5.34-test-warnings 0 p5.34-text-parsewords 0 p5.34-text-template 0 p5.34-time-hires 0 p5.34-time-local 0 p5.34-timedate 0 p5.34-try-tiny 0 p5.34-types-serialiser 0 p5.34-uri 0 p5.34-version 0 p5.34-xsloader 0 pcre2 0 perl5 0 perl5.34 0 pkgconfig 0 popt 0 py-bootstrap-modules 28 py311-build 12 py311-calver 0 py311-cython-compat 11 py311-editables 175 py311-flit_core 202 py311-hatchling 43 py311-installer 49 py311-packaging 29 py311-pathspec 27 py311-pluggy 970 py311-pygments 19 py311-pyproject_hooks 687 py311-setuptools 91 py311-setuptools_scm 12 py311-trove-classifiers 7 py311-typing_extensions 88 py311-wheel 60 py311-yaml 28 py312-build 175 py312-flit_core 43 py312-installer 692 py312-meson 49 py312-packaging 19 py312-pyproject_hooks 687 py312-setuptools 88 py312-wheel 0 pygments_select 0 python3_select 0 python3_select-311 0 python3_select-312 0 python310 0 python311 0 python312 0 python_select 0 python_select-311 0 python_select-312 0 re2c 0 readline 0 rsync 0 sqlite3 0 svt-av1 0 texinfo 0 tiff 0 urw-fonts 0 webp 0 x265 0 xar 0 xxhashlib 0 xz 0 yasm 0 zlib 0 zstd
comment:21 Changed 9 months ago by Dave-Allured (Dave Allured)
Here is a fix for the python portgroup that will create the missing sym links. I prefer this over changing my prefix or config file. In short, it provides the missing path normalization at the right place. Thanks @jmroot for the tip about glob not finding the correct binaries, that was the exact problem.
PR submitted: https://github.com/macports/macports-ports/pull/22735
Edit 2024 Feb 27: This PR was cancelled, superseded by the following fixes to base, by @jmroot. Looks good.
comment:22 Changed 9 months ago by jmroot (Joshua Root)
Resolution: | → fixed |
---|---|
Status: | reopened → closed |
comment:23 Changed 9 months ago by jmroot (Joshua Root)
Component: | ports → base |
---|---|
Milestone: | → MacPorts Future |
Summary: | py312-meson @1.3.1: Missing meson-3.12 command → Symlinks in prefix make destroot layout unpredictable |
comment:24 Changed 9 months ago by Dave-Allured (Dave Allured)
@jmroot et al, nice fix in that commit to Macports base. Thank you!
I have an imminent need for that fix, for testing something else that was not cured by my portgroup fix. What do you recommend? Copy the single file src/macports1.0/macports.tcl
into my local Macports install? Copy only the single patch? Clone the full development snapshot? Something else? When is the next base release expected?
comment:25 Changed 9 months ago by jmroot (Joshua Root)
If you don't mind being on the bleeding edge, you can install from master. Otherwise you can install from the release-2.9 branch, or apply just the desired patches directly to your installation. If you do the latter, make sure to get the followup commit [981ae2bed378c5cb513fcdb3fea788d3f61972e9/macports-base] as well.
comment:26 Changed 8 months ago by jmroot (Joshua Root)
Milestone: | MacPorts Future → MacPorts 2.9.2 |
---|
Build log for py312-meson and meson