Opened 5 years ago
Last modified 3 years ago
#60230 assigned defect
bsdtar --hfsCompression fails for non-root users
Reported by: | AgilentGCMS | Owned by: | eborisch (Eric A. Borisch) |
---|---|---|---|
Priority: | Normal | Milestone: | |
Component: | ports | Version: | 2.6.2 |
Keywords: | Cc: | eborisch (Eric A. Borisch), tobypeterson | |
Port: | libarchive |
Description
I recently ran 'port upgrade outdated', and after building some packages it failed with the error message
---> Activating libarchive @3.4.2_0 Error: Failed to activate libarchive: command execution failed
after which I am unable to do much with port, because it seems that port wants to install libarchive to fix broken packages first, which fails. The 'main.log' for libarchive is attached.
Attachments (1)
Change History (34)
Changed 5 years ago by AgilentGCMS
comment:1 Changed 5 years ago by AgilentGCMS
comment:2 Changed 5 years ago by mf2k (Frank Schima)
In the future, please add the port maintainer(s) to Cc (port info --maintainers libarchive
), if any.
comment:3 Changed 5 years ago by mf2k (Frank Schima)
Owner: | set to tobypeterson |
---|---|
Status: | new → assigned |
comment:4 Changed 5 years ago by eborisch (Eric A. Borisch)
You're operating with a custom (not /opt/local) MacPorts location, so it's always possible we've missed something in testing...
A few quick questions:
- When you're in the state where activating fails, what's the output of
which bsdtar
andbsdtar --version
? (Be sure to check after asudo port deactivate libarchive
, as this would happen before activating a new version.) - What type of filesystem is /Users/sbasu1/packages/macports on?
Thanks!
comment:5 Changed 5 years ago by eborisch (Eric A. Borisch)
Cc: | eborisch added |
---|
comment:6 follow-up: 7 Changed 5 years ago by iplasma
This happens to me for cctools. Tried libarchive 3.4.0_1 and 3.4.2_0, both have same problem with activation. Setting "hfscompression no" in macports.conf fixes it.
comment:7 Changed 5 years ago by eborisch (Eric A. Borisch)
Replying to iplasma:
This happens to me for cctools. Tried libarchive 3.4.0_1 and 3.4.2_0, both have same problem with activation. Setting "hfscompression no" in macports.conf fixes it.
Can you answer the first of the bulleted questions I had in my comment above? Also what OS version are you on? Thanks!
comment:8 follow-up: 9 Changed 5 years ago by AgilentGCMS
As I mentioned in my original ticket, before trying port upgrade outdated
, this was the bsdtar version
$ bsdtar --version bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4
Also,
$ which bsdtar /Users/sbasu1/packages/macports/bin/bsdtar
As for filesystem type, diskutil list
says APFS Volume Macintosh HD
.
comment:9 Changed 5 years ago by eborisch (Eric A. Borisch)
Replying to AgilentGCMS:
As I mentioned in my original ticket, before trying
port upgrade outdated
, this was the bsdtar version$ bsdtar --version bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4Also,
$ which bsdtar /Users/sbasu1/packages/macports/bin/bsdtarAs for filesystem type,
diskutil list
saysAPFS Volume Macintosh HD
.
During an upgrade, it first deactivates the (old version) port and then activates the new version. It is the result of which bsdtar
during this deactivated state that I’m looking for. Sorry if my original question was not sufficiently clear.
So, sudo port deactivate bsdtar && which bsdtar
would be the desired commands to run and report the result from. (It will not be the one from MacPorts as you have reported here unless something is wrong, as it shouldn’t be present at that point.)
comment:10 Changed 5 years ago by AgilentGCMS
OK, before I post the results, I need to clear up one thing. I *did* manage to port upgrade outdated
10 days ago by disabling hfscompression
. If that makes my answer useless, then I apologize, but I don't know how to go back to the state before I did port upgrade outdated
. Having said that, here's what happens currently.
$ port deactivate bsdtar Error: port deactivate failed: Image error: port bsdtar is not active. $ bsdtar --version bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4 $ which bsdtar /Users/sbasu1/packages/macports/bin/bsdtar
comment:11 Changed 5 years ago by eborisch (Eric A. Borisch)
Is this location where it is trying to install into, as well, or do you have multiple macports installations?
If this is where your macports installs into, then your on-disk state and MP’s database aren’t lining up — MacPorts thinks libarchive is not active, yet its contents (including bsdtar) clearly are in place, at least partially. I’m not sure how it got into this state, but it will be difficult to determine what is causing the activate/compression issue until that is resolved.
If you have two macports installations, then all bets are off; this isn’t a supported configuration.
comment:12 Changed 5 years ago by AgilentGCMS
No, I only have one macports installation. The 'port' command resolves to the correct location
$ which port /Users/sbasu1/packages/macports/bin/port
comment:13 Changed 5 years ago by tobypeterson
Rebuilt base to force usage of bsdtar; cannot reproduce this issue. I'm using the standard location.
comment:14 Changed 5 years ago by tobypeterson
Ok, I _can_ repro this manually by using /usr/bin/bsdtar, which is confusing.
/usr/bin/bsdtar -xvp --hfsCompression -f /opt/local/var/macports/software/libarchive/libarchive-3.4.2_0.darwin_19.x86_64.tbz2
For some reason, it doesn't fail (for me) during activate, even though it's using /usr/bin/bsdtar there as well.
comment:15 Changed 5 years ago by tobypeterson
Ah, got it! It's because I run as root. From the attached log: :debug:main Privilege de-escalation not attempted as not running as root.
comment:16 Changed 5 years ago by tobypeterson
This is really a base bug; /usr/bin/bsdtar (at least on macOS 10.15) does not appear to correctly support --hfsCompression
when running as non-root.
comment:17 Changed 5 years ago by tobypeterson
Component: | ports → base |
---|
comment:18 Changed 5 years ago by tobypeterson
Summary: | Failed to activate libarchive → /usr/bin/bsdtar --hfsCompression fails for non-root users (Failed to activate libarchive) |
---|
comment:19 follow-up: 22 Changed 5 years ago by tobypeterson
Based on #60176, macOS 10.14's /usr/bin/bsdtar is also unsuitable.
comment:20 Changed 5 years ago by tobypeterson
Owner: | changed from tobypeterson to eborisch |
---|
comment:21 Changed 5 years ago by tobypeterson
Cc: | tobypeterson added |
---|
comment:22 Changed 5 years ago by jmroot (Joshua Root)
Summary: | /usr/bin/bsdtar --hfsCompression fails for non-root users (Failed to activate libarchive) → bsdtar --hfsCompression fails for non-root users |
---|
Replying to tobypeterson:
Based on #60176, macOS 10.14's /usr/bin/bsdtar is also unsuitable.
We don't attempt to use it though. The 10.14 user's issue was with the libarchive port's bsdtar. Whereas on 10.15 /usr/bin/bsdtar claims to support hfscompression so we do use it.
All the information so far points to this being a new bug introduced in libarchive 3.4.2. It was confirmed that 3.4.0 worked fine with non-root installs and it seemed OK with 3.4.1 as well AFAIK.
comment:23 follow-up: 30 Changed 5 years ago by jmroot (Joshua Root)
Milestone: | → MacPorts 2.6.3 |
---|
This is both a bug in a port, and an OS bug that base needs to work around. We need to:
- Fix the libarchive port and get the fix accepted upstream. If someone could bisect the commits since 3.4.1, that would be very helpful.
- Make base not use hfscompression if it would be using a broken version of /usr/bin/bsdtar and not running as root.
comment:24 Changed 5 years ago by tobypeterson
Sorry, I may be misunderstanding; what's the bug in the port?
/usr/bin/bsdtar --hfsCompression generates a ton of errors; /opt/local/bin/bsdtar (libarchive-3.4.2_0) works fine.
edit: I see, some suspicion that libarchive 3.4.2 may be broken on 10.14. I'll poke around in a VM to see if I can repro that.
comment:25 Changed 5 years ago by tobypeterson
Reopened #60176 to continue investigating that. Let's track the base bug here.
comment:26 Changed 5 years ago by eborisch (Eric A. Borisch)
#60176 and this are both likely due to issues with extended attributes that shouldn't be there (show up with '@' on ls -l listing) combined with an issue within libarchive if:
- the activated file is set read-only
- AND has extended attributes (not compression iteslf, as this is applied separately)
- AND is not being extracted by root
See also my comment here on libarchive.
For #60176, this can be fixed by squashing stray resource forks (as the port already does for a number of files) in the manpages; I haven't looked further at this one yet.
comment:27 follow-up: 28 Changed 5 years ago by eborisch (Eric A. Borisch)
I don't have 10.15 running, but from here it appears it has libarchive 3.3.2, which has --hfsCompression, but does not have my fix for supporting non-root extraction in it (which went into 3.4.0).
We may just want to change our test to see if our libarchive is installed rather than if it claims to support --hfsCompression.
I still think supporting compression is a big win, especially for developers (header files are excellent compression fodder, and compilers have lots of them) or anyone on a laptop (fixed size storage; every byte counts.)
comment:28 Changed 5 years ago by jmroot (Joshua Root)
comment:29 Changed 4 years ago by jmroot (Joshua Root)
Upon further investigation, the case in #60176 has been broken all along, it's not a regression since 3.4.0 as I previously thought.
comment:30 Changed 4 years ago by jmroot (Joshua Root)
Replying to jmroot:
- Fix the libarchive port and get the fix accepted upstream. If someone could bisect the commits since 3.4.1, that would be very helpful.
It doesn't look like anyone ever took this upstream, so: https://github.com/libarchive/libarchive/issues/1415
comment:31 Changed 4 years ago by jmroot (Joshua Root)
comment:32 Changed 4 years ago by jmroot (Joshua Root)
Component: | base → ports |
---|---|
Milestone: | MacPorts 2.6.3 |
Workaround is back in base 2.6.3 but the bug in libarchive still needs to be addressed, so leaving this open for that.
comment:33 Changed 3 years ago by jmroot (Joshua Root)
This may have been fixed in 3.5.0; release notes mention "support for system extended attributes", but it's hard to check since the original test file has since been purged.
I found a similar issue in #56563. Even though a comment there said that the underlying issue was fixed in libarchive 3.4.0, and my libarchive is 3.4.2
I can confirm that the problem reported by #56563 is the same one I'm having, and the fix (disabling
hfscompression
inmacports.conf
for non-root installs) is the same as well. So perhaps the libarchive bug is back.