Opened 5 years ago

Last modified 3 years ago

#60230 assigned defect

bsdtar --hfsCompression fails for non-root users

Reported by: AgilentGCMS Owned by: eborisch (Eric A. Borisch)
Priority: Normal Milestone:
Component: ports Version: 2.6.2
Keywords: Cc: eborisch (Eric A. Borisch), tobypeterson
Port: libarchive

Description

I recently ran 'port upgrade outdated', and after building some packages it failed with the error message

--->  Activating libarchive @3.4.2_0
Error: Failed to activate libarchive: command execution failed

after which I am unable to do much with port, because it seems that port wants to install libarchive to fix broken packages first, which fails. The 'main.log' for libarchive is attached.

Attachments (1)

main.log (10.3 KB) - added by AgilentGCMS 5 years ago.

Download all attachments as: .zip

Change History (34)

Changed 5 years ago by AgilentGCMS

Attachment: main.log added

comment:1 Changed 5 years ago by AgilentGCMS

I found a similar issue in #56563. Even though a comment there said that the underlying issue was fixed in libarchive 3.4.0, and my libarchive is 3.4.2

$ bsdtar --version
bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4

I can confirm that the problem reported by #56563 is the same one I'm having, and the fix (disabling hfscompression in macports.conf for non-root installs) is the same as well. So perhaps the libarchive bug is back.

Last edited 5 years ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:2 Changed 5 years ago by mf2k (Frank Schima)

In the future, please add the port maintainer(s) to Cc (port info --maintainers libarchive), if any.

comment:3 Changed 5 years ago by mf2k (Frank Schima)

Owner: set to tobypeterson
Status: newassigned

comment:4 Changed 5 years ago by eborisch (Eric A. Borisch)

You're operating with a custom (not /opt/local) MacPorts location, so it's always possible we've missed something in testing...

A few quick questions:

  • When you're in the state where activating fails, what's the output of which bsdtar and bsdtar --version? (Be sure to check after a sudo port deactivate libarchive, as this would happen before activating a new version.)
  • What type of filesystem is /Users/sbasu1/packages/macports on?

Thanks!

comment:5 Changed 5 years ago by eborisch (Eric A. Borisch)

Cc: eborisch added

comment:6 Changed 5 years ago by iplasma

This happens to me for cctools. Tried libarchive 3.4.0_1 and 3.4.2_0, both have same problem with activation. Setting "hfscompression no" in macports.conf fixes it.

Last edited 5 years ago by iplasma (previous) (diff)

comment:7 in reply to:  6 Changed 5 years ago by eborisch (Eric A. Borisch)

Replying to iplasma:

This happens to me for cctools. Tried libarchive 3.4.0_1 and 3.4.2_0, both have same problem with activation. Setting "hfscompression no" in macports.conf fixes it.

Can you answer the first of the bulleted questions I had in my comment above? Also what OS version are you on? Thanks!

Last edited 5 years ago by eborisch (Eric A. Borisch) (previous) (diff)

comment:8 Changed 5 years ago by AgilentGCMS

As I mentioned in my original ticket, before trying port upgrade outdated, this was the bsdtar version

$ bsdtar --version
bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4

Also,

$ which bsdtar
/Users/sbasu1/packages/macports/bin/bsdtar

As for filesystem type, diskutil list says APFS Volume Macintosh HD.

comment:9 in reply to:  8 Changed 5 years ago by eborisch (Eric A. Borisch)

Replying to AgilentGCMS:

As I mentioned in my original ticket, before trying port upgrade outdated, this was the bsdtar version

$ bsdtar --version
bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4

Also,

$ which bsdtar
/Users/sbasu1/packages/macports/bin/bsdtar

As for filesystem type, diskutil list says APFS Volume Macintosh HD.

During an upgrade, it first deactivates the (old version) port and then activates the new version. It is the result of which bsdtar during this deactivated state that I’m looking for. Sorry if my original question was not sufficiently clear.

So, sudo port deactivate bsdtar && which bsdtar would be the desired commands to run and report the result from. (It will not be the one from MacPorts as you have reported here unless something is wrong, as it shouldn’t be present at that point.)

comment:10 Changed 5 years ago by AgilentGCMS

OK, before I post the results, I need to clear up one thing. I *did* manage to port upgrade outdated 10 days ago by disabling hfscompression. If that makes my answer useless, then I apologize, but I don't know how to go back to the state before I did port upgrade outdated. Having said that, here's what happens currently.

$ port deactivate bsdtar
Error: port deactivate failed: Image error: port bsdtar is not active.
$ bsdtar --version
bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4
$ which bsdtar
/Users/sbasu1/packages/macports/bin/bsdtar

comment:11 Changed 5 years ago by eborisch (Eric A. Borisch)

Is this location where it is trying to install into, as well, or do you have multiple macports installations?

If this is where your macports installs into, then your on-disk state and MP’s database aren’t lining up — MacPorts thinks libarchive is not active, yet its contents (including bsdtar) clearly are in place, at least partially. I’m not sure how it got into this state, but it will be difficult to determine what is causing the activate/compression issue until that is resolved.

If you have two macports installations, then all bets are off; this isn’t a supported configuration.

comment:12 Changed 5 years ago by AgilentGCMS

No, I only have one macports installation. The 'port' command resolves to the correct location

$ which port
/Users/sbasu1/packages/macports/bin/port

comment:13 Changed 5 years ago by tobypeterson

Rebuilt base to force usage of bsdtar; cannot reproduce this issue. I'm using the standard location.

comment:14 Changed 5 years ago by tobypeterson

Ok, I _can_ repro this manually by using /usr/bin/bsdtar, which is confusing.

/usr/bin/bsdtar -xvp --hfsCompression -f /opt/local/var/macports/software/libarchive/libarchive-3.4.2_0.darwin_19.x86_64.tbz2

For some reason, it doesn't fail (for me) during activate, even though it's using /usr/bin/bsdtar there as well.

Last edited 5 years ago by tobypeterson (previous) (diff)

comment:15 Changed 5 years ago by tobypeterson

Ah, got it! It's because I run as root. From the attached log: :debug:main Privilege de-escalation not attempted as not running as root.

comment:16 Changed 5 years ago by tobypeterson

This is really a base bug; /usr/bin/bsdtar (at least on macOS 10.15) does not appear to correctly support --hfsCompression when running as non-root.

comment:17 Changed 5 years ago by tobypeterson

Component: portsbase

comment:18 Changed 5 years ago by tobypeterson

Summary: Failed to activate libarchive/usr/bin/bsdtar --hfsCompression fails for non-root users (Failed to activate libarchive)

comment:19 Changed 5 years ago by tobypeterson

Based on #60176, macOS 10.14's /usr/bin/bsdtar is also unsuitable.

comment:20 Changed 5 years ago by tobypeterson

Owner: changed from tobypeterson to eborisch

comment:21 Changed 5 years ago by tobypeterson

Cc: tobypeterson added

comment:22 in reply to:  19 Changed 5 years ago by jmroot (Joshua Root)

Summary: /usr/bin/bsdtar --hfsCompression fails for non-root users (Failed to activate libarchive)bsdtar --hfsCompression fails for non-root users

Replying to tobypeterson:

Based on #60176, macOS 10.14's /usr/bin/bsdtar is also unsuitable.

We don't attempt to use it though. The 10.14 user's issue was with the libarchive port's bsdtar. Whereas on 10.15 /usr/bin/bsdtar claims to support hfscompression so we do use it.

All the information so far points to this being a new bug introduced in libarchive 3.4.2. It was confirmed that 3.4.0 worked fine with non-root installs and it seemed OK with 3.4.1 as well AFAIK.

comment:23 Changed 5 years ago by jmroot (Joshua Root)

Milestone: MacPorts 2.6.3

This is both a bug in a port, and an OS bug that base needs to work around. We need to:

  1. Fix the libarchive port and get the fix accepted upstream. If someone could bisect the commits since 3.4.1, that would be very helpful.
  2. Make base not use hfscompression if it would be using a broken version of /usr/bin/bsdtar and not running as root.
Last edited 5 years ago by jmroot (Joshua Root) (previous) (diff)

comment:24 Changed 5 years ago by tobypeterson

Sorry, I may be misunderstanding; what's the bug in the port?

/usr/bin/bsdtar --hfsCompression generates a ton of errors; /opt/local/bin/bsdtar (libarchive-3.4.2_0) works fine.

edit: I see, some suspicion that libarchive 3.4.2 may be broken on 10.14. I'll poke around in a VM to see if I can repro that.

Last edited 5 years ago by tobypeterson (previous) (diff)

comment:25 Changed 5 years ago by tobypeterson

Reopened #60176 to continue investigating that. Let's track the base bug here.

Last edited 5 years ago by tobypeterson (previous) (diff)

comment:26 Changed 5 years ago by eborisch (Eric A. Borisch)

#60176 and this are both likely due to issues with extended attributes that shouldn't be there (show up with '@' on ls -l listing) combined with an issue within libarchive if:

  • the activated file is set read-only
  • AND has extended attributes (not compression iteslf, as this is applied separately)
  • AND is not being extracted by root

See also my comment here on libarchive.

For #60176, this can be fixed by squashing stray resource forks (as the port already does for a number of files) in the manpages; I haven't looked further at this one yet.

comment:27 Changed 5 years ago by eborisch (Eric A. Borisch)

I don't have 10.15 running, but from here it appears it has libarchive 3.3.2, which has --hfsCompression, but does not have my fix for supporting non-root extraction in it (which went into 3.4.0).

We may just want to change our test to see if our libarchive is installed rather than if it claims to support --hfsCompression.

I still think supporting compression is a big win, especially for developers (header files are excellent compression fodder, and compilers have lots of them) or anyone on a laptop (fixed size storage; every byte counts.)

comment:28 in reply to:  27 Changed 5 years ago by jmroot (Joshua Root)

Replying to eborisch:

We may just want to change our test to see if our libarchive is installed rather than if it claims to support --hfsCompression.

That won't help, the reporter of #60176 was using our libarchive on 10.14.

comment:29 Changed 4 years ago by jmroot (Joshua Root)

Upon further investigation, the case in #60176 has been broken all along, it's not a regression since 3.4.0 as I previously thought.

comment:30 in reply to:  23 Changed 4 years ago by jmroot (Joshua Root)

Replying to jmroot:

  1. Fix the libarchive port and get the fix accepted upstream. If someone could bisect the commits since 3.4.1, that would be very helpful.

It doesn't look like anyone ever took this upstream, so: https://github.com/libarchive/libarchive/issues/1415

comment:31 Changed 4 years ago by jmroot (Joshua Root)

In 9d0ebd5b5b1a7fd0f3a05be85bf0e7267c63b70a/macports-base (master):

Disable hfscompression for non-root installs (again)

See: #60230

comment:32 Changed 4 years ago by jmroot (Joshua Root)

Component: baseports
Milestone: MacPorts 2.6.3

Workaround is back in base 2.6.3 but the bug in libarchive still needs to be addressed, so leaving this open for that.

comment:33 Changed 3 years ago by jmroot (Joshua Root)

This may have been fixed in 3.5.0; release notes mention "support for system extended attributes", but it's hard to check since the original test file has since been purged.

Note: See TracTickets for help on using tickets.