Opened 4 years ago

Last modified 3 years ago

#60792 reopened defect

rsync: re-enable HFS compression

Reported by: ryandesign (Ryan Carsten Schmidt) Owned by: mascguy (Christopher Nielsen)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: jbenninghoff@…, jimjag (Jim Jagielski), cooljeanius (Eric Gallager)
Port: rsync

Description

The rsync port used to use an official patch to add support for HFS compression. As far as I know, "HFS compression" continues to be supported on APFS and is a valuable feature that we want to enable if we can.

This patch was removed when the port was updated to 3.1.3 with the remark:

hfs-compression.diff is marked by upstream as broken as of 3.1.3

Homebrew seems to have followed our lead in removing this patch, but nobody seems to know why we did this. The rsync mailing list has been twice asked and nobody has replied. John, could you clarify where upstream says this patch is broken, or explain in what way it is broken?

In addition, we have the problem that when HFS compression support was added to the rsync port, we added not only hfs-compression.diff from the official patches directory but also hfs-compression-options.diff. Where did that second patch come from? Jim, did you make it?

Change History (17)

comment:1 in reply to:  description ; Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)

John responded by email:

Hello, I don’t recall where I found where samba.org marked it as broken. My recollection is that it was in a readme file included with either rsync or rsync-patches. http://rsync.samba.org/ftp/rsync/rsync-patches-3.1.3.tar.gz is no longer available, and the new location for rsync-patches, GitHub, doesn’t contain any such info.

I can tell you that I submitted the PR because I have an old PowerPC mac and a) rsync from an intel mac running rsync 3.1.3 to PowerPC mac running 3.1.2 was crashing. I’m pretty sure I tried to get the port working with the hfs-compression patch but was not successful, found _something_ that said it was broken, so I removed it. See: https://github.com/macports/macports-ports/pull/1299

FWIW, the patch itself is labeled as UNTESTED, and doesn’t appear to have been updated in the past 9 years, since 3.1.0.dev. Mike Bombich’s original patch isn’t around any more, either. http://www.bombich.com/rsync.html - last snapshot on the wayback machine, http://web.archive.org/web/20120805080158/http://www.bombich.com/rsync.html.

comment:2 in reply to:  1 ; Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)

John sent another reply by email:

I’m not a member so my email bounced…I seem to have found where it was marked as broken:

https://github.com/WayneD/rsync-patches/blame/master/hfs-compression.diff#L551

If you try applying the patch & compiling, this error will be kicked out:

xattrs.c:274:1: error: unknown type name '$$$ERROR$$$'
$$$ERROR$$$ the old patch needs reworking since rsync_xal_get() has totally changed!

This was first added for 3.1.3, still present in 3.2.2.

comment:3 in reply to:  2 Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)

Replying to jbenninghoff:

Hello, I don’t recall where I found where samba.org marked it as broken. My recollection is that it was in a readme file included with either rsync or rsync-patches. http://rsync.samba.org/ftp/rsync/rsync-patches-3.1.3.tar.gz is no longer available

It's available at https://download.samba.org/pub/rsync/src/rsync-patches-3.1.3.tar.gz. I had already looked in there and didn't see anything at the top of the patch that would indicate it didn't work. I looked now at rsync-3.1.3.tar.gz, and didn't find any mention of "hfs" in it.

I can tell you that I submitted the PR because I have an old PowerPC mac and a) rsync from an intel mac running rsync 3.1.3 to PowerPC mac running 3.1.2 was crashing.

Ok thanks, that's at least something I can try to reproduce the issue. Do you happen to remember what OS versions you were running?

I’m pretty sure I tried to get the port working with the hfs-compression patch but was not successful, found _something_ that said it was broken, so I removed it. See: https://github.com/macports/macports-ports/pull/1299

Yup, I had read that PR, but it didn't say why it was broken.

FWIW, the patch itself is labeled as UNTESTED, and doesn’t appear to have been updated in the past 9 years, since 3.1.0.dev.

The patch seems to at least get refreshed in each rsync-patches tar.gz. For example, the patchfile did change between 3.2.1 and 3.2.2. (Not its functionality, but the line numbers in the patch.)

Replying to jbenninghoff:

I seem to have found where it was marked as broken:

https://github.com/WayneD/rsync-patches/blame/master/hfs-compression.diff#L551

If you try applying the patch & compiling, this error will be kicked out:

xattrs.c:274:1: error: unknown type name '$$$ERROR$$$'
$$$ERROR$$$ the old patch needs reworking since rsync_xal_get() has totally changed!

This was first added for 3.1.3, still present in 3.2.2.

Ahhhh that explains it and gives me somewhere to look. Thanks!

comment:4 Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)

John responded by email:

Yes, the patchfile was deliberately broken before the 3.1.3 release with the $$$ERROR$$$ message. It has been maintained since then, but only minimally (just line numbers as you say).

Seeing that error, I’m quite sure that’s why I took out the patch.

comment:5 Changed 4 years ago by ryandesign (Ryan Carsten Schmidt)

I'm asking upstream about this patch here: https://github.com/WayneD/rsync-patches/issues/1

comment:6 Changed 4 years ago by mf2k (Frank Schima)

Owner: set to ryandesign
Status: newassigned

:)

comment:7 Changed 4 years ago by mascguy (Christopher Nielsen)

Cc: mascguy added

comment:8 Changed 4 years ago by mascguy (Christopher Nielsen)

This is on my official to-do list. Ryan/anyone, please assign this to me.

comment:9 Changed 4 years ago by mascguy (Christopher Nielsen)

Owner: changed from ryandesign to mascguy

comment:10 Changed 4 years ago by mascguy (Christopher Nielsen)

Cc: mascguy removed

comment:11 Changed 3 years ago by Haravikk

Was any progress made on this? I always found --hfs-compression extremely useful for saving space when syncing into a disk image, so its loss for newer versions of rsync was a bit of a blow. While my eventual aim *should* be to use ZFS or such instead, that's easier said than done.

If a full fix requires a lot more work, would it be possible as an alternative/stop-gap to create an "rsync-3.1.2" port specifically for 3.1.2 to make this easier for people to install and retain in the same way that we have multiple ports for different versions of mariadb etc.?

comment:12 Changed 3 years ago by mascguy (Christopher Nielsen)

I briefly looked at this several months ago, but had to defer it due to other commitments. However, it's still on my long-term To Do list.

If I can't make any progress over the next few weeks, I'll look at adding a 3.1.2-specifc port.

comment:13 Changed 3 years ago by cooljeanius (Eric Gallager)

Cc: cooljeanius added

comment:14 Changed 3 years ago by Christopher Nielsen <mascguy@…>

Resolution: fixed
Status: assignedclosed

In 4f145ee6bd117ce0285d5743642115075daf5600/macports-ports (master):

rsync-hfscomp: new port for rsync 3.1.2, with HFS compression
Fixes: #60792

comment:15 Changed 3 years ago by mascguy (Christopher Nielsen)

The new port rsync-hfscomp should be available within the next hour or so. To use, first resync your ports, and then do the following:

$ sudo port -f deactivate rsync
$ sudo port -N install rsync-hfscomp
$ sudo port -f uninstall rsync

Also, the various dependents, like git, have been updated to support either rsync-hfscomp or rsync. So going forward, you shouldn't have any issues.

Give it a try, and let us know if all is well.

comment:16 Changed 3 years ago by mascguy (Christopher Nielsen)

There's just one caveat with new port rsync-hfscomp: Builds are presently failing on Big Sur, due to gettimeofday().

Tracked by issue:63669

comment:17 Changed 3 years ago by mascguy (Christopher Nielsen)

Resolution: fixed
Status: closedreopened

While we now provide a separate port for rsync 3.1.2 with HFS compression (rsync-hfscomp), that's only a short-term fix. The ultimate goal is to rework those patches for the latest upstream release.

Reopening this, to cover work for the latter.

Note: See TracTickets for help on using tickets.