procmail @3.22 on OS X Yosemite hits "Abort trap: 6" if variable assignment uses backticks

[macports@…]

If my .procmailrc file contains a variable assignment like this

FROM=`sed 1d | formail -I reply-to: -I resent-from: -I return-path: | \
        formail -rzxTo:`

then procmail hits an "Abort trap: 6" as follows:

 mini1:~ collin$ cat Mail/backup/92862 | formail -imessage-id: | /opt/local/bin/procmail .procmailrc.1.18
 Abort trap: 6
 mini1:~ collin$

/usr/bin/procmail displays the same symptom. Both binaries are version 3.22 (they display the same version string as in procmail -v string), though they have some different attributes:

mini1:~ collin$ ls -l /usr/bin/procmail /opt/local/bin/procmail
-rwsr-sr-x  1 root  mail  97612 Dec  7 01:23 /opt/local/bin/procmail
-rwxr-sr-x  1 root  mail  84576 Sep  9 15:57 /usr/bin/procmail
mini1:~ collin$ file /usr/bin/procmail /opt/local/bin/procmail 
/usr/bin/procmail:       setgid Mach-O 64-bit executable x86_64
/opt/local/bin/procmail: setuid setgid Mach-O 64-bit executable x86_64
mini1:~ collin$ 

Removing the FROM=... line makes the problem go away--i.e., procmail does what I expect (and indeed what it does do on Linux, and what /usr/bin/procmail formerly did on Mac OS X 10.6.8).

The same .procmailrc script has worked OK for a few years on a mac mini. Until 2 days back, this mac mini had been running 10.6.8 (not sure when the last OS upgrade was, but it has been months).

I was hoping for a coredump so I naively tried "ulimit -c unlimited" but no coredump appeared after the abort.

The backtick sequence inside a recipe doesn't provoke an abort; the following works just fine:

:0 ic
| cd backup && rm -f dummy `ls -t | sed -e 1,128d`

It seems to be only a line of the form

VAR=`cmd`

that the current procmail (both /usr/bin/procmail and the port) hates.

Also: I tried replacing the long "FROM=" line above by

FROM=`cat`

which also proved disappointing

[danielluke (Daniel J. Luke)]

Note that this port is 'nomaintainer' in macports (and this doesn't sound like a macports bug, but a procmail bug), so you'll probably have better luck addressing this with upstream.

Having said that, you could try running your procmail under a debugger to at least get a backtrace of the problem...

[ryandesign (Ryan Carsten Schmidt)]

Replying to macports@…:

/usr/bin/procmail displays the same symptom.

This proves it's not a MacPorts-specific problem. Please report it to the developers of procmail.

[macports@…]

This proves it's not a MacPorts-specific problem.

D'oh! Good point. Apparently I'm an idiot.

Unfortunately the developers of procmail appear to have been inactive for about a dozen years, so this procmail "bug" may go unfixed.

[brian@…]

Many thanks, I had exactly the same problem and removing this line:

HOST=`hostname`

from ~/.procmailrc allowed me to workaround. Gonna go with that until the procmail developers become active again and fix this ...

[l2dy (Zero King)]

As of procmail @3.22_7:

(lldb) bt
* thread #1: tid = 0x4853, 0x00007fff951aaf06 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00007fff951aaf06 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff839914ec libsystem_pthread.dylib`pthread_kill + 90
    frame #2: 0x00007fff939c86df libsystem_c.dylib`abort + 129
    frame #3: 0x00007fff939c8856 libsystem_c.dylib`abort_report_np + 181
    frame #4: 0x00007fff939eea0c libsystem_c.dylib`__chk_fail + 48
    frame #5: 0x00007fff939eea1c libsystem_c.dylib`__chk_fail_overlap + 16
    frame #6: 0x00007fff939eea4d libsystem_c.dylib`__chk_overlap + 49
    frame #7: 0x00007fff939eeab6 libsystem_c.dylib`__strlcpy_chk + 68
    frame #8: 0x0000000100006e36 procmail`readparse(p="Return-path: <...>", fpgetc=(procmail`getb at cstdio.c:154), sarg=1, skipping=0) + 4678 at goodies.c:316
    frame #9: 0x0000000100003c90 procmail`mainloop + 4944 at procmail.c:923
    frame #10: 0x00000001000022e4 procmail`main(argc=2, argv=0x00007fff5fbffa88) + 4516 at procmail.c:461
    frame #11: 0x00007fff819165ad libdyld.dylib`start + 1
    frame #12: 0x00007fff819165ad libdyld.dylib`start + 1
(lldb) frame select 8
frame #8: 0x0000000100006e36 procmail`readparse(p="Return-path: <...>", fpgetc=(procmail`getb at cstdio.c:154), sarg=1, skipping=0) + 4678 at goodies.c:316
   313 		    }
   314 		   else
   315 	copyit:	    { size_t len=fencepost-p+1;
-> 316 		      if(strlcpy(p,startb,len)>=len)		   /* simply copy it */
   317 			 skipping|=1;			      /* did we truncate it? */
   318 		      if(got<=SKIPPING_SPACE)		/* can only occur if sarg!=0 */
   319 			 got=NORMAL_TEXT;
(lldb) frame variable
[...]
(char *) p = 0x0000000101000005 "Return-path: <...>"
(char *) startb = 0x0000000101000005 "Return-path: <...>"
(size_t) len = 2044
(lldb) frame select 9
frame #9: 0x0000000100003c90 procmail`mainloop + 4944 at procmail.c:923
   920 		skipspace();
   921 		if(testB('='))				   /* removal or assignment? */
   922 		 { *chp='=';
-> 923 		   if(readparse(++chp,getb,1,skiprc))
   924 		      continue;
   925 		 }
   926 		else
(lldb) frame select 10
frame #10: 0x00000001000022e4 procmail`main(argc=2, argv=0x00007fff5fbffa88) + 4516 at procmail.c:461
   458 		   if(buildpath(maildir,defmaildir,rcfile))
   459 		      break;
   460 		if(tryopen(0,rctype,dowarning))
-> 461 		 { register int rcs=mainloop();				   /* run it */
   462 		   if(rcs==rcs_DELIVERED)
   463 		      goto mailed;					 /* success! */
   464 		   if(rcs==rcs_EOF)

[csanchezdll]

The problem comes from the way strlcpy is used. On some cases (notably when substituting backticks) the buffers passed to strlcpy overlap (they are the same buffer, actually). In OS X, behaviour of strlcpy under this circumstances is noted as "undefined" in the man page.

I have created a pull request that disables the use of system strlcpy, and uses a replacement provided in procmail source code. This is the same that is done on systems without strlcpy (Linux, por example). ​https://github.com/macports/macports-ports/pull/7323

This might be a procmail bug or not. Surely it is not MacPorts-specific, but might be OSX-specific (strlcpy is not POSIX so I am unsure on what is the normative behavior for overlapping buffers). Anyways, procmail is so seldom used these days that sticking with the code tested on Linux just sounds sensible.

[csanchezdll]

In 0ab6f3fbcba57b30212f4abe1b32529e44784ae1/macports-ports (master):

procmail: avoid using strlcat/strlcpy

• strlcpy in macOS has undefined behaviour if strings overlap, which causes backtick substitution to fail.
• this patch foces to use a substitute function provided in procmail source for systems where strlcat/strlcpy are not available.

Fixes: #46623

[jmroot (Joshua Root)]

Replying to csanchezdll:

Surely it is not MacPorts-specific, but might be OSX-specific

It's the same on the BSDs (unsurprisingly since that's where Apple got the code). See e.g. ​https://www.freebsd.org/cgi/man.cgi?query=strlcpy&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE+and+Ports&arch=default&format=html

[csanchezdll]

Just for completeness, I did a small experiment with this program:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define STRING "test"

int
main (int argc, char *argv[])
{
  char p[sizeof (STRING)] = STRING;
  char *q = p;

  strlcpy (q, p, sizeof (STRING));

  puts (p);

  exit (EXIT_SUCCESS);
}

Under NetBSD, it works ok and prints "test" (I do not have access to a FreeBSD box at the moment).

Under OS X, it aborts ("Abort trap: 6", same as procmail). As an interesting side note, if I change sizeof (STRING) by strlen (p) + 1 I get the following warning (only under OS X):

test.c:13:26: warning: size argument in '__builtin___strlcpy_chk' call appears
      to be size of the source; expected the size of the destination
      [-Wstrlcpy-strlcat-size]
  strlcpy (q, p, strlen (p) + 1);
                 ~~~~~~~~^~~~~~
/usr/include/secure/_string.h:108:34: note: expanded from macro 'strlcpy'
                __builtin___strlcpy_chk (dest, __VA_ARGS__, __darwin_obs...
                                               ^~~~~~~~~~~
1 warning generated.

This suggests OS X compiler has some built-in for strlcpy that might have a different behavior than BSD libraries.

I agree OS X behavior is correct, as the man page says overlapping buffers yield undefined behavior. This should really be fixed upstream in procmail. But it has been unmaintained for years and the code for parsing is really difficult to follow. So using the provided replacement just gets the work done.