neomutt @20200925_0 w3m help / header cache problem / no authenticators

[arappe]

First, thank you for maintaining NeoMutt!

1. w3m: w3m does not compile on Big Sur. But I think that neomutt does not rely on w3m. One can use lynx. I edited the PortFile and can compile 20200925 and 20200814 on Big Sur after deleting w3m as a build dependency.

2. hcache: I think that the 20200925 update breaks the header cache. Executed on either Catalina and Big Sur, and compiled on either Catalina or Big Sur, 20200925 (specifically neomutt @20200925_0+doc+gdbm+gnutls+gpgme+idn+mutt+sasl) upon startup scans the header cache and then proceeds to reject it and request headers starting from 0. Killing neomutt, activating neomutt @20200814_0+doc+gdbm+gnutls+gpgme+idn+mutt+sasl and then upon startup neomutt scans the same stored header cache file and accepts it, only acquiring the few new headers since previous run.

3. No authenticators found. For neomutt compiled on Big Sur for both 20200925 and 20200814 I cannot send outgoing mail. I am told "No authenticators found." The same .muttrc file and the same email account can send outgoing mail in Catalina. (Multiple email accounts: yahoo, gmail, O365) And the "Darwin 19" neomutt compiled on Catalina (which still works on Big Sur) can send outgoing mail in Big Sur (either 20200925 or 20200814). The problem is with Big Sur compilation leading to loss of outgoing mail service. Any suggestions?

Thanks!

[lbschenkel (Leonardo Brondani Schenkel)]

Would you be willing to report 2 and 3 to upstream? I could do that, but I'm not sure if there's any advantage of me being an intermediary.

Regarding 1, I no longer remember with confidence but I have the impression that I tried all options, and w3m was the only one that rendered the doc faithfully. The others produced docs with visual defects. (But then again, I don't remember the details.) This doesn't mean this can't be re-evaluated. I must warn, though, that due to personal circumstances I'm quite short on free time at the moment.

[arappe]

Thanks for replying. About reporting 2 and 3 to upstream: (what is "upstream" exactly?) Do you mean to the maintainers of mutt (outside of MacPorts)? Can you tell me where/how I do that? I am willing to make the report.

About 1: within this trac system several have reported that w3m does not compile in Big Sur. The reply seems to be that w3m is unmaintained. So even if w3m is better than lynx (interesting by the way) requiring w3m at build time means "no neomutt for me". I thought that reporting the work-around was valuable. And we can continue to hope that someone fixes w3m.

[arappe]

Updates

1. w3m build now works. I see online that clang-9.0 is being used to facilitate compilation. So this issue can be set aside.
2. header cache works. Upon switch to Big Sur-compiled neomutt, the header cache is discarded...but only once. After getting the header cache with the Big Sur build, it is then respected by neomutt the next time that neomutt is run.
3. I think that the "no authenticators found" is, sadly, an issue for a MacPorts maintainer.

sudo port install neomutt +doc+gdbm+gnutls+gpgme+idn+mutt+sasl still gives an install that cannot send outgoing mail.

However, a standalone compilation of mutt-1.14.6.tar.gz with

configure -with-ssl=/opt/local --with-gdbm=/opt/local --prefix=/Users/xxxxx/DEV/code/software/macosx/muttdir/standalone_compile/mutt_install --enable-hcache --enable-gpgme --with-sasl --enable-imap

sudo gmake install

gives me a version of mutt on the same exact computer and with the same exact muttrc file that can send outgoing mail.

I tried many versions of the MacPorts variants:

1) neomutt @20200925_0

2) neomutt @20200925_0+doc+gdbm+gnutls+gpgme+gss+idn+mutt+sasl

3) neomutt @20200925_0+doc+gdbm+gnutls+gpgme+idn+mutt+sasl

4) neomutt @20200925_0+doc+gdbm+gnutls+gpgme+mutt+sasl

5) neomutt @20200925_0+doc+gdbm+gpgme+idn+mutt+sasl

6) neomutt @20200925_0+doc+gpgme+idn+mutt

7) neomutt @20200925_0+doc+gpgme+idn+mutt+sasl

8) neomutt @20200925_0+gnutls+gpgme+mutt+sasl

9) neomutt @20200925_0+gpgme+mutt+sasl

10) neomutt @20200925_0+mutt

11) neomutt @20200925_0+mutt+sasl

If there's some other combination that I should try, I'd be glad to know about it. Other suggestions?

[lbschenkel (Leonardo Brondani Schenkel)]

Thanks for the update. I'll see when I can investigate number 3, but as I said given my current personal circumstances I don't have much free time to dedicate to this project. Any investigation and/or information you can gather up in the meantime will be greatly appreciated.

[arappe]

Thanks. Not rushing you; I totally respect the time you have already put into maintaining neomutt for us!

I now have what I think is a totally working build (outside MacPorts) for neomutt as well as for mutt. So I think it must be the way the neomutt MacPorts variants are translating into the configure flags for the original distribution.

prompt% tar xvfz neomutt-20201120.tar.gz

prompt% cd neomutt-20201120

prompt% gmake clean

prompt% ./configure --gpgme --gnutls --sasl --ssl --gdbm --idn2 --disable-idn

prompt% sudo gmake

(gmake install didn't work in terms of installing in my preferred directory. Tried --prefix and DESTDIR. No luck.) (just run program from build directory)

I report this for other neomutt fans out there, that there is a workaround that's pretty accessible.

[tristancollins (Tristan Collins)]

I'm having the same issue. I recently upgraded from (A) Catalina -> to (B) Big Sur but kept MacPorts as it was from (A). No migration or updates. The neomutt binary from (A) worked fine on Big Sur.

I updated MacPorts today and got the error described in (3) above. Details below.

I copied the 'working' neomutt binary from the (A) Catalina backup to the (B) Big Sur system, and it fails just as in (3).

The differences in the outputs from neomutt -v seem to be the version of OpenSSL used (g->h), the Darwin version, the CFLAGS used and the notmuch library.

So perhaps the issue is with a change in OpenSSL?

The output of neomutt -v from a working (A) Catalina 20200814 version:

 System: Darwin 19.6.0 (x86_64)
 ncurses: ncurses 6.2.20200212 (compiled with 6.2.20200212)
 libiconv: 1.16
 libidn2: 2.3.0 (compiled with 2.3.0)
 GPGME: 1.14.0-unknown
 OpenSSL: OpenSSL 1.1.1g  21 Apr 2020
 libnotmuch: 5.2.0
 storage: gdbm
 compression: zlib

Configure options: --prefix=/opt/local --enable-doc --disable-idn --with-ncurses=/opt/local --with-nls=/opt/local --with-ssl=/opt/local --with-zlib=/opt/local --with-gdbm=/opt/local --with-gpgme=/opt/local --with-idn2=/opt/local --with-notmuch=/opt/local --with-sasl=/opt/local

Compilation CFLAGS: -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -std=c99 -D_ALL_SOURCE=1 -D_GNU_SOURCE=1 -D__EXTENSIONS__ -I/opt/local/include -DNCURSES_WIDECHAR

Default options:
  +attach_headers_color +compose_to_sender +compress +cond_date +debug
  +encrypt_to_self +forgotten_attachments +forwref +ifdef +imap +index_color
  +initials +limit_current_thread +multiple_fcc +nested_if +new_mail +nntp +pop
  +progress +quasi_delete +regcomp +reply_with_xorig +sensible_browser +sidebar
  +skip_quoted +smtp +status_color +timeout +tls_sni +trash

Compile options:
  -autocrypt +bkgdset +color +curs_set +fcntl -flock -fmemopen +futimens
  +getaddrinfo -gnutls +gpgme -gss +hcache -homespool +idn -inotify
  -locales_hack -lua +meta -mixmaster +nls +notmuch +openssl +pgp +regex +sasl
  +smime -sqlite +start_color +sun_attachment +typeahead
MAILPATH="/var/mail"
PKGDATADIR="/opt/local/share/neomutt"
SENDMAIL="/usr/sbin/sendmail"
SYSCONFDIR="/opt/local/etc"

From a non-working BigSur 20200925 MacPorts version:

System: Darwin 20.1.0 (x86_64)
ncurses: ncurses 6.2.20200212 (compiled with 6.2.20200212)
libiconv: 1.16
libidn2: 2.3.0 (compiled with 2.3.0)
GPGME: 1.14.0-unknown
OpenSSL: OpenSSL 1.1.1h  22 Sep 2020
libnotmuch: 5.3.0
storage: gdbm
compression: zlib

Configure options: --prefix=/opt/local --enable-doc --disable-idn --with-ncurses=/opt/local --with-nls=/opt/local --with-ssl=/opt/local --with-zlib=/opt/local --with-gdbm=/opt/local --with-gpgme=/opt/local --with-idn2=/opt/local --with-notmuch=/opt/local --with-sasl=/opt/local

Compilation CFLAGS: -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -arch x86_64 -std=c99 -D_ALL_SOURCE=1 -D_GNU_SOURCE=1 -D__EXTENSIONS__ -I/opt/local/include -DNCURSES_WIDECHAR

Default options:
  +attach_headers_color +compose_to_sender +compress +cond_date +debug
  +encrypt_to_self +forgotten_attachments +forwref +ifdef +imap +index_color
  +initials +limit_current_thread +multiple_fcc +nested_if +new_mail +nntp +pop
  +progress +quasi_delete +regcomp +reply_with_xorig +sensible_browser +sidebar
  +skip_quoted +smtp +status_color +timeout +tls_sni +trash

Compile options:
  -autocrypt +bkgdset +color +curs_set +fcntl -flock -fmemopen +futimens
  +getaddrinfo -gnutls +gpgme -gss +hcache -homespool +idn -inotify
  -locales_hack -lua +meta -mixmaster +nls +notmuch +openssl +pgp +regex +sasl
  +smime -sqlite +start_color +sun_attachment +typeahead
MAILPATH="/var/mail"
PKGDATADIR="/opt/local/share/neomutt"
SENDMAIL="/usr/sbin/sendmail"
SYSCONFDIR="/opt/local/etc"

I can't get the below sequence to result in a 'working' version on my system for issue (3) above:

prompt% gmake clean
prompt% ./configure --gpgme --gnutls --sasl --ssl --gdbm --idn2 --disable-idn
prompt% sudo gmake

[arappe]

As of today, I find that the MacPorts neomutt with BigSur has the same "no authenticators" problem as above. No change. Clean install of MacPorts. sudo port install neomutt (with flags as above) --> "no authenticators".

Therefore, the only way I can be helpful maybe is in figuring out why your standalone neomutt compile (from .tgz) didn't work.

You wrote "I can't get the below sequence to result in a 'working' version on my system for issue (3) above:

(do you mean it didn't compile? didn't run? still complains about no authenticators?) Post some more info on that sub-issue and maybe I can get you back on the road. And perhaps knowing how the standalone version can compile will make it easier for someone MacPorts-savvier than me to adapt the MacPorts version later on....

[tristancollins (Tristan Collins)]

Replying to arappe:

You wrote "I can't get the below sequence to result in a 'working' version on my system for issue (3) above:

(do you mean it didn't compile? didn't run? still complains about no authenticators?) Post some more info on that sub-issue and maybe I can get you back on the road. And perhaps knowing how the standalone version can compile will make it easier for someone MacPorts-savvier than me to adapt the MacPorts version later on....

Still complains about the no authenticators. I ended up up switching to Homebrew and using the version there - works fine.

[arappe]

Just a quick post to say that I upgraded to OSX 11.2.3 and Xcode 12.4 (I get the complaint that port cannot tell what version of the command line tools I have, despite my following all the hotlist suggestions)

With this OS X config I find that neomutt installs properly and can cache headers and authenticate. AFAICS neomutt in MacPorts now works as advertised. Not sure what changed; just wanted people to know.

% sudo port install neomutt +doc+gdbm+gnutls+gpgme+idn+mutt+sasl

% port installed|grep neomutt

neomutt @20200925_0+doc+gdbm+gnutls+gpgme+idn+mutt+sasl (active)

Just making a report that the problems I saw before seem to have worked out.