"port provides" not working on Tiger

[kencu (Ken)]

Same on both the i386 and the PPC build of Macports 2.5.2 on Tiger.

$ port
MacPorts 2.5.2
Entering shell mode... ("help" for help, "quit" to quit)
[Users/cunningh] > quit
Goodbye
tigerg5:~ cunningh$ port provides /opt/local/lib/gcc7/libstdc++.dylib
Error: process_cmd failed: sqlite error: near "COLLATE": syntax error (1) while executing query: SELECT id FROM registry.files WHERE (actual_path = ? COLLATE NOCASE) AND active

certainly can live without it. Haven't noticed any other issues on Tiger so far...

[jmroot (Joshua Root)]

Probably introduced by ​https://github.com/macports/macports-base/pull/19

[Ionic (Mihai Moldovan)]

Tiger's system sqlite version is probably too old. The earliest version I tested was 10.6.

How did the DB upgrade work in the first place? Shouldn't it have thrown an error, since we now create a COLLATE NOCASE index? Without that index, search operations are very slow.

[jmroot (Joshua Root)]

COLLATE in expressions (as opposed to in table definitions) was added in SQLite 3.3.13. ​http://sqlite.org/releaselog/3_3_13.html

[Ionic (Mihai Moldovan)]

Tiger seems to have 3.1.3, so definitely too old. We could replace that statement with a LIKE statement for older sqlite versions (or platforms). LIKE did use the COLLATE NOCASE index in my tests if I remember correctly, so it should be about equivalent.

[jmroot (Joshua Root)]

Or we could just set cs = 0 if the sqlite version is too old.

[Ionic (Mihai Moldovan)]

Yeah, but then we'd lose CS support. OTOH, LIKE would require special handling of wildcard characters (ESCAPE expression and proper setup for % and _), sooo...

[ryandesign (Ryan Carsten Schmidt)]

Would an acceptable solution be to bundle a newer copy of libsqlite3 with MacPorts base, like we already bundle Tcl?

[raimue (Rainer Müller)]

We also already require lots of workarounds in the update_db statements for Tiger. I would say it is about time to drop support for Tiger (even the latest update is over 10 years old), but I know that others will not like the idea...

[kencu (Ken)]

The error comes up in some other operations as well, like rev-upgrade. For now, Tiger-users should not use MacPorts 2.5.2 I would say. Too late, maybe.

I do note that the MacPorts configure script accepts

--with-sqlite3prefix

so for right now I will try to build a current copy of sqlite into my /opt/bootstrap MacPorts installation (that is currently still running MacPorts-2.4.1 through laziness), and leverage that as I already do with curl to build my official install into /opt/local.

[kencu (Ken)]

so that does work, in the end. Perhaps not a trivial workaround, but fairly simple.

macOS 10.4.11 8S2169 i386

$ port
MacPorts 2.5.2
Entering shell mode... ("help" for help, "quit" to quit)
[Users/cunningh] > quit
Goodbye

$ port provides /opt/local/lib/libgcc/libstdc++.6.dylib

/opt/local/lib/libgcc/libstdc++.6.dylib is provided by: libgcc

one very tiny hiccup. The Makefile in src/cregistry/Makefile doesn't propogate the --with-sqlite3prefix include paths, so I had to add that manually to that Makefile.

.c.o:
	${CC} -c -DUSE_TCL_STUBS ${CFLAGS} ${CPPFLAGS} -I/opt/bootstrap/include ${SHLIB_CFLAGS} $< -o $@

[kencu (Ken)]

As a reminder, Yan12125 has shown the way to fairly easily bundle in xz, curl, and libressl to macports-base. I imagine it might not be a huge stretch to bundle in sqlite3, depending on what other deps sqlite3 needs, as Ryan suggests.

<​https://github.com/yan12125/macports-base/branches>

That way MacPorts-base could stay relatively unimpeded by spaghetti code, and all the older systems could get up to date versions of the MacPorts underpinnings.

[kencu (Ken)]

This small patch to macports-base appears to make src/cregistry/Makefile respond to the --with-sqlite3prefix configure argument:

--- src/cregistry/Makefile.in.orig      2018-06-07 13:05:14.000000000 -0700
+++ src/cregistry/Makefile.in   2018-06-07 13:23:15.000000000 -0700
@@ -13,6 +13,8 @@
 
 # required for strdup(3) on Linux and OS X
 CPPFLAGS+=-D_XOPEN_SOURCE=600
+CPPFLAGS+=${SQLITE3_CFLAGS}
+LDFLAGS+=${SQLITE3_LIBS}
 
 all:: ${STLIB_NAME} ${SQLEXT_NAME}

and with that the build proceeds apace.

The folder it is referencing /opt/bootstrap/include does have a number of different headers in it, and it's not a pure header collection for sqlite3... but no issues noted so far with that.

[kencu (Ken)]

Hah. We had a bundled copy of sqlite3 once upon a time, but it was removed years ago.

<​https://github.com/macports/macports-base/commit/487430c644337a7732d2d0ba1951478163519712>

[jmroot (Joshua Root)]

Probably worth noting that if we update to Tcl 8.6, we get sqlite in the bargain.

[ryandesign (Ryan Carsten Schmidt)]

Tcl 8.6 does come with a bundled copy of sqlite3, but it's not the latest version.

I have been wanting to bring up the possibility of updating our bundled Tcl to 8.6, but that's a much larger and more problematic proposition and I don't think I have the free time to work on that right now.

Bundling the latest copy of sqlite3 with MacPorts seems unproblematic to me. It will fix this problem; it will give other macOS versions a newer sqlite too which might fix other as yet undiscovered bugs, performance problems or security flaws; and allows us to simplify our code by removing workarounds for older versions. If other devs disagree, please explain.

[jmroot (Joshua Root)]

In 6ba4c34427930dd28780d7040632f6d97c2cff01/macports-base (master):

Restore compatibility with sqlite3 < 3.3.13

Tiger won't get the case-insensitivity improvement from ​https://github.com/macports/macports-base/pull/19, but working like
it did before is better than failing.

Fixes: #56620

[jmroot (Joshua Root)]

In 32e081a0ea7f7e3c5631ecdc3bb4d292272c28bb/macports-base (release-2.5):

Restore compatibility with sqlite3 < 3.3.13

Tiger won't get the case-insensitivity improvement from ​https://github.com/macports/macports-base/pull/19, but working like
it did before is better than failing.

Fixes: #56620
(cherry picked from commit 6ba4c34427930dd28780d7040632f6d97c2cff01)