libsdl: cursor lacks transparency in High Sierra (macOS 10.13)

[markuskimius]

SDL 1.2 mouse cursors lack transparency under macOS 10.13. Pixels in the cursor icon that should be transparent are displayed as inverse-transparent pixels instead (XOR'ed with white). See the attached example code.

The issue occurs with the default cursor as well as cursors that are loaded by calling SDL_CreateCursor().

The issue does not occur in macOS 10.12, built against the SDL 1.2 library (libsdl) that ships with MacPorts pkg for Sierra.

The issue was observed in macOS 10.13, built against the SDL 1.2 library (libsdl) that ships with MacPorts pkg for High Sierra.

[markuskimius]

Example code and resulting screenshots on macOS 10.12 and 10.13

[ryandesign (Ryan Carsten Schmidt)]

It doesn't sound like a MacPorts-specific problem. Have you reported it to the developers of libsdl? Have you checked whether the problem is also present in libsdl2?

[markuskimius]

The problem is not present in libsdl2.

The issue has not been reported to the developers of libsdl yet. I thought perhaps it's a porting issue because the issue is observable on a single version of the OS. The issue is also observable when the binary built on 10.12 is executed on 10.13 with libsdl shared libraries from the MacPorts pkg for 10.12, which also led me to check with the porting team first.

I'm not sure SDL developers work on SDL 1.x anymore but let me check with them.

[markuskimius]

​Bug 4076 has been filed with the SDL development team.

[jmroot (Joshua Root)]

We don't build SDL any differently on High Sierra. If the same binary gives different behaviour on different OS versions, that would suggest that it is the OS that is doing something different.

[kencu (Ken)]

I noticed this issue in the builds of basiliskii and sheepshaver on high sierra, both of which use SDL 1.2 at present.

[markuskimius]

SDL developers have made a patch which I have confirmed addresses the issue. Thank you!

• The SDL branch including the patch: ​https://github.com/kanjitalk755/SDL/tree/forHighSierra
• The SDL bug tracker discussion thread: ​https://bugzilla.libsdl.org/show_bug.cgi?id=4076

[kencu (Ken)]

I found only this part of the referenced patch was needed (other parts are already in MacPorts):

$ cat patch-sdl-1.2-10.13-cursor-fix.diff
diff --git a/src/video/quartz/SDL_QuartzWM.m b/src/video/quartz/SDL_QuartzWM.m
index d526424d4..6a0c98695 100644
--- a/src/video/quartz/SDL_QuartzWM.m
+++ b/src/video/quartz/SDL_QuartzWM.m
@@ -55,7 +55,7 @@ void QZ_FreeWMCursor     (_THIS, WMcursor *cursor) {
     
     /* copy data and mask, extending the mask to all black pixels because the inversion effect doesn't work with Cocoa's alpha-blended cursors */
     for (i = 0; i < (w+7)/8*h; i++) {
-        planes[0][i] = data[i] ^ 0xFF;
+        planes[0][i] = ~data[i] & mask[i];
         planes[1][i] = mask[i] | data[i];
     }

The cursor artifact is gone on BasiliskII with this patch, but cursor movement (refresh?) seems more sluggish -- maybe that's my imagination.

Sometimes, when the screen is initializing, I can still see a cursor mask artifact, but not in the running program. I haven't tested the effect of this patch on any systems prior to 10.13 as yet.