Fixed bug 3697 - Main thread gets stuck on left mouse down

Amruth Raj

- My app runs in full screen to play video(I use SDL_WINDOW_FULLSCREEN_DESKTOP)
- Cmd-tab to go out of full screen to another app
- Cmd-tab again to get back to my app
- Press left mouse button at one of the edges of the screen, don't release yet.
After this point the main thread is stuck until I release the left mouse button and hence video rendering doesn't happen anymore.

On debugging more, I see that thread 0 is stuck as shown below with sendEvent processing left mouse down. It comes out only after it receives a left mouse up. There are some frames below which show NSWindowResizing, but my window flag doesn't have SDL_WINDOW_RESIZABLE set.

Thread 0:: CrBrowserMain Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x00007fffbe13d34a mach_msg_trap + 10
1 libsystem_kernel.dylib 0x00007fffbe13c797 mach_msg + 55
2 com.apple.CoreFoundation 0x00007fffa889d434 __CFRunLoopServiceMachPort + 212
3 com.apple.CoreFoundation 0x00007fffa889c8c1 __CFRunLoopRun + 1361
4 com.apple.CoreFoundation 0x00007fffa889c114 CFRunLoopRunSpecific + 420
5 com.apple.HIToolbox 0x00007fffa7dfdebc RunCurrentEventLoopInMode + 240
6 com.apple.HIToolbox 0x00007fffa7dfdcf1 ReceiveNextEventCommon + 432
7 com.apple.HIToolbox 0x00007fffa7dfdb26 _BlockUntilNextEventMatchingListInModeWithFilter + 71
8 com.apple.AppKit 0x00007fffa6396a54 _DPSNextEvent + 1120
9 com.apple.AppKit 0x00007fffa6b127ee -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
10 com.apple.AppKit 0x00007fffa66f568d +[NSWindow(NSWindowResizing) _mouseHysteresisCheck:withExpiration:andDistance:finalMouseLocation:] + 525
11 com.apple.AppKit 0x00007fffa65eedb5 -[NSWindow(NSWindowResizing) _hitTestWithHysteresisCheck:forEvent:allowWindowDragging:] + 394
12 com.apple.AppKit 0x00007fffa6c8f0db -[NSWindow(NSEventRouting) _handleMouseDownEvent:isDelayedEvent:] + 1873
13 com.apple.AppKit 0x00007fffa6c8ca6c -[NSWindow(NSEventRouting) _reallySendEvent:isDelayedEvent:] + 1942
14 com.apple.AppKit 0x00007fffa6c8bf0a -[NSWindow(NSEventRouting) sendEvent:] + 541
15 org.libsdl.SDL2 0x000000010d46d74a -[SDLWindow sendEvent:] + 90
16 com.apple.AppKit 0x00007fffa6b10681 -[NSApplication(NSEvent) sendEvent:] + 1145
17 org.libsdl.SDL2 0x000000010d46532b -[SDLApplication sendEvent:] + 139
18 org.libsdl.SDL2 0x000000010d466b2f Cocoa_PumpEvents + 495
19 org.libsdl.SDL2 0x000000010d44c1d5 SDL_PumpEvents_REAL + 53
20 org.libsdl.SDL2 0x000000010d44c2f5 SDL_WaitEventTimeout_REAL + 53
21 org.libsdl.SDL2 0x000000010d44c2b7 SDL_PollEvent_REAL + 23
22 org.libsdl.SDL2 0x000000010d51bb24 SDL_PollEvent + 36
23 libTest.dylib 0x000000010cf3e0e8 SDLEventProcessor::processEvents(int) + 568
24 Test 0x000000010cde6bba BrowserApp::RunAppMessageLoop(BAInstData*, CefStringBase, CefStringBase) + 810
25 Test 0x000000010ce04bbc main + 17980
26 libdyld.dylib 0x00007fffbe016235 start + 1

I further noticed that while entering full screen in SDL_cocoawindow.m NSResizableWindowMask is set. If I clear it inside windowDidEnterFullScreen, then, the issue doesn't repro.

This is discussed at https://discourse.libsdl.org/t/main-thread-gets-stuck-on-left-mouse-down/22753/3 and thanks to Eric for the pointers.

src/video/cocoa/SDL_cocoawindow.m

@@ -674,6 +674,7 @@ SetWindowStyle(SDL_Window * window, NSUInteger style)
         pendingWindowOperation = PENDING_OPERATION_NONE;
         [self setFullscreenSpace:NO];
     } else {
+        SetWindowStyle(window, 0);
         if ((window->flags & SDL_WINDOW_FULLSCREEN_DESKTOP) == SDL_WINDOW_FULLSCREEN_DESKTOP) {
             [NSMenu setMenuBarVisible:NO];
         }