Hi, SmartMatrix Library uses a low priority non-blocking interrupt for doing all the number crunching in the background to prepare the data to shift out to the panels. This means that about roughly half the time your sketch is blocked, but only for about a millisecond at a time. That might be fast enough for you to look for a GPIO inside loop() and respond to it, but I’m not sure.
Higher priority ISRs aren’t blocked by the SmartMatrix Library, so if you can get the button press events from an ISR, that would be even better.
I plan to read bitmaps from the SD card
You can try to modify the Bitmaps example sketch to read and decode bitmaps from the SD card, and write them to the backgroundLayer just like the example is doing. Or do a Google search and see if someone else has already made a sketch, like this one (this may need to be brought up to date with SmartMatrix Library 3, see MIGRATION in the SmartMatrix Library folder: https://github.com/jasoncoon/Teensy32x32MatrixSlideshow