The standard SPI port (pins 10-13) is blocked by the HUB75 panel driver. There was no way around that. But there are two more hardware SPI ports on Teensy 4.0 and 4.1 and the driver and shield do not interfere with them. SPI1 uses pins 0,1,26,27 which are free. And if you are not using an SD card, there is also SPI2 which uses the DATA1, DATA0, CLK, and CMD pins of the SD card interface. These are numbered as pins 34-37 on Teensy 4.0, renumbered as pins 42-45 on Teensy 4.1.
On Teensy 4.0, you will need to carefully solder wires to the very small SMT pads on the bottom side to access pins 26-27 and 34-37.
On Teensy 4.1, pins 0,1,26,27 are easy to access on the external headers. Pins 42-45 are only available through the SD card connector so you would need something like this Sparkfun microSD breakout to access them.
Check the “Teensy4 Pins” spreadsheet here for more details about the pin assignments: GitHub - KurtE/TeensyDocuments: Some of my own Teenys documents such as XLS documents showing pin assignments and the like