Hey, welcome! This was one of the first Arduino hardware projects I tried out as well.
Teensy 3.2 is definitely supported. It's a fully compatible, drop in replacement for the 3.1.
I'd recommend the SmartMatrix v3 shield, since it has the 5v level shifters for better compatibility with more panels: https://www.adafruit.com/products/1902
Where'd you get your 32x64 panel? If it's from Adafruit, it's a pretty safe bet that it's compatible. If you can provide any more information, like a link to the tech specs, a picture of the back, etc, we can help make sure it's compatible.