Just in case you didn’t see it already: SmartMatrix Library ESP32 Port
I agree there are too many form factors for ESP32 boards. I probably won’t be releasing a shield for ESP32, at least not anytime soon, but everything I’ve done on the project is open source and at least partially working