Flickering display when dimming

I display a flag with 3 rectangles on my 16x32 (custom) display. All is nice and steady until I try to dim the display where it starts to flicker and the more the dimming, the more the flicker. Not a real big problem since I need as much brightness as possible, but thinking about the future when I might need it (like smooth scrolling text).
Any ideas? Is it my choice of components (DM13 latches, 74540 drivers) slowing everything down? Teensy 3.2 running at 92MHz.

If you can see flickering then your refresh rate is too low. Try printing it with Serial.println(matrix.getRefreshRate());.

One of my favourite features of SmartMatrix is how tunable it is. Try changing kMatrixRefreshDepth in your sketch. Reduce it to 24, or any multiple of 3. This is a trade-off between refresh rate and how many colours can be displayed.