I’m sorry to hear about these issues. It’s hard to know what could be causing the lockups after a few hours. If I had to guess, it might be from getting unexpected behavior or data from the SD card, which the SD card or GIF decoder library can’t handle. There are some stability improvements in SmartMatrix 3.0, the latest release, which might be worth switching to if you are able.
Are you able to hook up a USB cable to one of the units that won’t play GIFs at all? The Teensy is outputting debug information over USB-Serial while running the AnimatedGIFs program. That could help track it down.
What version of the SmartMatrix Shield are you using? Version 3 adds level shifting buffers on all the signals going to the panel for better compatibility with some panels that require 5V signals. It’s possible but unlikely that your panel requires 5V, but worked ok at 3.3V for a while and now doesn’t.
Are you able to swap the SmartMatrix Shield from one panel to another, to try to isolate the issue?