When you write the number to the screen, you replace the GIF frame data, so it’s lost. You can draw the same number of characters each time, and use the drawString() option to clear the background. You’ll have a solid color behind the text, which you may not want:
void SmartMatrix::drawString(int16_t x, int16_t y, const rgb24& charColor, const rgb24& backColor, const char text)
You could use repeated calls to getPixel() to read the GIF frame data from behind the text, store it somewhere, then put it back when you’re done with drawPixel().