SmartMatrix Library ESP32 Port

Ok, thank you @Louis

I have edited the code to show the single mallocs and it looks like there is no block large enough for the second matrixUpdateFrames malloc.

Starting SmartMatrix DMA Mallocs
Step 0: Heap Memory Available: 260076 bytes total, 113792 bytes largest free block: 
Step 0: 8-bit Accessible Memory Available: 167400 bytes total, 113792 bytes largest free block: 
Step 0: 32-bit Memory Available: 260076 bytes total, 113792 bytes largest free block: 
Try to allocate: 65536 bytes... Done.

Step 1: Heap Memory Available: 194524 bytes total, 92676 bytes largest free block: 
Step 1: 8-bit Accessible Memory Available: 101848 bytes total, 48240 bytes largest free block: 
Step 1: 32-bit Memory Available: 194524 bytes total, 92676 bytes largest free block: 
Try to allocate: 65536 bytes... 
assertion "matrixUpdateFrames[1] != NULL" failed: file "lib\SmartMatrix-teensylc\src/SmartMatrixMultiplexedRefreshEsp32_Impl.h"

It already uses kRefreshDepth = 24. 65536 bytes would be 8 bytes per pixel. Is that correct?

I will try it with the new code soon.

65536 bytes would be 8 bytes per pixel. Is that correct?

Yes, that’s correct. I believe on the ESP32 the way I implemented the code you need one byte of RAM per pixel times the number of bits of colordepth (kRefreshDepth/3).

@peterpoetzi @Louis just curious: does ESP32 work with RGBpanels without level shifters, or level shifters are required? Never mind, I found the answer in the README I hadn’t seen (well I had been reading the wrong one that is). The answer is “it depends” (on the panel you’re talking to).

Now, I’m reading the correct README: (BTW, it’s a bit confusing to check out a teensylc branch to get ESP32 support, worth renaming?).
For pinout, I can’t use the teensy shield, so I’m direct wiring with level shifters I put on a proto board (is there a better way?)
I looked at ./src/MatrixHardware_ESP32_V0.h and I see 14 pins defined

#define R1_PIN  2
#define G1_PIN  15
#define B1_PIN  4
#define R2_PIN  16
#define G2_PIN  27
#define B2_PIN  17

#define A_PIN   5
#define B_PIN   18
#define C_PIN   19
#define D_PIN   21
#define E_PIN   12    // is that pin #8 on the 16 pin connector?
#define LAT_PIN 26
#define OE_PIN  25
#define CLK_PIN 22 shows pin mapping and I think E_PIN is #8 labelled as GND there, correct?

I’m confused by this text in the README though:
“The 5x ADDX lines are output using the RGB data lines and stored using an external latch, freeing up more pins on the ESP32
With the addition of the external latch, there are only 8 bits of data to output via I2S, and so each clock cycle’s data fits into a uint8_t instead of uint16_t. With the I2S peripheral in 8-bit mode instead of 16-bit mode, the amount of RAM used to store refresh buffers is cut in half”

I don’t have the external latch since I’m not using the shield (there is no premade shield for ESP32, is there? I see the reference to eagle files but I don’t have a shield that was actually made by someone and I think right now it’s fab your own?)
Does it mean that if I direct wire the 14 lines, the driver won’t work?
Or do I just need to
(no idea what shield/wiring #define GPIOPINOUT SMARTLED_SHIELD_V0_PINOUT refers to)


The schematic mentioned in the README shows you how to wire it up. If you don’t want to follow the schematic and use the latch, then change the pinout definition of GPIOPINOUT in the MatrixHardware_ESP32_V0.h header to ESP32_FORUM_PINOUT - this is also mentioned in the README.

I think E_PIN is #8 labelled as GND there, correct

Probably, check the schematic

I think right now it’s fab your own?

Yes, or breadboard your own

no idea what shield/wiring #define GPIOPINOUT SMARTLED_SHIELD_V0_PINOUT refers to

It’s the one described in the schematic and board files included in the hardware folder

@Louis just to make sure I’m looking at the right readme, I’m currently reading
I don’t see any schematics. Am I looking at the right file?

It mentions extras/hardware where I found SmartLEDShield_ESP32_V0_sch.pdf . Is that the one?
If so, it mentions 74373PW and 74S253D chips I don’t have and can’t order today (going in a plane tomorrow).

Stepping back, can I use that branch with direct wiring from ESP32 to the panel (through a couple of level shifters I do have)?
If so, do I indeed need to
and comment out
or is direct wiring untested/not working at this time?

Thanks, Marc

By the way, to explain a bit more, I’m planning on writing code during my vacation to run RGB panel at 96x64 + a neopixel strip (requires interrupts) + an infrared input (requires interrupts).
I have teensy 3.6 working now for display only but I was hoping to bring ESP32 as backup to see if it works better with those 2 extra devices.

You’re looking at the right README.

Yes, that’s the right schematic.

It’s been a while since I used direct wiring. I believe it should still be working. If I made a change that broke it intentionally, I probably would have updated the README. Yep, use #define GPIOPINOUT ESP32_FORUM_PINOUT

See for where the pinout is documented (probably in the example code provided by Sprite_TM)

Ok, thanks for the tips.
So, the good news is that I have it working somewhat. It looks mostly ok but there are issues.
First on wiring, I only used 2 level shifters, and shifted all 16 pins (the 2 that were ground I just put on ground on the level shifters input).
The most visible thing is that on layer demo, in
SMARTMATRIX_HUB75_32ROW_MOD16SCAN layer4 and layer5 don’t display at all. It’s not a problem with colors, if I change their colors it doesn’t matter.

SMARTMATRIX_HUB75_64ROW_MOD32SCAN strangely shows all 5 layers. but layer 4 and 5 are scrolling in the wrong direction (horizontal).

Now, I don’t actually need more than one layer so it doesn’t matter, but still weird. The regular demo has ghosting on fonts that scroll but it’s not visible on my phone video. It thankfully does not flash like it does here:

Edit: I just tried 64x64 panel on ESP32 and that demo looked pretty close to normal (I don’t know it by heart, so maybe I missed a detail or two). Strangely the refresh rate demo at the end shows only 16fps, however the background task takes care of a faster refresh, so I don’t see flickering. Aaah, I see a comment in the code that changing refresh rate is broken, so I’ll ignore that number. I don’t need to change it anyway as long as I can’t see flickering.

Is there a good way to validate what works and what’s off by a bit?
I’ll try and add some capacitors on the level shifters to see if it helps.

I used this:
const uint8_t kMatrixHeight = 64; // known working: 16, 32, 48, 64
const uint8_t kRefreshDepth = 36; // known working: 24, 36, 48
const uint8_t kDmaBufferRows = 4; // known working: 2-4, use 2 to save memory, more to keep from dropping frames and automatically lowering refresh rate
//const uint8_t kPanelType = SMARTMATRIX_HUB75_32ROW_MOD16SCAN;
const uint8_t kPanelType = SMARTMATRIX_HUB75_64ROW_MOD32SCAN;

In #if (GPIOPINOUT == ESP32_FORUM_PINOUT) I added this wiring comment which you’re welcome to copy/integrate if you’d like:

                ESP32 pin / comment
1	R1	2	Red Data (columns 1-16)
2	G1	15	Green Data (columns 1-16)
3	B1	4	Blue Data (columns 1-16)
4	GND	GND	Ground
5	R2	16/RX2	Red Data (columns 17-32)
6	G2	27	Green Data (columns 17-32)
7	B2	17/TX2	Blue Data (columns 17-32)
8	E	12	Demux Input E for 64x64 panels
9	A	5	Demux Input A0
10	B	18	Demux Input A1
11	C	19	Demux Input A2
12	D	21	Demux Input E1, E3 (32x32 panels only)
13	CLK	22	LED Drivers' Clock
14	STB	26	LED Drivers' Latch
15	OE	25	LED Drivers' Output Enable
16	GND	GND	Ground

I’ll make a separate comment on development questions between teensy and ESP32.
Because teensy patches my arduino install, I usually have 2 arduinos
arduino-teensy and arduino.
They launch as separate binaries with their own windows. Sadly they share the same preferences, so I have to re-set the compile target back and forth between teensy and ESP32.

Now my bigger problem/question is with the Smartmatrix ESP32 branch in teensylc. I now have this:

saruman:~/arduino/libraries$ l -d SmartMatrix* SM_ESP32*
lrwxrwxrwx 1 merlin merlin   8 Dec 22 13:42 SmartMatrix -> SM_ESP32/
drwxr-xr-x 1 merlin merlin 166 Dec 16 18:58 SmartMatrix.orig/
drwxr-xr-x 1 merlin merlin 166 Dec 21 18:27 SM_ESP32/

Currently I move the symlink back and forth because both libraries are called the same and I don’t have a good way to switch between them, and even then I have to invalidate the arduino cache which pretty much means restarting the IDE.
I think I already compiled for ESP32 once by mistake while using the master branch and it seemed to work anyway, so I’m not even sure what’s different between the branches.

That said, my method sucks and I’d like to do something easier. Any suggestions?

Add unique header files to the SmartMatrix Library inside each instance of the SmartMatrix Library. Have the header file contain #include <SmartLEDShieldV4.h> for the Teensy one, and #include <SmartMatrix3.h> for the ESP32 one.

So in your Teensy sketch you `#include <SmartMatrix_Teensy_Unique_Header.h>

In your ESP32 sketch you `#include <SmartMatrix_ESP32_Unique_Header.h>

Good tip, I’ll try that, thanks. To be clear the teensylc/ESP32 branch is not compatible with teensy v3.x, so I can’t just use that branch for both, correct?

Hmm, some weirdness with the Layers sketch on ESP32. Not sure why.

You can try lowering the CLK rate on the ESP32 sketch. I can’t remember off hand how to do that, but there’s support for doing it.

Here is a better demo of teensy v3.6 (top) vs ESP32 direct wiring (bottom)

the demo mostly works, you can see a bit of ghosting on the white font (strangely no ghosting when I use 64x64 panel output instead of 64x32). Strangely too the first background is dark red by default with teensy, but full black with ESP32.

On the bright side, ESP32 is 99% working for me out of the box with direct wiring, so thank you for that :slight_smile:

I originally had some thoughts/questions about sync rate changing and how to not make it look so bad when filming, but I split that to a separate thread: Working around sync offsets and flicker when filming/taking pictures

Actually since I’ll be working with both the ESP32 and teensy trees: is it accurate that master and teenylc(aka ESP32) have diverged and that one is not inclusive of the other?
Or does one of the trees work on both chips and I don’t have to switch back and forth?

teensylc branch supports both but the Teensy code was extensively refactored and probably contains more bugs than the latest release.

This is a known issue (unknown solution). I noted this in the README: “Need a ~10ms delay between matrix.begin() and drawing to backgroundLayer”

Thanks for clearing up the teensylc branch. I’ll make sure only to use it for ESP32 then. And no big deal with the red background issue, I was just checking the differences between the two. All in all ESP32 worked pretty well, even with direct wiring which you haven’t used in a while.
Given that it’s unlikely that I’ll get neopixel strips working with teensy ( SmartMatrix + FastLED Neopixels + Interrupt driven IR) this is good news. I’ll try and do more work with ESP32 since it seems to work well enough.

Joyeux Noël, Marc

Just FYI, I have this up an running video, and it’s great at 64x32 (30+ fps) but it DRAGS as the matrix, and hence the data, get larger. At 64x64 its only 10-15 fps, and so on.

I think this is because I am caching the frames in PSRAM.

I also added Alpha blending and chroma keying to layers. Can I send you (Louis) a zip to review or did I need to check it into github myself? I’ve never done that…

I think it would be best for both of us if you committed your changes to GitHub, but if it’s a real barrier right now, send it to me and I can look it over.

Hi Dave, I’m going to be working on a major SmartMatrix Library release over the next month or so, so now would be a good time for me to get your changes integrated into the library. If using GitHub is too difficult, please send me a zip with anything you’d like to commit. Thanks!