Working around sync offsets and flicker when filming/taking pictures

marcmerlin · March 25, 2019, 3:02pm

@Louis Thanks for explaining the flip-flop.
Maybe we can convince @Jason to make a new version of his ESP32 shield that includes a flip-flop. Until then, it would be too much weird soldering on top of his board, so I think I’ll continue without it.
Looking forward to an ESP32 smartmatrix shield, though

Yes, what saved me is that the room is dark and I’m very bright, so the camera switches to a longer shutter speed, giving the time to the display to update enough that scanning is not visible.

I still need to check @seratosteve 's fork when I get a chance, unless you get to it first. It’d be great if it were a regular github fork though, easier to diff

marcmerlin · March 25, 2019, 9:32pm

@seratosteve, I made another SM fork and imported your code so that the diff is easier to see: Added SCAN_BY_BITPLANE from Steve West. · MarcFork/SmartMatrix@fdceb17 · GitHub
Steve, would you be ok forking my branch and doing development in your own github so that we can keep track of your history and changes instead of doing unzip/diff/reverse engineered github checkins?

@Louis I also wrote my own ESP32 PINOUT compatible with @Jason’s ESP32 16 channel level shifter shield. Would you accept a patch to merge that in your teensylc branch?

seratosteve · March 25, 2019, 9:45pm

@marcmerlin sure thing

Louis · March 25, 2019, 11:19pm

Thanks for the GitHub fork @marcmerlin (and for the code @seratosteve). I tried it here and sure enough, I don’t see scanning when my iPhone 6’s slow motion video camera mode is pointed at the panel. I do see that lines 0 and 16 on my panel seem to be out of place. With this gradient bitmap showing, it looks to me like what’s on line 0 should be on line 15, and what’s on line 16 should be on line 31.

I’m seeing a ton of the green ghosting issue when running the MultipleTextLayers example sketch. That’s one of the main reasons I moved away from doing frame-based refresh toward row-based refresh. Without a good fix for this issue, I don’t see frame-based refreshing working well for content that has a lot of blank pixels.

One idea: maybe have some really dim color applied to the background, so no pixels are actually blank, and the charged up row can discharge. Another idea: skip driving OE on rows that don’t have any content, so they aren’t actually lit up. (Probably bad idea, because we’re lighting up at least two rows on these panels, more on something like a P10/4 panel)

Louis · March 25, 2019, 11:36pm

I tried the idea of filling the screen with some dim value. Doesn’t help much. I added a backgroundLayer filled with {0x10, 0x10, 0x10} (smallest visible power of 2 gray with the default settings), and still see significant green streaks.

I’m also seeing another issue here, the text isn’t crisp, it’s bleeding into the row above:

I’m not sure this is the same issue as the green streaks issue. It’s matching the same color as the text, and bleeding into the row above, and only the row above, where the green streaks issue sometimes travels across several rows and in the bottom direction (and is green only).

Gut feel says the issue with the text bleeding into the row above is just a bug that can be fixed with code. The green streaks is a more fundamental hardware issue that shows up when switching rows a lot, like you do with frame based refresh.

marcmerlin · March 25, 2019, 11:43pm

Regarding this bit, this is the wiring I made for @Jason’s board:

github.com

marcmerlin/SmartMatrix/blob/teensylc/src/MatrixHardware_ESP32_V0.h#L63


      
          #define BIT_G2  (1<<4)   
          #define BIT_B2  (1<<5)   
          
          // Control Signals
          #define BIT_LAT (1<<6) 
          #define BIT_OE  (1<<7)  
          
          #define BIT_A (1<<8)    
          #define BIT_B (1<<9)    
          #define BIT_C (1<<10)   
          #define BIT_D (1<<11)   
          #define BIT_E (1<<12)   
          
          
          #if (GPIOPINOUT == ESP32_JC_RIBBON_PINOUT)
              #pragma message "Jason Coon ESP32 NodeMCU shield wiring"
          // This pinout takes a ribbon cable and flattens it, pin order is 1, 9, 2, 10 ...
          // it connects to https://www.tindie.com/products/jasoncoon/16-output-nodemcu-esp32-wifi-ble-led-controller/
          // https://cdn.tindiemedia.com/images/resize/byvBO67Uofqr3gmoYuaBRB-dLXE=/p/fit-in/1370x912/filters:fill(fff)/i/13194/products/2018-10-27T20%3A22%3A44.751Z-IMG_20181026_082016725.jpg
          // *** WARNING, I cut the trace on Jason's board that went to pin 3, and patched a wire
          // to pin 27 so that I can use RX/TX serial debugging ****

Diff wise, it’s:

plus

(I had to rework @Jason 's board to redirect pin 3 which I used for serial debugging to unused pin 27)

Louis · March 25, 2019, 11:43pm

It’s just a change to the ESPhardware.h file, right? Should be fine

marcmerlin · March 25, 2019, 11:46pm

Correct just a change to MatrixHardware_ESP32_V0.h as per the 2 diffs I sent you above, and this comment one I just added

seratosteve · March 25, 2019, 11:56pm

Louis are you using an external row latch? My code doesn’t support that.

Louis · March 26, 2019, 2:09pm

Nope, I’m using the GPIOPINOUT SMARTLED_SHIELD_V0_PINOUT option. I’m using your fork with the only one change is the setting GPIOPINOUT ESP32_FORUM_PINOUT

When I go back to the main teensylc branch, with the same GPIOPINOUT ESP32_FORUM_PINOUT setting, the scrolling text is crisp.

marcmerlin · March 28, 2019, 4:56am

Added ESP32_JC_RIBBON_PINOUT for Jason Coon's ESP32 level shifter board by marcmerlin · Pull Request #71 · pixelmatix/SmartMatrix · GitHub takes care of adding Jason Coon’s wiring.
If you can accept that, then @seratosteve can send a PR to add his own wiring if need be.
@seratosteve is there a reason why you made your own ESP32_D1R32_PINOUT when ESP32_FORUM_PINOUT already existed?
In my case I did use ESP32_FORUM_PINOUT using I switched to @Jason’s premade board and I was forced to use that wiring, hence the PR above, but otherwise on a breadboard, I did use ESP32_FORUM_PINOUT

seratosteve · March 29, 2019, 7:57pm

Marc, I didn’t even look at the ESP32_FORUM_PINOUT wiring, I just connected up the wires the simplest way I could.

Louis, your bad results could be explained if you’re using loadMatrixBuffers48. My prototype method is only implemented in loadMatrixBuffers24.

marcmerlin · March 29, 2019, 8:14pm

Cool to see that you’re getting away with not using level shifters
The one thing that sucks with ESP32 is how many wiring packages there are. It’s between hard and impossible to get a wiring that works for all.

Louis · March 29, 2019, 11:41pm

Got it. I recompiled the MultipleTextLayers sketch with kRefreshDepth = 24 and it looks better, but there are still plenty of visible issues:

@seratosteve are you seeing these in your setup, or do I still have some settings that don’t match yours?

Louis · April 2, 2019, 4:23pm

OK, the green streaks issue is very panel dependent. I only tried on two panels here, and both had the issue, and I incorrectly assumed all panels had the issue. After @marcmerlin showed what it looked like on his panels I tried again on a much wider selection of panels, and it’s either very visible, or not an issue at all, depending on the panel. I haven’t traced it down to a possible corrolation with chipsets used on the panel yet, but I did notice it wasn’t an issue on the more finer pitch panels, which are also panels I purchased more recently than the rest (so maybe they have newer chipsets?).

I’m more open to the idea of supporting this as an optional feature now. It has to be optional because in my opinion it looks terrible (depending on the content displayed) on panels that leave green streaks. I’m not planning on spending much time on this now though.

marcmerlin · April 2, 2019, 6:54pm

Sounds good, thanks.
Note that while it doesn’t improve the memory use at all for me, and I may be ok enough with 120Hz for mostly dark/night use where cameras go into slower shutter mode anyway, his change looks like a worthwhile improvement, at least as an option.
The patch is also reasonably minimal.

CoreyCoop · April 3, 2021, 3:28am

Has any more work been done on this issue? I’m looking at using a Teensy 4.1 on 32x64 displays and having them reshot on broadcast TV as a score-board sort of arrangement. I found taking a still with my camera at 1/320 or less produced a good image, and interpreted that to mean a video camera at 60fps would be fine. After reading this, I took a video with my phone, and found problems. I don’t know why I didn’t do this before.

Louis · April 6, 2021, 9:48am

The video for the SmartLED Shield for Teensy 4 Crowd Supply campaign was shot with an iPhone 11 Pro at 30/60 FPS. I think I bumped up the refresh rate to 249Hz (default 240 on the Teensy 4), as that avoided rolling blank lines. If you’re still having issues after fine tuning the refresh rate, post a video and your settings

MStirlingC · May 8, 2024, 2:14pm

Hi, Steve. I know this is an old post. I’m super interested in being able to video my 64x64 display outdoors using your flicker-reducing library. I’m using an ESP32 with the “forum pin-out”.

Do you recall which of the library files you needed to alter in order to do the flicker reduction? If I can just replace those individual files, it will make it easier for me to integrate.

MStirlingC · May 8, 2024, 2:37pm

These were the files (I think) @marcmerlin showed as being changed in his github branch:
MatrixHardware_ESP32_V0.h
SmartMatrixMultiplexedCalcEsp32.h
SmartMatrixMultiplexedCalcEsp32_Impl.h
SmartMatrixMultiplexedRefreshEsp32.h
SmartMatrixMultiplexedRefreshEsp32_Impl.h
esp32_i2s_parallel.c
esp32_i2s_parallel.h