Unable to load 12288 bytes from serial into BackgroundLayer

Adam_Haile 2016-08-19 13:26:25 UTC #1

I’m trying to load pixel data over Serial so that I can communicate with my display via my animation library BiblioPixel and it works great for a 32x64 panel, but when I try to up it to two of these panels for a total of 32x128 it just hangs while trying to load into the background buffer.

uint16_t count = 0;
uint8_t emptyCount = 0;
rgb24 *buffer = backgroundLayer.backBuffer();

if (size == packSize)
{
    while (count < packSize - 1)
    {
        c = Serial.readBytes(((char*)buffer) + count, packSize - count);
        if (c == 0)
        {
            emptyCount++;
            if(emptyCount > EMPTYMAX) break;
        }
        else
        {
            emptyCount = 0;
        }

        count += c;
    }
}

uint8_t resp = RETURN_CODES::SUCCESS;
if (count == packSize)
{
    backgroundLayer.swapBuffers(false);
}
else
    resp = RETURN_CODES::ERROR_SIZE;

Serial.write(resp);

The above example seems to crash the Teensy. It simply never returns. At first I thought that it was a max serial buffer size issue, but then I replaced “rgb24 *buffer” with “byte buffer[12288]” and, while it doesn’t update the actual display because I cannot figure out how to get that buffer into the BackgroundLayer (see here), it does read the data and return properly so that as far as BiblioPixel is concerned it’s sending the data. So it would seem that the issue lies in backgroundLayer or backBuffer. My guess is that the buffer returned by backBuffer is not 12288 bytes as expected and I’m overwriting into areas of memory not actually owned by the backgroundLayer buffer. Which doesn’t make a whole lot of sense since it works for half the pixel count… but that’s my best guess.

Any thoughts here?

Louis 2016-08-19 14:12:14 UTC #2

It’s probably a RAM issue. What does the compiler say for RAM used when you compile your sketch? Try reducing the SmartMatrix memory, you can change refresh depth from the default of 36 to 24, and lower the number of buffer rows to two from four.

Adam_Haile 2016-08-19 14:26:52 UTC #3

Hmm… maybe so:

Sketch uses 25,192 bytes (9%) of program storage space. Maximum is 262,144 bytes.
Global variables use 44,248 bytes (67%) of dynamic memory, leaving 21,288 bytes for local variables. Maximum is 65,536 bytes.

Yeah, that doesn’t leave a lot of wiggle room. Weird though that making my own buffer works fine, but I guess there’s other RAM overhead when actually saving to the BackgroundLayer object.

With 2 buffer rows:

Sketch uses 25,192 bytes (9%) of program storage space. Maximum is 262,144 bytes.
Global variables use 38,104 bytes (58%) of dynamic memory, leaving 27,432 bytes for local variables. Maximum is 65,536 bytes.

24 refresh depth and 2 rows:

Sketch uses 24,736 bytes (9%) of program storage space. Maximum is 262,144 bytes.
Global variables use 35,288 bytes (53%) of dynamic memory, leaving 30,248 bytes for local variables. Maximum is 65,536 bytes.

But even after that it has the same problem

Adam_Haile 2016-08-19 23:11:32 UTC #4

github.com

pixelmatix/SmartMatrix/blob/6895f94f1c136c435df42986c5e73f68e7e548f7/src/SmartMatrix3.h#L163


static uint8_t layer_name##Bitmap[width * (height / 8)];                                              \
static SMLayerScrolling<RGB_TYPE(storage_depth), scrolling_options> layer_name(layer_name##Bitmap, width, height)  


#define SMARTMATRIX_ALLOCATE_INDEXED_LAYER(layer_name, width, height, storage_depth, indexed_options) \
typedef RGB_TYPE(storage_depth) SM_RGB;                                                                 \
static uint8_t layer_name##Bitmap[2 * width * (height / 8)];                                              \
static SMLayerIndexed<RGB_TYPE(storage_depth), indexed_options> layer_name(layer_name##Bitmap, width, height)  


#define SMARTMATRIX_ALLOCATE_BACKGROUND_LAYER(layer_name, width, height, storage_depth, background_options) \
typedef RGB_TYPE(storage_depth) SM_RGB;                                                                 \
static RGB_TYPE(storage_depth) backgroundBitmap[2*width*height];                                        \
static SMLayerBackground<RGB_TYPE(storage_depth), background_options> layer_name(backgroundBitmap, width, height)  




#include "SmartMatrix_Impl.h"


#endif

Why is backgroundBitmap twice the size of the matrix resolution?

static RGB_TYPE(storage_depth) backgroundBitmap[2widthheight];

I’m writing to that buffer (acquired via backBuffer()) but I was assuming that it’s size was just width*height (and that each sub element is of course 24 bits). I’m pulling in 24 bits per pixel over serial and trying to write them directly to the backBuffer. Also, I noted that the FeatureDemo uses just as much memory, and my testing shows that I can read from Serial just fine, but as soon as I try to write that Serial data directly into the backBuffer it barfs.

Louis 2016-08-20 01:11:51 UTC #5

RGB_TYPE(storage_depth) is either rgb24 or rgb48 depending on the options chosen for SmartMatrix in your sketch. rgb24 is 3 bytes per pixel, and rgb48 is 6 bytes per pixel. What are you using for storage_depth?

Are you using malloc in your sketch? 58% of dynamic memory used doesn’t seem like a lot, but that doesn’t include any memory allocated using malloc.

Adam_Haile 2016-08-20 01:50:49 UTC #6

Storage depth? You mean color depth? 24
Here’s all I’ve got so far.

gist.github.com

https://gist.github.com/adammhaile/70e594c22055c80f6a391895b32e7566

BiblioPixelSmartMatrix.ino

#include <SmartMatrix3.h>
#include <EEPROM.h>

#define FIRMWARE_VER 2

namespace CMDTYPE
{
	enum CMDTYPE
	{
		SETUP_DATA = 1,

This file has been truncated. show original

Louis 2016-08-20 02:11:43 UTC #7

Why is backgroundBitmap twice the size of the matrix resolution?

Double buffering

I skimmed through your sketch, nothing besides this jumped out as being off, but I went through quickly.

			while (count < packSize - 1)

Shouldn’t that be while (count < packSize)? It’s late here and maybe I’m not reading the code correctly.

Adam_Haile 2016-08-20 02:47:16 UTC #8

Huh… I’m sure I had some reason to do the “packSize - 1” thing, but cannot remember why at the moment. Though it’s never been an issue in other uses. For example, a 32x64 pixel display works fine here, but it’s when I go to two 32x64 panels that it has an issue. Wish there was a better way to get some debug output in my usage… unfortunately it hard to send debug back when the serial is in use and the protocol is binary.

Louis 2016-08-20 03:02:29 UTC #9

How about using the UART and a USB-Serial cable? https://www.adafruit.com/product/954

Adam_Haile 2016-08-20 12:58:28 UTC #10

Fair enough… all my UART cables are embedded in something at the moment. Will have to see what I can take apart or order a new one.

One thing I did try that got it to work, though I’m not super happy with is to create a 12288 byte uint8_t buffer, and read into that via serial… then memcpy it to backBuffer. Updated gist: gist.github.com/adammhaile/70e594c22055c80f6a391895b32e7566

This ends up working quite nicely, though I’m sure there’s a performance hit for having to read in 12KB and then copy it. The calculated memory usage is more (that buffer is a static) but certainly not filling the RAM:

Sketch uses 24,696 bytes (9%) of program storage space. Maximum is 262,144 bytes.
Global variables use 47,568 bytes (72%) of dynamic memory, leaving 17,968 bytes for local variables. Maximum is 65,536 bytes. And note that the extra RAM used is nearly exactly the size of that new buffer (oddly, 8bytes less, must’ve freed up something elsewhere).

So, as far as I can tell, it truly is just that it crashes when trying to serial read directly into the backBuffer. Which now makes me wonder if somehow it’s an issue with the way the Serial.read() function is implemented?

I’ll try to do the UART thing soon, but this basically proves to me that it’s happening on the Serial.read() line… That should be writing into a raw rgb24* buffer which I’m assuming should be of size widthheight3 ? It has to be since it works for a smaller display. There must be something with the way Serial.read works or rgb24 and backbuffer works that I’m missing.

Louis 2016-08-24 20:42:37 UTC #11

Can you try to limit Serial.read() from writing to the end of the buffer? Maybe it’s writing too many bytes?

Can you fill the buffer with a pattern, use Serial.read() to write to part of it, and make sure that only what you expected to be filled is filled?

Adam_Haile 2016-08-27 15:53:39 UTC #12

Been trying to debug… nothing lets me write directly to the buffer that I’ve tried.
Out of curiosity I used a binary search pattern of limiting the max bytes it tries to read. I found that above 10820 bytes, it hangs on Serial.read()… or at least crashes the firmware. I’ve confirmed it never gets past Serial.read(). Trying to find if there’s something with Arduino or Teensy that is magic about 10820

Louis 2016-08-29 21:38:48 UTC #13

That’s an odd magic number, sorry I’m out of suggestions. I don’t know what Serial.read() does behind the scenes

Louis 2016-09-19 15:50:15 UTC #14

@Adam_Haile Just had a thought on this. The Teensy has two equal size SRAM arrays, I wonder if the crash is happening at the junction between two arrays. Can you add a new variable to your code, shifting the position of the buffer, and see if that magic number changes?

Do you know how to get at and interpret the .map file the linker generates?

Louis 2016-09-19 15:51:20 UTC #15

From the Freescale Reference Manual:

3.5.3.2 SRAM Arrays
The on-chip SRAM is split into two equally-sized logical arrays, SRAM_L and SRAM_U.
The on-chip RAM is implemented such that the SRAM_L and SRAM_U ranges form a contiguous block in the memory map. As such:
• SRAM_L is anchored to 0x1FFF_FFFF and occupies the space before this ending address.
• SRAM_U is anchored to 0x2000_0000 and occupies the space after this beginning address.
Valid address ranges for SRAM_L and SRAM_U are then defined as:
• SRAM_L = [0x2000_0000–(SRAM_size/2)] to 0x1FFF_FFFF • SRAM_U = 0x2000_0000 to [0x2000_0000+(SRAM_size/2)-1]

Adam_Haile 2016-09-19 16:17:33 UTC #16

Interesting… very possible that’s it. No idea how to get at the map files. But I’ll try to test this out shortly.

Louis 2016-09-19 16:35:23 UTC #17

It doesn’t look like Teensyduino enables the map file by default. Here’s how to do it:

https://forum.pjrc.com/threads/24825-when-variables-go-bad?p=39731&viewfull=1#post39731

You’ll need to modify Boards.txt inside of Arduino. I saved this modified line from a year ago when I needed a map file, but it was from an older version of Arduino/Teensyduino, hopefully it still works:

teensy31.build.flags.ld=-Wl,-Map={build.path}/output.map,--gc-sections,--relax,--defsym=__rtc_localtime={extra.time.local} "-T{build.core.path}/mk20dx256.ld"

Also, that link has another commandline tool that might help as well. Haven’t tried it myself

Adam_Haile 2016-09-26 21:49:47 UTC #18

Ok, figured out running with the map… not really sure what I’m looking for though.

gist.github.com

https://gist.github.com/adammhaile/572cd0c9a1d87a5267b4938e05807ca3

elf_nm_output.txt

         w __deregister_frame_info
         w __fini_array_end
         w __fini_array_start
         w __libc_fini
         w __register_frame_info
         U __stack
         w _Jv_RegisterClasses
         U hardware_init_hook
         U software_init_hook
00000000 T _VectorsFlash

This file has been truncated. show original

output.map

Archive member included because of file (symbol)

C:\Users\admin\AppData\Local\Temp\buildb021674c3765f772d1e0d5e5b9c8c263.tmp/core\core.a(memcpy-armv7m.S.o)
                              C:\Users\admin\AppData\Local\Temp\buildb021674c3765f772d1e0d5e5b9c8c263.tmp\sketch\BiblioPixelSmartMatrix.ino.cpp.o (memcpy)
C:\Users\admin\AppData\Local\Temp\buildb021674c3765f772d1e0d5e5b9c8c263.tmp/core\core.a(eeprom.c.o)
                              C:\Users\admin\AppData\Local\Temp\buildb021674c3765f772d1e0d5e5b9c8c263.tmp\sketch\BiblioPixelSmartMatrix.ino.cpp.o (eeprom_initialize)
C:\Users\admin\AppData\Local\Temp\buildb021674c3765f772d1e0d5e5b9c8c263.tmp/core\core.a(mk20dx128.c.o)
                              C:\Users\admin\AppData\Local\Temp\buildb021674c3765f772d1e0d5e5b9c8c263.tmp\sketch\BiblioPixelSmartMatrix.ino.cpp.o (__cxa_guard_acquire)
C:\Users\admin\AppData\Local\Temp\buildb021674c3765f772d1e0d5e5b9c8c263.tmp/core\core.a(pins_teensy.c.o)
                              C:\Users\admin\AppData\Local\Temp\buildb021674c3765f772d1e0d5e5b9c8c263.tmp/core\core.a(mk20dx128.c.o) (rtc_set)

This file has been truncated. show original

Adam_Haile 2016-09-26 22:22:00 UTC #19

Ah, ok… yup. backgroundBitmap starts at 1fff_e5c0 and the lower memory space starts that 1fff_8000 (2000_0000 - 100000 / 2) which means that starting location of e5c0 is 26,048 bytes into the 32KB lower memory space, leaving only 6720 bytes available. Which is a little weird because before my magic number was 10820 bytes that could be read before it barfed… can’t get the math to work out for that just yet. But obviously there’s a boundry issue. Though, if SRAM_L ends at 1FFF_FFFF and SRAM_U starts at 2000_0000 they are for all purposes continguous. So I still don’t get why there’s an issue… the details your provided from the data sheet even say it’s contiguous, so it shouldn’t be a problem.

Louis 2016-09-26 23:27:56 UTC #20

Can you add a dummy buffer before backgroundBuffer to change the position of backgroundBuffer and see if the magic number shifts? You probably don’t have enough memory to force backgroundBuffer past the lower/upper memory space boundary, otherwise I’d suggest trying that. I’m not sure how to set the position of the dummy buffer to be before and not after backgroundBuffer, there’s probably a trick to that.