Having fun with RGBPanels. I temporarily built myself a 384x192 display with 9 scan row RGBPanels linked 3 levels wide and 3 rows run in parallel.
All running on an rPi4 with GitHub - hzeller/rpi-rgb-led-matrix: Controlling up to three chains of 64x64, 32x32, 16x32 or similar RGB LED displays using Raspberry Pi GPIO given that ESP32 or teensy can’t really do more than 128x64 on a single chip.
My refresh rate is just above 100Hz on an rPi4. Realistically you can’t run many more pixels than 64x384 or maybe 64x512 off a single chip, whatever your chip is, because of the limitations of RGBPanels and the required refresh to get something that doesn’t flicker to the eye (this one already flickers on camera unless you force a slow shutter)
It would be great to see a similar library for more capable chips like an fpga. Lots of people have experimented but reproducing it isn’t trivial.
I agree, I’m just not much of an FPGA person, also many FPGAs cost more than a raspberry pi 
The main issue is to go beyond this resolution by clustering chips, and then using a library on top to send data to all of them in parallel.
There 2 ways to do this
Approach #1 is what @Louis is working on: run the the panel on one side, and emulate a matrix of pixels that FastLED can talk to. If you get this, you can then use my NeoMatrix library to cluster them all into a bigger display.
Approach #2 is to use some network protocol to synchronize all those rPis and send them the frame bits they need. Afterall rPi4s support gigabit ethernet, so you could send them full frame video in chunks appropriate to each.
The issue with @Louis approach is that it won’t let you go much beyond this existing 384x192 because that’s 216KB to hold a single copy of the framebuffer.
This means you’d need a teensy 3.5/3.6 just to hold it, or an ESP32 with slow-ish PSRAM (it won’t fit in regular RAM), and if you cluster just 2 of those, you’d fill up an entire teensy 3.6 with a single copy of the double sized framebuffer.
At the tiny cost of an esp32 I’d say one per panel is perfectly sensible but an rPi per panel isn’t. So all that’s left is to find the best controller to orchestrate the division of labour with sufficient fast memory and cpu speed. A top end stm32 might be better than role a rPi for that because it’s a much simpler task and you can pick the right mix of capabilities.
How do the existing professional systems do it?
You don’t need a rPi per panel, as I just showed an rPi can run about 256x256 resolution (8 panels). An ESP32 can only run 64x128, so you’d need 8 ESP32s to match one rPi. That puts them about equal in total price.
Professional systems, AFAIK, have a bunch of FPGAs connected to each panel.
