Streaming a <canvas> to LEDs

A detailed guide on how to stream an HTML <canvas> element to an RGB LED matrix using Node.js and a Raspberry Pi.

Introduction

I've been having a lot of fun with these RGB LED matrix panels. I'm using them to display real-time stats, notifications, pixel art, and video games. They have a unique low-resolution style that you can't achieve with a standard display.

This project might seem a little daunting to those who haven't dived into the hardware world. I wanted to make it more accessible to web devs by creating the open_in_newnode-pixel-pusher module.

If you can draw on a <canvas>, you can display it on LEDs!

We'll be using open_in_newrpi-rgb-led-matrix under the hood. It has very detailed documentation, so please refer to that for extra details.

Parts

• open_in_newRaspberry Pi 4 Model B

  • Raspberry Pi USB-C Power Supply
  • 16GB microSD Card
• open_in_newElectrodragon Active-3 Breakout Board
• RGB LED Matrix Panel (64x64 - 2.5mm Pitch - 1/32 Scan)

  • IDC Ribbon Cable (Hub75 / 2x8 IDC Cable)
  • Power Cable (4-pin 3.96mm pitch)
• 5V Power Supply

That's my exact setup, but you can:

• Use a different panel size
• Use a different Pi (open_in_newSupported Raspberry Pi versions)
• Use a different open_in_newBreakout board or an open_in_newAdafruit HAT/Bonnet
• Wire by hand without a breakout board (open_in_newWiring guide)

Installing DietPi

First, let's flash open_in_newDietPi to the microSD card. DietPi is a lightweight, non-GUI, distribution for the Raspberry Pi. This is ideal to cut CPU usage and interference with the LEDs.

1. open_in_newDownload the DietPi image for Raspberry Pi
2. Extract the DietPi .7z archive to get the .img file
3. open_in_newDownload Etcher
4. Run Etcher
5. Select the DietPi .img file
6. Select the drive of your SD card
7. Click Flash!

For WiFi configuration and troubleshooting please refer to the open_in_newDietPi Getting Started guide.

Preparing the Hardware

1. Make sure the power is off while you're assembling everything
2. Raspberry Pi:

   1. Insert the microSD card
   2. Add the breakout board
   3. Connect the ethernet cable
   4. Connect the USB-C power cable
3. Connect the breakout board to the panel's input socket with a ribbon cable
4. Connect the power cable to the panel

Adding Extra Panels

You can add more panels by daisy chaining them with the ribbon cables. Connect the output socket of one panel to the input socket of the next.

Adding more panels will slow down the refresh rate. Try to use 2 or 3 parallel chains from the breakout board to achieve a faster refresh rate. Each parallel chain must contain the same number of panels.

Preparing the Pi Software

ssh root@192.168.0.100

Go through the basic DietPi setup. When you reach the menu with multiple options, select Install at the bottom without additional software.

When you're in, let's get everything installed:

apt install git make build-essential

git clone --recursive https://github.com/jmswrnr/rpi-matrix-pixelpusher.git
cd rpi-matrix-pixelpusher

make

Running the Pi Server

Now we have everything installed on the Pi, we are able to run the PixelPusher server!

./pixel-push \
  --led-show-refresh \
  --led-rows=64 \
  --led-cols=64 \
  --led-parallel=2 \
  --led-chain=2 \
  --led-slowdown-gpio=2 \
  -u 65507

Client Examples

Now the Pi is now running the PixelPusher server, we can draw to it using open_in_newNode.js on our dev machine. You can find examples used in this guide on open_in_newGitHub.

# Run this on your dev machine, not the Pi
git clone https://github.com/jmswrnr/canvas-led-examples
cd canvas-led-examples

Node.js Client

This example uses open_in_newNode.js and open_in_newnode-canvas. This is a great way to render if you only need 2D rendering without a GUI or preview.

This canvas implementation doesn't support WebGL. See the open_in_newCompatibility Status page for more details.

cd node
npm install
npm run start

After running this example, you should see all your LEDs light up green!

Let's take a look into how this works:

const PixelPusher = require('node-pixel-pusher')

const service = new PixelPusher.Service()

service.on('discover', (device) => {
  createRenderer(device)
})

const nodeCanvas = require('canvas')

const MAX_FPS = 30

function createRenderer(device) {
  const width = device.deviceData.pixelsPerStrip
  const height = device.deviceData.numberStrips
  const canvas = nodeCanvas.createCanvas(width, height)
  const ctx = canvas.getContext('2d')

  console.log(`Creating renderer ${width}x${height} ${MAX_FPS}fps`)

  device.startRendering(() => {
    // Render
    ctx.fillStyle = 'green'
    ctx.fillRect(0, 0, width, height)

    // Get data
    const ImageData = ctx.getImageData(0, 0, width, height)

    // Send data to LEDs
    device.setRGBABuffer(ImageData.data)
  }, MAX_FPS)
}

Electron Client

This example also renders everything green, but using open_in_newElectron instead of only open_in_newNode.js. This gives us open_in_newChromium performance, a GUI, and WebGL support!

cd electron
npm install
npm run start

We can now preview the render on a <canvas> in the open_in_newElectron window:

const mainWindow = new BrowserWindow({
  webPreferences: {
    nodeIntegration: true,
    backgroundThrottling: false
  }
})

mainWindow.loadFile('index.html')

function createRenderer(device) {
  const width = device.deviceData.pixelsPerStrip
  const height = device.deviceData.numberStrips
  const canvas = document.createElement('canvas')
  canvas.width = width
  canvas.height = height
  document.body.appendChild(canvas)
  const ctx = canvas.getContext('2d')

  console.log(`Creating renderer ${width}x${height} ${MAX_FPS}fps`)

  device.startRendering(() => {
    // Render
    ctx.fillStyle = 'green'
    ctx.fillRect(0, 0, width, height)

    // Get data
    const ImageData = ctx.getImageData(0, 0, width, height)

    // Send data to LEDs
    device.setRGBABuffer(ImageData.data)
  }, MAX_FPS)
}

Three.js Client

Now we're using open_in_newElectron and have WebGL support, we can also render using open_in_newThree.js!

cd electron-threejs
npm install
npm run start

For this example, I'm using 4 panels connected with 2 parallel chains:

Let's see how we can integrate the open_in_newThree.js renderer:

const renderer = new THREE.WebGLRenderer()
document.body.appendChild(renderer.domElement)
renderer.setSize(width, height)
const offscreen = new OffscreenCanvas(width, height)
const offscreenCtx = offscreen.getContext('2d')

device.startRendering(() => {
  // Render
  renderer.render(mainScene, camera)

  // Get data
  offscreenCtx.drawImage(renderer.domElement, 0, 0)
  const ImageData = offscreenCtx.getImageData(0, 0, width, height)

  // Send data to LEDs
  device.setRGBABuffer(ImageData.data)
}, MAX_FPS)

Here's an alternative render method if you want to dive a bit deeper:

const renderer = new THREE.WebGLRenderer()
renderer.setSize(width, height)
const target = new THREE.WebGLRenderTarget(width, height)
renderer.setRenderTarget(target)
const buffer = new Uint8Array(width * height * 4)

device.startRendering(() => {
  // Render
  renderer.render(mainScene, camera)

  // Get data
  renderer.readRenderTargetPixels(target, 0, 0, width, height, buffer)

  // Send data to LEDs
  device.setRGBABuffer(buffer)
}, MAX_FPS)

Conclusion

This can be the start of an awesome project and I hope this guide inspires you to build something with it!

To get you started, here are some of my random project ideas:

• 📈 Real-time metrics display for the office
• 🖼️ Poster/picture frame for the house
• ✨ Low-resolution magic mirror
• 🧱 Cover a wall or desktop
• 🕰️ Desktop widgets

I'd love to see anything you make with this, so please share it with me if you do! If you have any questions please let me know on open_in_newTwitter or stop by my open_in_newTwitch live stream where I'm often coding and answering questions!

My personal projects with these panels include a Deskmat and an Interactive Twitch Display. I plan on writing separate case studies for these projects, so I won't go into detail here.

But here's Conway's Game of Life on my Deskmat: