Streaming a <canvas> to LEDs

A detailed guide on how to stream an HTML <canvas> element to an RGB LED matrix using Node.js and a Raspberry Pi.

Introduction

I've been having a lot of fun with these RGB LED matrix panels. I'm using them to display real-time stats, notifications, pixel art, and video games. They have a unique low-resolution style that you can't achieve with a standard display.

This project might seem a little daunting to those who haven't dived into the hardware world. I wanted to make it more accessible to web devs by creating the node-pixel-pusher module.

If you can draw on a <canvas>, you can display it on LEDs!

We'll be using rpi-rgb-led-matrix under the hood. It has very detailed documentation, so please refer to that for extra details.

Parts

• Raspberry Pi 4 Model B
  • Raspberry Pi USB-C Power Supply
  • 16GB microSD Card
• Electrodragon Active-3 Breakout Board
• RGB LED Matrix Panel (64x64 - 2.5mm Pitch - 1/32 Scan)
  • IDC Ribbon Cable (Hub75 / 2x8 IDC Cable)
  • Power Cable (4-pin 3.96mm pitch)
• 5V Power Supply

That's my exact setup, but you can:

• Use a different panel size
• Use a different Pi (Supported Raspberry Pi versions)
• Use a different Breakout board or an Adafruit HAT/Bonnet
• Wire by hand without a breakout board (Wiring guide)

Installing DietPi

First, let's flash DietPi to the microSD card. DietPi is a lightweight, non-GUI, distribution for the Raspberry Pi. This is ideal to cut CPU usage and interference with the LEDs.

1. Download the DietPi image for Raspberry Pi
2. Extract the DietPi .7z archive to get the .img file
3. Download Etcher
4. Run Etcher
5. Select the DietPi .img file
6. Select the drive of your SD card
7. Click Flash!

For WiFi configuration and troubleshooting please refer to the DietPi Getting Started guide.

Preparing the Hardware

1. Make sure the power is off while you're assembling everything
2. Raspberry Pi:
   1. Insert the microSD card
   2. Add the breakout board
   3. Connect the ethernet cable
   4. Connect the USB-C power cable
3. Connect the breakout board to the panel's input socket with a ribbon cable
4. Connect the power cable to the panel

Adding Extra Panels

You can add more panels by daisy chaining them with the ribbon cables. Connect the output socket of one panel to the input socket of the next.

Adding more panels will slow down the refresh rate. Try to use 2 or 3 parallel chains from the breakout board to achieve a faster refresh rate. Each parallel chain must contain the same number of panels.

Preparing the Pi Software

ssh root@192.168.0.100

Go through the basic DietPi setup. When you reach the menu with multiple options, select Install at the bottom without additional software.

When you're in, let's get everything installed:

apt install git make build-essential

git clone --recursive https://github.com/jmswrnr/rpi-matrix-pixelpusher.git
cd rpi-matrix-pixelpusher

make

Running the Pi Server

Now we have everything installed on the Pi, we are able to run the PixelPusher server!

./pixel-push \
  --led-show-refresh \
  --led-rows=64 \
  --led-cols=64 \
  --led-parallel=2 \
  --led-chain=2 \
  --led-slowdown-gpio=2 \
  -u 65507

Client Examples

Now the Pi is now running the PixelPusher server, we can draw to it using Node.js on our dev machine. You can find examples used in this guide on GitHub.

# Run this on your dev machine, not the Pi
git clone https://github.com/jmswrnr/canvas-led-examples
cd canvas-led-examples

Node.js Client

This example uses Node.js and node-canvas. This is a great way to render if you only need 2D rendering without a GUI or preview.

This canvas implementation doesn't support WebGL. See the Compatibility Status page for more details.

cd node
npm install
npm run start

After running this example, you should see all your LEDs light up green!

Let's take a look into how this works:

const PixelPusher = require('node-pixel-pusher')

const service = new PixelPusher.Service()

service.on('discover', (device) => {
  createRenderer(device)
})

const nodeCanvas = require('canvas')

const MAX_FPS = 30

function createRenderer(device) {
  const width = device.deviceData.pixelsPerStrip
  const height = device.deviceData.numberStrips
  const canvas = nodeCanvas.createCanvas(width, height)
  const ctx = canvas.getContext('2d')

  console.log(`Creating renderer ${width}x${height} ${MAX_FPS}fps`)

  device.startRendering(() => {
    // Render
    ctx.fillStyle = 'green'
    ctx.fillRect(0, 0, width, height)

    // Get data
    const ImageData = ctx.getImageData(0, 0, width, height)

    // Send data to LEDs
    device.setRGBABuffer(ImageData.data)
  }, MAX_FPS)
}

Electron Client

This example also renders everything green, but using Electron instead of only Node.js. This gives us Chromium performance, a GUI, and WebGL support!

cd electron
npm install
npm run start

We can now preview the render on a <canvas> in the Electron window:

const mainWindow = new BrowserWindow({
  webPreferences: {
    nodeIntegration: true,
    backgroundThrottling: false
  }
})

mainWindow.loadFile('index.html')

function createRenderer(device) {
  const width = device.deviceData.pixelsPerStrip
  const height = device.deviceData.numberStrips
  const canvas = document.createElement('canvas')
  canvas.width = width
  canvas.height = height
  document.body.appendChild(canvas)
  const ctx = canvas.getContext('2d')

  console.log(`Creating renderer ${width}x${height} ${MAX_FPS}fps`)

  device.startRendering(() => {
    // Render
    ctx.fillStyle = 'green'
    ctx.fillRect(0, 0, width, height)

    // Get data
    const ImageData = ctx.getImageData(0, 0, width, height)

    // Send data to LEDs
    device.setRGBABuffer(ImageData.data)
  }, MAX_FPS)
}

Three.js Client

Now we're using Electron and have WebGL support, we can also render using Three.js!

cd electron-threejs
npm install
npm run start

For this example, I'm using 4 panels connected with 2 parallel chains:

Let's see how we can integrate the Three.js renderer:

const renderer = new THREE.WebGLRenderer()
document.body.appendChild(renderer.domElement)
renderer.setSize(width, height)
const offscreen = new OffscreenCanvas(width, height)
const offscreenCtx = offscreen.getContext('2d')

device.startRendering(() => {
  // Render
  renderer.render(mainScene, camera)

  // Get data
  offscreenCtx.drawImage(renderer.domElement, 0, 0)
  const ImageData = offscreenCtx.getImageData(0, 0, width, height)

  // Send data to LEDs
  device.setRGBABuffer(ImageData.data)
}, MAX_FPS)

Here's an alternative render method if you want to dive a bit deeper:

const renderer = new THREE.WebGLRenderer()
renderer.setSize(width, height)
const target = new THREE.WebGLRenderTarget(width, height)
renderer.setRenderTarget(target)
const buffer = new Uint8Array(width * height * 4)

device.startRendering(() => {
  // Render
  renderer.render(mainScene, camera)

  // Get data
  renderer.readRenderTargetPixels(target, 0, 0, width, height, buffer)

  // Send data to LEDs
  device.setRGBABuffer(buffer)
}, MAX_FPS)

Conclusion

This can be the start of an awesome project and I hope this guide inspires you to build something with it!

To get you started, here are some of my random project ideas:

• 📈 Real-time metrics display for the office
• 🖼️ Poster/picture frame for the house
• ✨ Low-resolution magic mirror
• 🧱 Cover a wall or desktop
• 🕰️ Desktop widgets

I'd love to see anything you make with this, so please share it with me if you do! If you have any questions please let me know on Twitter or stop by my Twitch live stream where I'm often coding and answering questions!

My personal projects with these panels include a Deskmat and an Interactive Twitch Display. I plan on writing separate case studies for these projects, so I won't go into detail here.

But here's Conway's Game of Life on my Deskmat: