Rp2040 tof

Softening Hardware with Modular-Things

use virtualization to easily write programs that run across multiple hardware devices

The Modular-Things project is an approach to building modular hardware systems which can be composed in software. The underlying technology is a small networking library for packetizing and routing messages across devices (OSAP) and a web-based editor for composing that hardware into functional systems in software.

Stated simply our tools let you take hardware devices and add a library to them which allows you to plug that device into a browser-based editor where you can write programs that control it and other devices. Because it's JavaScript in the browser you can also write interfaces in HTML/CSS/JS and import various high-level libraries which interact with and control your physical computing system.

For a succinct introduction (3 min) check out our video for CHI2023 by clicking below.

An example Modular Things construction kit can be seen below with circuits available here.

Project Background

Modular-Things is written by Quentin Bolsee, Leo McElroy and Jake Read based on ongoing work at the MIT Center for Bits and Atoms and Hack Club.

It is based on the surprising performance of modern USB, an insight that we picked up from the urumbu project (paper). Under the hood, it uses osap to route packets and serialize data. Also inspired by the virtual-machine architecture pioneered by Nadya Peek and Ilan Moyer, way back in the way back.

Installation and Usage

You can use the latest modular-things by navigating to https://modular-things.com/, or follow these instructions to run it locally:

• install node.js - please use version 16.15.1 or above
• clone this repo
• run npm install within this repo
• run npm run dev to startup a local server !

Loading Local Files

...modular-things/?file=yourFile.js&panel=devices&panelWidth=70

where file is at repo/examples/

Writing New Modular-Things

For embedded (arduino) codes, install the OSAP library, which you can download as a zip here: https://github.com/jakeread/osap-arduino, and should also be available via arduino's library manager.

Also, install the FlashStorage_SAMD library, via the library manager.

We use the ArduinoCore-fab-sam for these circuits, which you can install into Arduino via the notes in that repo. For the RP2040 XIAO boards, we use Earle F. Philhower, the Third's PICO build, and for D21 XIAOs, you can follow the XIAO docs.

Using the Automatic RPC

As of osap 0.5.0, we can use an improved design pattern to build devices. At this time, this will only work on RP2040 based devices (which use a more modern compiler), but it is possible to back port this to c++11 based devices (most Arduinos).

The basic pattern is pretty simple, we have a function:

// types include float, String, int, and boolean 
float someFunction(String oneArgument, int anotherArgument){
  // ... your function body 
}

And we include it as a method in the devices' software object with this one-liner, handing it the function poniter, and the names of the function and its arguments:

OSAP_Port_RPC<decltype(&someFunction)> rpcMuleTwo(&someFunction, "someFunction", "oneArgument, anotherArgument");

Once we do that, and compile and load the code, the module should appear automatically when it's plugged into modular-things.com/

Below is a full example of a device that uses these Automatic RPCs.

#include <osap.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>


// display setup, which is wired to the XIAO's labelled SCL/SDA pins 
#define SCREEN_WIDTH 128 // OLED display width, in pixels
#define SCREEN_HEIGHT 64 // OLED display height, in pixels
#define SCREEN_ADDRESS 0x3C
Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT);


// a potentiometer and a button 
// the pot is on XIAO D2/A2, which is GPIO num 28
// the button is on XIAO D3, which is GPIO num 29 (and will need a pullup)
#define PIN_POTENTIOMETER 28 
#define PIN_BUTTON 29 
#define PIN_LED_BLINK PIN_LED_R 
#define PIN_LED_WD PIN_LED_G 


// we instantiate an OSAP runtime, give it a serialport and a name: 
OSAP_Runtime osap;
OSAP_Gateway_USBSerial serLink(&Serial);

Aligning Two Tiny Programs

To explain how module authorship works, we've just tidied up the rgbb example, which you can find here (arduino) and here (javascript).

Basically: we have an Arduino code that we give a "typeName" - that is then discovered by OSAP via the Modular-Things client. Modular-Things then looks for a matching JS or TS constructor, and if it finds one, instantiates it and passes in the devices' unique name (a string identifier).

The constructed object can then use that string ID, as well as individual port string IDs, to route messages to its paired object.

The result is that the "virtual thing" (javascript) acts as a proxy for the arduino-thing, making for cleaner programming model when multiple things exist in the same system.

#include <osap.h>

// -------------------------- Define Pins for R,G and B LEDs, and one Button

#define PIN_R 14
#define PIN_G 15
#define PIN_B 16
#define PIN_BUT 17

// -------------------------- Instantiate the OSAP Runtime, 

OSAP_Runtime osap;

// -------------------------- Instantiate a link layer, 
// handing OSAP the built-in Serial object to send packetized 
// data around the network 

OSAP_Gateway_USBSerial serLink(&Serial);

// -------------------------- Adding this software-defined port 
// allows remote services to find the type-name of this device (here "rgbb")
// and to give it a unique name, that will be stored after reset 

OSAP_Port_DeviceNames namePort("rgbb");

import Thing from "../../../src/lib/thing";

// the name given to us here is the "uniqueName" of the matched 
// device, we use this as a kind of address 
export default class RGBB extends Thing {
  // we can define methods that interact with the device, 
  // using the 'send' primitive, which writes data (bytes) and gets data (bytes) 
  async setRGB(r: number, g: number, b: number) {
    let datagram = new Uint8Array(3);
    datagram[0] = 255 - r * 255;
    datagram[1] = 255 - g * 255;
    datagram[2] = 255 - (b * 255) / 2;
    // to send data, we use 
    // `osap.send(name: string, targetPort: string, data: Uint8Array)`
    // ... and please use 'async' funcs with 'await' in front of 
    // network calls 
    // ... as you have probably figured out, "setRGB" here routes to 
    // the function defined around line 43 in the arduino example
    await this.send("setRGB", datagram);
  }

  async getButtonState() {
    // named-ports that return data resolve that data like-so:
    let res = await this.send("getButtonState", new Uint8Array([]));

At the time of writing, we mostly write modular-things using SAMD21 and RP2040 platforms. It is likely that OSAP will run on other microcontrollers, but there are funky-things that happen with Arduino's internal abstractions of i.e. Serial objects (which OSAP uses to button up link layers) and Flash-Storage libraries (to store names). Drop us a line if you have a favourite platform that you would like to see supported.