Skip to the content
2 layers135 × 135 mmVa5pcb, the printed circuit board in 3D

ESP32-S3 push-to-talk voice assistant using OpenAI Realtime API

Size
135 × 135 mm
Copper layers
2
Thickness
1.6 mm
Parts
16
Nets
34
Pads
119
Tracks, vias
247, 24
Pours
17
Routed
Yes
KiCad
KiCad 10.0 (file 20260206)
Created
4 May 2026
Updated
2 Nov 2026

Overview

Va5pcb is an open-source printed circuit board by iollama, from the iollama/Voice-Assistant-5 repository on GitHub, published under the GPL-3.0 licence. It is a 2-layer board of 135 × 135 mm with 16 parts on 34 nets, routed with 247 tracks and 24 vias.

It is built around the ESP32-S3, INMP441 and PCM5102APW. Its parts include 13 connectors, 2 resistors and 1 switch. It belongs with the ESP32 boards of this library.

Main components

3

README

readme.md

Voice Assistant 5 — STEM Oriented ESP32 Voice Assistant With No Personality

Press the button, talk, let go — and get an instant spoken reply, with an animated emoji face that grins, frowns and laughs along with the conversation. Less than $20 in parts, a case you print yourself, running on your own OpenAI key.

The part that makes it worth building: it has no character until you give it one.

Voice Assistant 5

Everything that gives this thing a personality lives behind a web page the device serves. The system persona prompt, one of ten voices, the spoken language, and the face itself: drop in a GIF, PNG, JPG or MP4 and it adopts immediately. No recompile, no cable, no SD card, no code. Write Nietzsche. Write Churchill. Write Daffy Duck. Hand it to someone else and let them write over it.

That design came out of a classroom. VA5 is used in schools, where students research a figure, write the persona themselves, load it on, and then have to hold a conversation with what they wrote. A thin characterisation falls apart within a minute or two of being interrogated, so they go back and rewrite it. Write, test by talking, revise — which is prompt engineering, except nobody has to call it that.

Thirty students cannot each be flashing firmware. That single constraint explains most of the design decisions in this repo: why the whole persona is editable from a phone, and why personas export as a .zip so one set of devices can cycle between classes.

Features

  • Press to talk, get an instant reply -- hold the button to speak, release and it answers right back (no wake word, no waiting)
  • Animated emoji face shows you how the assistant feels -- the assistant itself picks the mood and the display reacts in real time
  • Control the assistant personality -- write who it is and how it behaves, right from your phone, no code needed
  • Glance to know what it's doing -- a colored light tells you when it's listening, thinking, or speaking
  • The assistant adjusts its own volume -- say "speak up" or "quieter please" and it adjusts in real time; the new level persists across reboots
  • Plug in your own OpenAI key -- you control the account and the cost, no subscription in the middle
  • Make the face your own -- drop in a GIF, PNG, JPG, or MP4 from the web portal and the assistant uses it as the on-device emoji; reset back to the defaults any time
  • It remembers the conversation -- pick up where you left off instead of starting from scratch
  • Set up WiFi once, anywhere -- connect from your phone the first time; it remembers up to 6 networks and picks the right one wherever you take it
  • Build it yourself for ~$20 -- 3D-printed case included, full source open and hackable, parts links provided

Initial Setup

  1. Connect to the device's WiFi. On first boot the assistant creates an open WiFi network named VOICE-AGENT-XXYY (the XXYY is the last 2 bytes of its MAC address). No password.
  2. Set your home WiFi. Your phone should pop the captive portal automatically -- if not, open a browser and go to http://192.168.4.1. Under Wi-Fi Networks, enter your network name and password, hit Add network, then Connect & reboot. The device remembers it from then on -- and up to five more.
  3. Open the settings page. Once it's on your network, browse to http://voice-agent-XXYY.local (same XXYY as the AP name). If .local doesn't resolve on your network, use the IP address printed to the Serial Monitor.
  4. Make it your own. By default the assistant is obsessed with Pokémon and will work them into every reply -- charming for about ten minutes. Edit the System Prompt field on the settings page to give it any persona you like.

Under the Hood

A push-to-talk voice assistant built on the ESP32-S3, using the OpenAI Realtime API for speech-to-speech conversation over secure WebSockets. Features dual-core FreeRTOS architecture, animated emoji display with audio waveform visualization, a web configuration portal, and RGB LED status indicators.

Core 0 handles all network and protocol tasks (WiFi, WebSocket, Base64 encode/decode). Core 1 handles all hardware I/O (I2S microphone/speaker, button, display). The two cores communicate through PSRAM ring buffers and volatile flags.

The assistant can also call OpenAI function tools during a response — currently set_display_emotion (drives the emoji display), set_volume (adjusts speaker volume, persists across reboots), and show_network_info (shows the device IP and token count on screen).

Hardware Components

ComponentDescriptionCommentsLink
ESP32-S3 N16R8Dev board with 16 MB flash and 8 MB OPI PSRAMbuy the N16R8 varientAli, Amazon
INMP441I2S MEMS microphone-Ali, Amazon
MAX98357AI2S audio amplifier breakoutBetter to buy solderedAli, Amazon
PCM5102A (optional)I2S DAC breakout (GY-PCM5102A) with 3.5 mm stereo jackSkip if you only want the onboard speaker. Drives headphones or any powered speaker (e.g. JBL Go/Clip/Flip) via a 3.5 mm AUX cable.Ali, Amazon
Speaker1-3W 4/8-ohm or similar small speakerUp to 57mm, use foam for pressure if the speaker is too thinAli, Amazon
GC9A01Round TFT display, 240x240, SPIGet the square oneAli, Amazon
WS2812 NeoPixelRGB LED (built-in on most ESP32-S3 dev boards)Nothing to buy, it's built in-
Tactile Push Button12x12 mm momentary switch (PTT)-Ali, Amazon
Jumper WiresMale-to-male/female (for the non PCB version)-Ali, Amazon
USB-C CableData + power, Type C to Type AYou should have one at homeAli, Amazon
Breadboard400 Tie PointsYou'll only need one power railAli, Amazon
ScrewsM2 and M3 Self-Tapping ScrewsI recommend a full kitAli, Amazon

Personas

A persona is a character the device becomes: a system prompt, a voice, a language and an emoji set, bundled together. The settings page has a Browse gallery that pulls ready-made ones straight from this repo, and you can export your own as a .zip to back up, share, or hand to a class.

Shipped categories include Science (Ada Lovelace, Marie Curie, Sir Isaac Newton), Historical (Abraham Lincoln, Winston Churchill) and Philosophy (Friedrich Nietzsche, Yoda, Daffy Duck) — alongside Biblical, Theological, and a Functional category holding an ESP32 that explains its own hardware.

Writing your own is the interesting part. Vague personas produce vague conversations — the difference between "you are Winston Churchill" and a paragraph on what he knew, how he spoke, what he refused to discuss and where he was wrong is immediately audible.

Personas live in personas/. Pull requests welcome.

There is also a persona-creating skill for Google's Antigravity in .agents/skills/persona-creator/ — the same workflow the personas above were built with.

Custom PCB (Optional)

Don't want to breadboard it? pcb/ has a custom 2-layer breakout PCB that wires up the same components fixed pin-for-pin (matching the tables below) and adds a solder-in expansion header for spare GPIOs. This is the exact board design that was ordered, assembled, and is running the unit here at VA5 HQ — not an unverified first spin.

What's in pcb/:

  • va5pcb.kicad_pcb + va5pcb.kicad_pro -- the KiCad board project. Open it in KiCad (8.0+) to inspect footprint placement, silkscreen, or the routed copper before ordering.
  • fab/ -- bare-board gerbers + Excellon drill (*.gbr, *.drl, .gbrjob), already exported and ready to hand to a fab house (JLCPCB, PCBWay, etc.) as-is. Standard 2-layer board, no unusual stackup.

The board was scripted-routed then hand-finished in KiCad (a couple of stragglers a headless autorouter leaves behind); run your own DRC pass in KiCad if you want to double-check before ordering. The parametric generator that produced this board (component placement, net list, silkscreen) isn't part of this release -- pcb/ ships the finished design, not the pipeline that built it.

Populate it with the same parts as the Hardware Components table above (skip the breadboard and jumper wires -- solder straight onto the board footprints instead). The PCM5102A headphone DAC is optional, same as the breadboard build; if you install it, use the PCB build parts (pcb_ring.stl, pcb_top.stl, pcb_bottom.stl) from 3D Printed Enclosure below, which carry a matching 3.5 mm jack hole in the ring. The board's spare-GPIO header is documented in Expansion Header (PCB build only) below.

Pin Assignments

Microphone -- INMP441

INMP441 PinESP32-S3 PinNote
SD (data)GPIO 40I2S data in
WSGPIO 41Word select / LRCLK
SCKGPIO 42Bit clock
L/RGNDSelects left channel
VDD3.3V
GNDGND

Speaker Amplifier -- MAX98357A

MAX98357A PinESP32-S3 PinNote
DINGPIO 17I2S data out (shared with PCM5102A if both populated)
BCLKGPIO 47Bit clock (shared)
LRCGPIO 21Word select / LRCLK (shared)
SD3.3V or GPIO 383.3V keeps the amp always on (legacy wiring). Tie to GPIO 38 instead if you've also populated the PCM5102A and want to switch between them from the portal.
VIN5V
GAIN3.3V6 dB gain (recommended) -- avoids amp clipping/distortion into an 8 Ω speaker at high volume
GNDGND

Headphone DAC (optional) -- PCM5102A

Populate this only if you want a headphone jack or want to feed a powered speaker (e.g. JBL) via 3.5 mm AUX. With both chips installed, the portal Audio Output selector picks which one plays; firmware mutes the other via its enable pin.

PCM5102A PinESP32-S3 PinNote
DINGPIO 17I2S data out (shared with MAX98357A)
BCKGPIO 47Bit clock (shared)
LCKGPIO 21Word select / LRCLK (shared)
SCKGNDUse internal PLL (no MCLK from ESP32)
XSMTGPIO 39Active-low mute; HIGH = play
FMT, FLT, DEMPGND (factory-set on most breakouts)I2S format, normal filter, no de-emphasis
VIN3.3VMost GY breakouts accept 3.3V or 5V; check silkscreen
GNDGND
L / R / GND3.5 mm stereo jackBuilt into the breakout PCB

The portal Audio Output selector (below Volume) picks Speaker (MAX98357A) or Headphones (PCM5102A). The choice persists across reboots. Changes are rejected with HTTP 409 during an active conversation -- save while idle (green LED).

You can also just say "switch to headphones" or "use the speaker". VA5 confirms on the current output and switches right after that reply. A small speaker / headphones icon above the on-screen volume bar shows where audio is going. (Without a PCM5102A installed, "headphones" goes silent; say "switch back to the speaker" to undo.)

Display -- GC9A01

GC9A01 PinESP32-S3 PinNote
SCLKGPIO 6SPI clock
MOSI (SDA)GPIO 7SPI data
CSGPIO 5Chip select
DCGPIO 4Data/command
RSTGPIO 2Hardware reset
BLK3.3VBacklight (always on)
VCC3.3V
GNDGND

Other

ComponentPinESP32-S3 PinNote
PTT ButtonSignalGPIO 1INPUT_PULLUP, active LOW
PTT ButtonGNDGND
NeoPixel RGB LEDDataGPIO 48Built-in on most S3 boards, no jumper needed. Also brought out on the expansion header (see below) for future repurposing.
MAX98357A SD (optional)EnableGPIO 38Only needed if you populated the PCM5102A and want runtime switching. Otherwise tie SD to 3.3V.
PCM5102A XSMT (optional)MuteGPIO 39Only populated when PCM5102A is installed. HIGH = play, LOW = mute.

Power Summary

RailComponents
3.3VINMP441, GC9A01, MAX98357A SD pin (legacy wiring only), GC9A01 BLK, PCM5102A VIN (optional)
5VMAX98357A VIN
GNDAll components (common ground)

Expansion Header (PCB build only)

The PCB build exposes every ESP32-S3 GPIO that isn't already driven by a peripheral, plus power rails, on unpopulated 2.54 mm through-hole pads. Solder a male or female header strip if you want one — pick whichever fits your add-on. Most pads sit in the lower band of the board; four GPIOs are on a small screen-side header in the gap between the ESP and the display.

HeaderPadsSilkscreen labels
Data — bottom (J9)8 GPIO9 48 10 11 12 13 14 19
Data — screen-side (J13)4 GPIO15 16 18 8
Power — bottom (J10)73V 3V G G G 5V 5V (2×3.3 V / 3×GND / 2×5 V)

Notes on the data row:

  • The four ESP32-S3 boot-strapping pins (GPIO 0, 3, 45, 46) are not broken out — they have boot-time constraints, so the header exposes only freely-usable GPIO.
  • GPIO 19 is also the chip's native USB D−. GPIO 20 (the other native USB line, D+) is not broken out on this header. If a future build uses the chip's native USB, GPIO 19 becomes unavailable too.
  • GPIO 48 is the on-module RGB status LED. It's brought out for future repurposing, but anything you wire to it will fight the firmware's status indicator until you disable that path in code.

Silkscreen labels are shortened (3V rather than 3V3, G rather than GND) so each label fits within the 2.54 mm pad pitch.

Build Toolchain

Pick one of the two toolchains below -- both build the same .ino source. Versions are pinned in toolchain.md.

Option A -- Arduino IDE

Set these under the Tools menu before uploading:

SettingValue
BoardESP32S3 Dev Module
USB CDC On BootEnabled
CPU Frequency240MHz (WiFi)
Core Debug LevelNone
USB DFU On BootDisabled
Erase All Flash Before Sketch UploadDisabled
Events Run OnCore 1
Flash ModeQIO 80MHz
Flash Size16MB (128Mb)
JTAG AdapterDisabled
Arduino Runs OnCore 1
USB Firmware MSC On BootDisabled
Partition SchemeCustom (uses partitions.csv from sketch folder)
PSRAMOPI PSRAM
Upload ModeUART0 / Hardware CDC
Upload Speed921600
USB ModeHardware CDC and JTAG

The custom partition table (partitions.csv) provides 3 MB for the app and ~11 MB for LittleFS (emoji frames).

Install libraries via Arduino Library Manager (Sketch > Include Library > Manage Libraries...):

LibraryAuthorNote
ArduinoJson (v6.x+)Benoit BlanchonJSON parsing for API messages
Adafruit NeoPixelAdafruitRGB status LED driver
Arduino_GFX_LibrarymoononournationDisplay driver (required only when USE_DISPLAY 1)

QR codes (captive-portal join + network info screen) use the esp_qrcode API bundled with the ESP32 board package -- no separate library install.

ArduinoWebsockets is vendored in the src/ directory with two critical bug fixes applied. Do not install it via Library Manager -- the sketch includes the local patched copy automatically. See websockets_patches.md for details.

Option B -- PlatformIO

Everything (board settings, partition scheme, libraries, build flags) is pinned in platformio.ini. After cloning and copying config.h.sample to config.h (see below):

bash
pio run -t upload          # compile + upload sketch
pio run -t uploadfs        # upload LittleFS data partition (emoji frames)
pio device monitor         # serial monitor at 115200, with esp32 exception decoder

Or, in VS Code, install the PlatformIO IDE extension and use the toolbar buttons (Build / Upload / Upload Filesystem Image / Serial Monitor).

platformio.ini uses the pioarduino fork of the espressif32 platform -- the default upstream platform is stuck on Arduino-ESP32 core 2.x and is missing a header that Arduino_GFX_Library 1.6.5 needs. The pin 55.03.312-1 matches Arduino-ESP32 3.3.12.

The vendored ArduinoWebsockets is found via PlatformIO's src_dir = voice_agent_5 setting -- do not add arduinoWebsockets to lib_deps.

Configuration

config.h

A config.h.sample file is included in the sketch folder. Copy it to config.h and fill in your credentials:

bash
cp voice_agent_5/config.h.sample voice_agent_5/config.h

Then edit config.h and replace the placeholder values with your OpenAI API key (and optionally WiFi credentials):

cpp
#define OPENAI_API_KEY "sk-your-openai-api-key"
#define WIFI_SSID "your-network-name"
#define WIFI_PASSWORD "your-network-password"

config.h is listed in .gitignore and will not be committed. WiFi credentials are also configurable at runtime via the captive portal and stored in NVS. The API key can also be set via the web UI (stored in NVS, overrides config.h).

WiFi Setup (Captive Portal)

  1. On first boot (or when none of the saved networks can be reached), the device creates a WiFi access point named VOICE-AGENT-XXYY (last 2 bytes of MAC)
  2. The RGB LED flashes yellow to indicate configuration mode
  3. The display shows a WiFi QR code and join instructions — scan it with your phone to join the network automatically
  4. A captive portal page opens automatically (or navigate to 192.168.4.1)
  5. Under Wi-Fi Networks, enter your network name and password and click Add network (or press Scan and pick it from the list), then Connect & reboot
  6. The device restarts and connects to your WiFi network
  7. Once connected, the display shows the IP address and a QR code — scan it or enter the IP in a browser to reach the settings page
  8. The RGB LED turns green when ready

Shortcut: hold the push-to-talk button while powering the device on to go straight to the config portal, without waiting for it to try the saved networks first.

Multiple WiFi Networks

The device remembers up to 6 networks, so it can move between home, the office and a phone hotspot without being set up again.

  • At power-on it scans, then joins the most recently used saved network that is in range. If that one fails it tries the next, and so on; only when they have all failed does it open the config portal. The display names each network as it tries it.
  • A failed connection never erases the network. An AP that was simply switched off stays in the list.
  • Add networks ahead of time. You can save a network you are nowhere near — type its name and password, and it will be used the next time it is in range. Adding a network does not reboot the device or disturb the current connection.
  • Full list? Saving a seventh network drops the least recently used one to make room, and the page tells you which one went. Networks you have saved but never actually connected to are dropped first.
  • Switching now. Press Connect on any saved network to reboot into it. The device attempts that exact network first, whether or not it showed up in a scan. If it can't be reached, it falls back to its usual order rather than refusing, and tells you so when you next load the settings page: "Couldn't reach UdisS24P — still on Tirosh-g." Forget removes one; forgetting the network you are currently using does not cut you off, it just stops the device from choosing it again.

Passwords are stored on the device and are never shown again — not on the settings page and not through its API. They are, however, stored unencrypted in the ESP32's NVS flash, so treat physical access to the board as access to those passwords.

Agent Settings (Web Portal)

Once connected to WiFi, open a browser and navigate to http://voice-agent-XXYY.local (or the IP address printed to Serial Monitor).

From the settings page you can configure:

  • System Prompt -- Control the assistant's persona and response style
  • Voice -- One of the 10 OpenAI Realtime voices (marin, cedar, alloy, ash, ballad, coral, echo, sage, shimmer, verse); selection takes effect on the next session
  • Language -- The language the assistant speaks. Default Automatic matches whatever language you speak; pick a specific language to force every reply into it. Takes effect on the next session
  • Volume (0 - 100%) -- Speaker playback volume
  • Audio Output -- Speaker or Headphones (only if the optional PCM5102A DAC is installed)
  • Import / Export Profile -- Browse a gallery of ready-made personas, or back up / transfer your own persona, voice, volume and emoji as a single .zip file. Secrets (Wi-Fi, API key) are never exported. Needs the browser to be online. See voice_agent_5/docs/profile-import-export.md.

Less-common options live behind the Admin Zone toggle further down the page: Wi-Fi Networks (see Multiple WiFi Networks above), your API Key (override the compiled config.h key, stored in NVS), conversation behavior (Persist Conversation, Verbose Logging), and token usage.

All settings are saved to NVS and persist across reboots. Use Restore defaults (in the Admin Zone) to reset the agent settings.

The Admin Zone also has Provision default persona, which pulls the published default persona -- prompt, voice and emoji -- from the project repo and writes it into the device. Use it to set up a board that was flashed with firmware but no emoji image (so the display would otherwise be blank), or to load a newly published default. Unlike Restore defaults (which reverts to the compiled-in fallback prompt and on-device emoji), this downloads the current default from the web, so your browser needs internet. It writes the device's read-only default emoji set directly and checks free storage before changing anything.

Emoji Display Setup

Prerequisites

  • GC9A01 display wired per the pin table above
  • USE_DISPLAY set to 1 in voice_agent_5.ino (default)
  • Arduino_GFX_Library available (Library Manager for Arduino IDE; auto-resolved from lib_deps for PlatformIO)

Replacing Emojis from Your Browser

Once the device is on WiFi, browse to http://voice-agent-XXYY.local/emojis (or hit Open editor on the main settings page). Drop in a GIF, PNG, JPG, or H.264 MP4 for any of the seven moods and the assistant will use it immediately -- no re-flash, no Python, no SD card. The shipped defaults are stored separately and are never overwritten, so there is no way to brick the device through customization; reset any mood (or all of them) at any time. See voice_agent_5/docs/emoji-customization.md for the full walkthrough, supported formats, and how to share an emoji pack.

The customization page needs your browser to have internet access (it loads the GIF decoder and zip libraries from a CDN). The device itself stays local-only.

Uploading the LittleFS Filesystem

The emoji animations are stored as pre-rendered RGB565 binary frames in a LittleFS partition. They must be flashed separately from the sketch. The sketch and filesystem go to different partitions -- they don't overwrite each other. You only need to re-upload the filesystem when the frame files change.

With PlatformIO:

bash
pio run -t uploadfs

platformio.ini already points data_dir at voice_agent_5/data/, so this packages the 56 .bin frame files into a LittleFS image and flashes it to the data partition.

With Arduino IDE (one-time plugin install):

  1. Go to https://github.com/earlephilhower/arduino-littlefs-upload/releases
  2. Download the latest .vsix file
  3. Copy it to ~/.arduinoIDE/plugins/ (create the plugins folder if it doesn't exist)
  4. Restart Arduino IDE

Then to upload:

  1. Close the Serial Monitor (it locks the COM port)
  2. Press Ctrl+Shift+P to open the Command Palette
  3. Type Upload LittleFS and select "Upload LittleFS to Pico/ESP8266/ESP32"
  4. Wait for it to finish

Verify: Open Serial Monitor after uploading both sketch and filesystem. You should see:

text
LittleFS mounted.
Display: loaded 8 frames for 'neutral'
Display: GC9A01 initialized

Re-generating Emoji Frames (Optional)

If you want to regenerate the data/*.bin files from the source GIFs in feelings/:

Requirements: Python 3 with Pillow (pip install Pillow)

Run:

bash
cd voice_agent_5
python utils/convert_gifs.py

This extracts 8 evenly-spaced frames from each GIF, resizes to 150x150, converts to RGB565 little-endian binary, and writes to data/default/<emotion>_<frame>.bin. The default/ subdirectory is the shipped emoji set; runtime overrides written from the captive portal land in data/custom/ on the device and never touch the defaults.

Emoji Credits

Emoji animations sourced from Google Fonts Emoji, used under the Creative Commons Attribution 4.0 International License (CC BY 4.0).

Library Patches

ArduinoWebsockets v0.5.4 has two upstream bugs that prevent WSS connections on ESP32: a missing setInsecure() method in SecuredEsp32TcpClient, and a missing fallback to insecure mode in the ESP32 branch of upgradeToSecuredConnection(). Both cause every secure WebSocket connection to fail silently. (Both are still present upstream as of v0.5.4.)

The library is vendored in src/ with fixes already applied -- no manual patching required. For full details, the exact code changes, and instructions for updating to a newer version, see websockets_patches.md. For the licensing side -- upstream version, licence, and the full list of what was modified -- see VENDORED.md.

Tuning Knobs

Compile-Time Constants

These are defined in voice_agent_5.ino and require recompilation to change:

ConstantDefaultDescription
SAMPLE_RATE_IN24000Microphone sample rate (Hz). Required by the OpenAI Realtime GA API (audio/pcm is fixed at 24 kHz).
SAMPLE_RATE_OUT24000Speaker playback rate (Hz). Must match OpenAI Realtime API output format.
JITTER_BUFFER_MS300Milliseconds of audio to buffer before starting playback. Higher = smoother but more latency.
LOW_WATER_BYTES3072Rebuffer threshold (~64 ms at 24 kHz). When the speaker buffer drops below this during playback, pause and re-accumulate before resuming.
DMA_BLOCK_SIZE1024I2S DMA transfer size in bytes per read/write call.
WIFI_CONNECT_TIMEOUT_MS8000Time to wait for each saved network in range before trying the next one.
WIFI_SOLE_TIMEOUT_MS15000Time to wait when exactly one saved network is in range — there is nothing to fall through to but the config portal, so it gets a longer budget.
WIFI_MAX_NETWORKS6How many WiFi networks are remembered (in wifi_store.h). Saving another drops the least recently used one.
EMOJI_FRAME_MS120Delay between emoji animation frames (~8 fps). Lower = faster animation.
EMOJI_SIZE_PX150Emoji frame dimensions (width and height). Must match convert_gifs.py output.
EMOJI_NUM_FRAMES8Number of animation frames per emotion. Must match convert_gifs.py output.
USE_DISPLAY1Set to 0 to disable display entirely (serial-only mode, no GC9A01 or Arduino_GFX needed).

Runtime Settings (Web UI / NVS)

These are configurable at runtime via the web portal and persist across reboots:

SettingDefaultRangeDescription
System Prompt"You are a helpful voice assistant..."up to 12 KB (12288 chars)Controls assistant persona
Voicemarinalloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, cedarOpenAI Realtime voice; applies on next session
LanguageAutomaticAutomatic or one of 13 languagesForces every reply into the chosen language; Automatic matches whatever you speak. Applies on next session
Persist ConversationtruebooleanChain responses via previous_response_id
Verbose LoggingtruebooleanDetailed serial debug output
Volume500 - 100Speaker volume (PCM sample scaling)
Audio OutputSpeakerSpeaker / HeadphonesWhich output plays (only meaningful with the optional PCM5102A installed)

3D Printed Enclosure

The box/ directory contains the STL files for a circular 145 mm enclosure designed to hold all components. The geometry is a parametric build123d (Python) model; the .stl files here are pre-generated from it, so you don't need to render anything yourself.

Each printable part is its own STL, and the file prefix tells you which build it belongs to — print every 3dp_* file or every pcb_* file, never a mix. No part is shared between the two builds.

  • Breadboard / jumper-wire build (3dp_*, 7 parts): 3dp_ring.stl (carries the same 3.5 mm jack passthrough as the PCB ring, for the optional PCM5102A), 3dp_top.stl (the top plate carries printed bosses for the ESP32, display, amp, mic, speaker, and button), 3dp_bottom.stl, plus the small parts that mount components to the top plate: 3dp_speaker_y_bracket.stl, 3dp_cantilever_bracket.stl (ESP32 clamp), 3dp_inline_button_frame.stl, and 3dp_inline_button_bracket.stl.
  • PCB build (pcb_*, 3 parts): pcb_ring.stl (includes a 3.5 mm jack passthrough aligned with the optional PCM5102A DAC), pcb_top.stl (completely flat — the speaker is glued, so there are no bosses), pcb_bottom.stl (carries four standoffs that set the PCB height above the bottom plate).

The parametric build123d source (where dimensions, component positions, and clearances are defined) is maintained in the project's development repo and isn't bundled with this release — just pick the STL set matching your build above and print.

The design uses a top/ring/bottom construction: the top plate holds all components (ESP32, display, speaker, mic, amplifier, button) facing down into the body — from outside you see its flat face. It is printed feature-side-up and flipped over at assembly; the model is pre-mirrored so that, once flipped, everything lines up with the PCB below. The ring forms the enclosure wall (USB cutout toward the ESP; orient the headphone hole toward the bottom), and the bottom plate closes the base. Parts are secured with M3 screws into printed bosses.

State Machine / LED Colors

StateLED ColorMeaning
STATE_WIFI_WAITYellow (solid)Connecting to WiFi
STATE_WIFI_CONFIGYellow (flashing)AP mode, awaiting WiFi credentials
STATE_READY_FOR_INPUTGreenIdle, ready for push-to-talk
STATE_RECORDINGPinkMic active, streaming audio to OpenAI
STATE_THINKINGBlueWaiting for API response
STATE_SPEAKINGCyanPlaying response audio

Adding a Tool

Tools are OpenAI function calls the model makes autonomously during a response. All tool registration lives in voice_agent_5/tools.ino — adding a tool is two steps and touches only that one file.

Step 1 — Write the handler:

cpp
static void handleMyToolCall(const String& payload) {
    DynamicJsonDocument doc(256);
    if (deserializeJson(doc, payload) != DeserializationError::Ok) return;

    beginToolCall(doc["call_id"] | "");   // sets up g_pending_call_id and flags

    // parse arguments, do your work, optionally set g_tool_result_output
    g_tool_result_output = "done";
}

Important: handlers run inside the WebSocket onMessage callback and cannot call wsClient.send() directly. Use the flag mechanism (beginToolCall + g_tool_result_output) — the protocol loop sends the result after response.done arrives. For cross-core side effects (display, NVS writes), set a volatile bool flag and handle it on Core 1 in loop().

Step 2 — Register the tool:

Add an entry to the TOOLS[] array in tools.ino:

cpp
{
    "my_tool_name",
    "Plain-text description the model reads to decide when to call this.",
    "{\"type\":\"object\",\"properties\":{"
    "\"param1\":{\"type\":\"string\",\"description\":\"What param1 does\"}"
    "},\"required\":[\"param1\"]}",
    handleMyToolCall
}

buildToolsJson() will include the new schema in the next session.update, and dispatchToolCall() will route calls to your handler. No changes to protocol.ino needed.

For the full async tool call flow diagram, see voice_agent_5/docs/architecture.md.

Architecture Documentation

For detailed technical documentation covering dual-core architecture, ring buffer internals, audio pipeline, protocol details, emoji display system, and memory layout, see architecture.md.

Troubleshooting

  • LED stays yellow: the device is still working through its saved WiFi networks. It tries each one that is in range for 8 seconds before falling back to SoftAP config mode, so with six saved networks this can take about a minute. Hold the push-to-talk button while powering on to skip straight to the config portal.
  • No sound from speaker: Verify the MAX98357A SD pin is connected to 3.3V (not GND).
  • Microphone not picking up audio: Check INMP441 wiring (especially L/R to GND for left channel).
  • Can't reach .local address: mDNS may not work on all networks/devices. Use the IP address printed to Serial Monitor instead.
  • Display shows text instead of emoji: LittleFS filesystem not uploaded, or frame files missing. Re-upload the filesystem using the LittleFS plugin.
  • "Display: gfx->begin() FAILED!": Check display SPI wiring (especially DC, CS, RST pins).

License

VA5 is licensed under GPL-3.0 (GNU General Public License, version 3).

You can build these devices, use them, modify them, teach with them, and sell them. None of that needs anyone's permission. If you'd like to talk about a commercial arrangement or a classroom deployment, you're welcome to — hello@oriliventures.com — but it isn't a condition of any of the above.

The obligation that comes with GPL-3.0 is share-alike: if you hand someone the firmware, or a device running it, modified or not, they're entitled to the source under this same licence.

Third-party components

  • ArduinoWebsockets — a modified copy of gilmaimon/ArduinoWebsockets v0.5.4 is vendored in voice_agent_5/src/. It is GPL-3.0, and that is why this project is GPL-3.0. What was changed and why: VENDORED.md; the exact patches: websockets_patches.md.
  • Emoji animations — from Google Fonts Emoji, under CC BY 4.0.
  • base64 encoder — © 2004-2008 René Nyffenegger, zlib-style licence, notice retained in voice_agent_5/src/tiny_websockets/internals/wscrypto/base64.hpp.
  • SHA-1 implementation — from 983/SHA1, credit retained in voice_agent_5/src/tiny_websockets/internals/wscrypto/sha1.hpp.

Parts

16
QtyValueFootprintReferences
2ESP32-S3PinSocket_1x22_P2.54mm_VerticalJ1, J2
2INMP441PinSocket_1x03_P2.54mm_VerticalJ3, J4
2USB-C 5VR_0603_1608MetricR1, R2
1EXP-AUXPinSocket_1x02_P2.54mm_VerticalJ11
1EXP-DATAPinSocket_1x08_P2.54mm_VerticalJ9
1EXP-PWRPinSocket_1x07_P2.54mm_VerticalJ10
1EXP-SCRPinSocket_1x04_P2.54mm_VerticalJ13
1GC9A01PinSocket_1x08_P2.54mm_VerticalJ8
1MAX98357APinSocket_1x07_P2.54mm_VerticalJ5
1PCM5102APinSocket_1x09_P2.54mm_VerticalJ6
1PCM5102APinSocket_1x06_P2.54mm_VerticalJ7
1PTTSW_PUSH-12mmSW1
1USB-C 5VUSB_C_Receptacle_GCT_USB4135-GF-A_6P_TopMnt_HorizontalJ12

Similar boards

  • 4 layers85.5 × 112.2 mmTranslateIA, the printed circuit board in 3D

    TranslateIA

    lapalmaelettronica/Progetti

    Una lista dei progetti che ho pubblicato su instagram, potete trovarci pcb, codici, schemo elettrici e stampe 3d!

    77 parts
  • 2 layers42 × 30.8 mmWifiaudiorx, the printed circuit board in 3D

    Wifiaudiorx

    BearLaboratory/wifiaudio-rx-hardware

    基于ESP32的WiFi无线麦克风接收端

    52 parts20GPL-3.0
  • 2 layers124 × 40 mmRedFIS, the printed circuit board in 3D

    RedFIS

    tothpetiszilard/a3_8p_redFIS

    Hacking into Audi A3 8P Driver Information System

    77 parts15AGPL-3.0
  • 2 layersMain pcb, the printed circuit board in 3D

    Main pcb

    ostev/keysynth

    An equation synthesiser

    299 parts
  • 2 layers56.4 × 78.4 mmAIOsense, the printed circuit board in 3D

    AIOsense

    Schluggi/AIOsense

    ESPHome based all-in-one sensor

    25 parts162MIT
  • 2 layers70 × 100 mmIot badge-433, the printed circuit board in 3D

    Iot badge-433

    galopago/iot-badge-legacy

    Electronic conference badge for programming conventions, hacking events, etc.

    77 parts4
  • 2 layers39.1 × 39.9 mmNebula, the printed circuit board in 3D

    Nebula

    Thecactusman0/Nebula-openwatch

    Nebula is a ESP32 based smartwatch that features a rotating bezel encoder

    85 parts4
  • 2 layers74.9 × 100.3 mmSpectrum analyzer, the printed circuit board in 3D

    Spectrum analyzer

    JadenS180/spectrum-analyzer

    ESP32-based real-time audio spectrum analyzer with custom PCB design. Features INMP441 I2S microphone, SSD1306 OLED display, FFT processing, and real-time WiFi streaming to a browser-based visualizer via WebSockets.

    11 parts2MIT

Questions

How big is the Va5pcb PCB?

It measures 135 × 135 mm and has 2 copper layers, 1.6 mm thick.

What parts does the Va5pcb use?

16 parts, built around the ESP32-S3, INMP441 and PCM5102APW. The full list of its parts is on this page, with links to each part's symbol and footprint where PCBKit's library has them.

Where can I download the Va5pcb KiCad files?

From the iollama/Voice-Assistant-5 repository on GitHub, at the commit this page was made from: the board is pcb/va5pcb.kicad_pcb. PCBKit opens the board in the browser, with nothing to install.

Can I use the Va5pcb design in my own project?

Its repository publishes it under the GPL-3.0 licence: you may reuse it on that licence's terms (keep the author's notice, and for a copyleft licence share your changes alike).