Eye

SarcasmOS

SarcasmOS is a personality-first voice assistant. The core idea is simple: take a smart assistant like Alexa or Google Home, then give it a face, a voice, and most importantly, a personality. Instead of answering like a neutral corporate box, SarcasmOS replies in Spanish with dry humor, sarcasm, and animated expressions inspired by Bender.

This project focusses on the software and AI side of the assistant, but it also includes various hardware elements, like multiple PCBs, sensors, motors, and quite a bit of 3D printing. The goal is to have a memorable, sarcastic, and show-like assistant that can be used both for fun and real tasks like controlling smart home devices, playing music reviewing your agenda, or answering questions.

Demo Video

Do not click here

Why did we Build This?

Futurama is a classic tv show we love (and you should too). Bender is the most memorable character, and his sarcastic personality is a perfect fit for a voice assistant. We wanted to build a voice assistant that is not just a plain, neutral box, but one that has a personality and you won't just use it to get things done, but also to have fun and be entertained

Images

Brain PCB

Eye PCB

Mouth PCB

Language Scope

SarcasmOS is designed to work in Spanish first.

The language model prompt, personality, expected user input, and generated responses are all tuned for Spanish. Other languages may work accidentally depending on the upstream model, but they are outside the intended behavior of this project.

Features

• Listens to speech input.
• Transcribes speech to text.
• Generates Spanish answers with a sarcastic character voice.
• Converts the answer back to speech.
• Syncs audio playback with an animated face.
• Stores memory of past interactions.
• Has a web interface for chat, audio, history and configuration.
• Has integrated speaker and microphone.
• Custom animations for the eyes and mouth.
• 8k mouth display (128x64 pixels = 8192 pixels).

BOM

It is recommended to use the online google spreadsheet for the most up to date BOM with links and prices.

Current State

SarcasmOS is complete, including its Brain, Eye, and Mouth firmware. The finished hardware can listen for its wake phrase, understand a spoken request, use tools, answer in a Spanish Bender-style voice, and coordinate the animated face while it works and speaks.

The Brain is the runtime controller: it talks to the two Eyes over I2C and to the Mouth over acknowledged ESP-NOW. STT, LLM, and TTS remain cloud services, but the physical assistant does not require the Python web backend to run.

Build and Run the Firmware

Bring the boards up in this order so the Brain can be configured with the Mouth's MAC address and matching 2.4 GHz Wi-Fi/ESP-NOW channel.

1. Mouth

Install PlatformIO, connect the Mouth ESP32-S3, and run:

cd Code/Mouth-NeonPCB
pio run -e custom_esp32s3_mini_n8 --target upload
pio device monitor -e custom_esp32s3_mini_n8

At boot, note the station MAC address and ESP-NOW channel printed in the serial monitor. The Mouth uses channel 1 by default; its onboard button can select channels 1 through 13. It must match the Brain's Wi-Fi channel. See the Mouth firmware guide for local animation tests, panel wiring, channel controls, and display emulation.

2. Eyes

Install the Raspberry Pi Pico SDK, connect each RP2040 Eye through SWD, and flash the regular I2C firmware:

cd Code/Eye
./flash.sh --left --regular --build --upload --swd-monitor
./flash.sh --right --regular --build --upload --swd-monitor

The left and right Eyes use I2C addresses 0x30 and 0x31, respectively. The helper also provides self-test and standalone animation-demo firmware. See the Eye firmware guide for OpenOCD setup, manual builds, UART monitoring, alternate animation packs, and the desktop editor.

3. Brain

Install ESP-IDF, connect the Brain ESP32-S3, and run:

cd Code/Brain
idf.py set-target esp32s3
idf.py menuconfig
idf.py build
idf.py flash monitor

Under SarcasmOS Brain in menuconfig, set the Wi-Fi credentials, provider tokens, TTS voice ID, timezone, and the Mouth station MAC recorded above. You can also configure the wake phrase, audio sensitivity, Eye transition timing, and ESP-NOW channel. After flashing, the USB console can test every subsystem, send typed or spoken requests, and update persistent configuration stored in NVS. See the Brain firmware guide for all settings, console commands, API endpoints, native tools, and the independent PCB self-test firmware.

Web Demo

For the already setup web demo, visit https://sarcasmos-7f3c9a2e.easyrumbo.com/.

For the best experience and no ratelimiting, run the local web app.

From Code/AI/Workflow/SarcasmOS-web:

start-all.bat

On macOS/Linux:

./start-all.sh

This starts:

• Proxy (combines all services into 1 port for easy access): http://localhost:9000
• Frontend: http://localhost:5173
• Backend: http://localhost:8001

You can also start them separately:

python -m http.server 5173
python -m uvicorn backend.app:app --host 0.0.0.0 --port 8001

Configuration

The backend reads environment variables from .env files, especially:

• Code/AI/Workflow/SarcasmOS-web/backend/.env
• Code/AI/Workflow/.env

Important variables:

• HACK_CLUB_AI_KEY, or separate provider keys.
• OPENROUTER_API_TOKEN for the language model.
• REPLICATE_API_TOKEN for STT/TTS if using Replicate.
• MINIMAX_VOICE_ID for the TTS voice.
• FFMPEG_PATH if ffmpeg is not available on PATH.

More technical detail:

• Code/AI/Workflow/SarcasmOS-web/README.md
• Code/AI/Workflow/SarcasmOS-web/PROJECT_OVERVIEW.md

Architecture

Microphone / wake phrase / text request
                  |
                  v
          ESP32-S3 Brain
           |      |      \
           |      |       +--> Cloud STT, LLM/tools, and TTS
           |      |
           |      +----------> Mouth over ESP-NOW
           |
           +-----------------> Left + right Eyes over I2C
                  |
                  v
       Speaker audio + synchronized animated face

Things that happened when carrying out the project

When it came to making our project a reality we had several problems, because of customs we had to pay almost $200 more than what we had budgeted. We ended up paying more than $600 for the project 😭

When we ordered the electronic components, 4 were missing, for some reason. We told the company and they sent us the missing components (they were less than $1). When they arrived, which they did way after we finshed the project, and they wanted us to pay customs more than $16 for $0.80 of chips...

The flexible display we wanted to use also got lost in the mail, and we had to pay $60 for one, but it didn't arrive either. We ended up using a different display and driver, which we already had from a previos Hackclub YSWS (Neon).

Credits

Built as a physical AI assistant project for Hack Club / Fallout. SarcasmOS is not trying to be polite. It is trying to be memorable.