Ollie

Role: Created everything, from circuits, to software, to mechanical design.

Type: Commision

Time: March 2022 - September 2022

Ollie is an animatronic puppet I built for local philly artist Jayson Musson, Ollie is capable of person detection, using a TensorFlow Lite ML model tailored specifically for the Raspberry Pi. He follows museum visitors and addresses them with unique voice memos based on the number of visitors. Ollie has six servo motors, 4 microcontrollers, a custom circuit for real time lip-syncing based on the audio signal, and a media player for storing the voice memos. He was fabircated using 3D-printing, laser-cutting, CNC machining, and of course some minor jerry-rigging.

❮

Opening Night

Ollie and me on opening night.

Ollie's "Brain"

Ollie's brain consisted of a Raspberry Pi 4, 2 esp32s, an Arduino Mega, and an audio playback device.
The Raspbbery Pi ran TensorFlow Lite for people detection. It sent the number of people detected to the Arduino Mega via serial and moved a servo which controlled Ollie's body so he'd follow guests in realtime.
A kalman filter was used to keep track of people in the frame to provide smooth movement and funny quips depending on when people entered or left the frame.
One ESP32 extracted the audio wave envelope from the raw output of an analog butterworth op amp filter. I used i2s to get a high speed DMA analog input on the ESP32. This ESP32 then drove a servo which controlled the mouth for in-time lip syncing.
The other esp32 controlled Ollie's head servos, and read recorded motion animations from an SD card.
Finally, the Arduino Mega coordinated all the devices, triggering audio playback and sending commands to the other microcontrollers based on the number of people detected by the Raspberry Pi.