Five Notes

Project Overview

Five Notes is an interactive music game designed to introduce and reinterpret the traditional Chinese musical instrument Bianzhong (chime bells) through embodied interaction, sound, and generative visuals.

The project combines embedded sensing, motion-based interaction, and audiovisual design to create a playful experience centered around the ancient Chinese pentatonic scale.

Project Description

The interaction is built around an ESP8266-based embedded system with motion sensing. A gyroscope continuously estimates the orientation and movement of a wine glass, allowing the player’s gestures to control musical and visual elements in real time.

The visual environment was created using TouchDesigner and Adobe After Effects. Rather than following the conventional rhythm-based mechanics of many music games, Five Notes emphasizes relaxed, exploratory, and personalized interaction. Players are encouraged to improvise freely through movement, sound, and visual feedback.

To make the experience accessible to people without formal musical training, the interaction is based on the traditional Chinese pentatonic scale:

Gong · Shang · Jue · Zhi · Yu

Using only five tones simplifies the musical structure while maintaining a connection to traditional Chinese musical culture.

The use of chime bells as the musical medium is intended to evoke the resonant and lingering character of traditional Bianzhong performances. Gesture-based interaction recreates a sense of freely shaping sound through movement, while the wine glass serves as a playful physical interface inspired by scenes of traditional Chinese banquets and social gatherings.

Illustration Video

System Design