We designed a wall instrument. People turned it into a dance floor.
We designed a wall instrument. People turned it into a dance floor.
FlipTiles, a spatial sound installation where flipping physical tiles creates unexpected melodies
Category
Interaction Design, Spatial Computing
Tools
P5.js · Procession · Camera · External speakers
Timeline
2 months (November 2025 - December 2025)
Team
Atisha Kudesia, Nicole Lucero-Huerta
FlipTiles, a spatial sound installation where flipping physical tiles creates unexpected melodies
Category
Interaction Design, Spatial Computing
Tools
P5.js · Procession · Camera · External speakers
Timeline
2 months (November 2025 - December 2025)
Team
Atisha Kudesia, Nicole Lucero-Huerta
A wall covered in black tiles. Flip one, and it reveals a sound. Flip another. A different sound. There's no score, no timer, no win condition. Just a surface that responds to your hands and rewards curiosity with music.
A wall covered in black tiles. Flip one, and it reveals a sound. Flip another. A different sound. There's no score, no timer, no win condition. Just a surface that responds to your hands and rewards curiosity with music.
FlipTiles explores what happens when you strip interaction down to its most elemental form: touch something, hear something, feel something.
FlipTiles explores what happens when you strip interaction down to its most elemental form: touch something, hear something, feel something.
"Not every interaction needs a goal. Some just need a reason to reach out and touch."
"Not every interaction needs a goal. Some just need a reason to reach out and touch."
The project was built around three ideas: learning through embodied experimentation, the belief that interaction can exist purely for pleasure, and accessibility that cuts across literacy, language, and age.
A child and a grandparent should be able to walk up to this wall and immediately start making something together.
Most interactive installations ask you to achieve something. FlipTiles asks you to play, in the oldest sense of the word.
Most interactive installations ask you to achieve something. FlipTiles asks you to play, in the oldest sense of the word.
An iPhone mounted on a tripod acts as the camera, feeding a live view of the tile wall to a laptop running Procession.
The system uses brightness detection to determine whether each tile is flipped open or closed — dark side up means silent, color side up triggers a sound through an external speaker. Each tile maps to a different note or tone, so the wall becomes a playable instrument.
An iPhone mounted on a tripod acts as the camera, feeding a live view of the tile wall to a laptop running Procession.
The system uses brightness detection to determine whether each tile is flipped open or closed — dark side up means silent, color side up triggers a sound through an external speaker. Each tile maps to a different note or tone, so the wall becomes a playable instrument.
"The most interesting interactions were the ones we didn't design."
"The most interesting interactions were the ones we didn't design."
The installation was tested at School of Visual Arts, Open Studio, and people immediately broke the rules (in the best way).
Some visitors flipped one tile at a time, listening carefully to each sound before choosing the next.
Others grabbed handfuls, flipping multiple tiles at once to create overlapping, chaotic melodies they couldn't have planned.
And a few people ignored the tiles entirely, walking and jumping in front of the camera, discovering that their bodies could trigger sounds too.
The brightness-based detection had a flaw: people's clothing sometimes triggered sounds. Dark shirts blended with closed tiles; light scarves registered as open ones. It made the system harder to explain but accidentally expanded what "interaction" meant. Your body became part of the instrument.
The installation was tested at School of Visual Arts, Open Studio, and people immediately broke the rules (in the best way).
Some visitors flipped one tile at a time, listening carefully to each sound before choosing the next.
Others grabbed handfuls, flipping multiple tiles at once to create overlapping, chaotic melodies they couldn't have planned.
And a few people ignored the tiles entirely, walking and jumping in front of the camera, discovering that their bodies could trigger sounds too.
The brightness-based detection had a flaw: people's clothing sometimes triggered sounds. Dark shirts blended with closed tiles; light scarves registered as open ones. It made the system harder to explain but accidentally expanded what "interaction" meant. Your body became part of the instrument.
What Stayed With Me
What Stayed With Me
Goallessness is its own design challenge. Without a score or objective, the interaction has to be intrinsically rewarding — every flip has to feel good enough to make you flip another.
Goallessness is its own design challenge. Without a score or objective, the interaction has to be intrinsically rewarding, every flip has to feel good enough to make you flip another.
Goallessness is its own design challenge. Without a score or objective, the interaction has to be intrinsically rewarding, every flip has to feel good enough to make you flip another.
People will always surprise you. Designing for single-tile flips and watching people create full-body performances taught me more about interaction than any user test.
People will always surprise you. Designing for single-tile flips and watching people create full-body performances taught me more about interaction than any user test.
Simple detection, rich behavior. Brightness thresholds are about as low-tech as computer vision gets. But mapped to sound across hundreds of tiles, the simplest input produced the most complex, unpredictable output.
Simple detection, rich behavior. Brightness thresholds are about as low-tech as computer vision gets. But mapped to sound across hundreds of tiles, the simplest input produced the most complex, unpredictable output.