REV · C
Yetti DJ Yetti AV
See the music.
A DJ with a PhD in cognitive science. After years of watching lights on ‘sound mode’ just twitch at the bass, I built ones that see the whole song.
Seeing sound isn't a metaphor. Your brain's been doing it all along.
Frame of
Inference
The room is dark. A wall of picture frames. Inside each, a piece of vinyl album art. A single beam of light drifts in the space between the frames.
Walk up to the console and choose a song. Any song, even one this machine has never heard. For the next two minutes you watch it "listen": it pulls the track apart, finds its shape, and paints an album cover — drawn from the song itself, so no two covers are ever alike — live, while you wait. A projector wakes and paints your new cover into a frame on the wall. Nothing here was made in advance.
Then the light beam is yours to control. Guide it over a cover and its song rises with it, muffled until the light fully lands, then clear. The light is the play button: wherever it points, a song plays. And the cover you've lit doesn't hold still: it breathes, animating with the sound it's playing. It first looked like a static cover — now it's dancing to the beat.
On the console, press one of three colors, and one part of the song steps forward alone: just the voice, just the melody, just the beat. The cover follows: only the part you've soloed keeps moving, while the rest holds still. The light is your hand on the record, bringing each one to life.
Two borrowed senses — a DJ’s ears, a synesthete’s eyes.
Frame of Inference gives you two senses you weren’t born with. The first is a DJ’s ears: it pulls a song down to its stems — a voice, a melody, a beat — and lets you lift any one out and hold it on its own. Solo a part, and only its motion keeps moving; everything else goes still.
The second is a synesthete’s eyes. Afferent draws each stem through the shortcuts your brain already takes — high to bright, low to heavy, rough to jagged — so what you’re seeing isn’t a reaction bolted onto the music. It’s the music itself, shown the way a synesthete would hear it: in color, in light.
Seeing sound is not a metaphor. When a violin climbs, you see it rise, bright, sharp, light. Synesthesia is only the loud end of a wiring every brain carries: sound and sight were never fully separate.
Afferent treats that wiring as a constraint, not a taste. It measures the sound’s own qualities — how bright, how high, how rough, how fast — and turns each into a number the picture has to obey.
High to bright. Low to weight. Rough to jagged. Fast to fragmented. Measured by neuroscience, the same in a toddler as in you, across cultures.
Each feature becomes a line of Audio-Visual Markup Language: spectral centroid → brightness, pitch → elevation and size, dissonance → edge, tempo → motion. The renderer obeys the biology.
A DJ’s ear pulls a mix apart into voices, the same unconscious separation your brain runs on every song. Here, a song is drawn down to its stems, and you can lift any one out, hear the voice alone, the beat alone, and see it alone too.
Source separationM and neural beat-trackingM isolate and time-align a song’s vocals, melody, and percussion, and each stem drives its own living layer, so soloing a part solos its image too.
The wish to see sound is old: Newton’s spectrum, Scriabin’s organ of light, Fischinger’s films. The spotlight joins that lineage as a prism: it splits one song into its colors.
What they did by hand, the machine now infers, and renders, on a song it’s hearing for the first time. It pulls the song apart and paints it alive, in a couple of minutes, right there in the room. A prepared visualizer only knows its maker’s playlist; this one listens for itself.
You already hear all of this. Now you can see it too.
Behind it all is Afferent , an engine that encodes a song into sight. It pulls a track into its stems, reads their structure, instruments, and lyrics, and compresses them into AVML: a compact score for the music’s shape. From that score, it renders the living image.
Encode the sound; decode the visuals.
M — already runs on Modal, or easily could. Elastic GPU burst is what makes it render on demand: live, on a song it's never heard.
Past Work
Before the gallery, the dancefloor. Rigs I built to make light move with the music — wood, mirror, NeoPixels, DMX. The last one is where Afferent started.
FACET
A shallow five-sided pyramid that plays light two ways: white faces mapped face-by-face from the front, a mirrored underside that catches a moving head and throws it back across the room.
“The rig that made me want the light to understand the music”
Wiring SoundSwitch into HeavyM to make this react was so cumbersome it made me want to build the tool myself. FACET is where Afferent started.
PICOPANELS
Six-foot fabric monoliths that wake to the music — a pixel grid, a strobe, and glowing edges hidden under hand-sewn cloth, each its own DMX channel, the whole show choreographed track by track.
“Scripted the show by hand, one track at a time.” Afferent is the machine that writes that script.
VERTEX
A cube balanced on its point, spinning to the beat — each face edge-lit from NeoPixels sewn into the fabric, three lasers firing from every corner along its edges, power fed through a slip ring as the whole body turns.
“Power, support and spin — all through a single point.” Lightshow programmed via static loops and chases — Afferent is where Vertex can learn to react to a specific song.
About Yetti
Kiwi born, SF-based DJ, engineer, and audio artist.
I'm a DJ with a PhD in cognitive science, which is a long way of saying I've spent years on two sides of the same question: how the brain makes sense of sound, and how to make sound something you can see. My doctorate was actually sleep and memory, and I still do machine-learning research on the body's signals by day; seeing sound is just the obsession that wouldn't leave me alone.
Turns out they're the same question. Seeing sound isn't a metaphor, it's what your brain already does. A synesthete hears a chord and sees a color, the same one every time, whether they like it or not. The rest of us run a quieter version of the same program: high notes read as bright, low as heavy, a rough sound as a jagged shape, wiring so deep it holds in infants, across every culture we've tested, even in people who've never seen.
So I build instruments that play those mappings back to you. Not visuals slapped over music, visuals pulled out of it: the shape of a sound, made visible. Some of it will feel right in a way you can't quite explain. That's the part I'm building.