We all carry decades of history in our posture. Layers of inherited, performed, and forgotten cultural memories are etched into how we stand, point, or fold our arms. These motions are not arbitrary; they are scripts passed down through generations like heirlooms. A raised arm might belong to a striker in 1977, a dancer in 2003, or today’s Tesla CEO after a presidential speech.
Gesture analysis, once the realm of anthropologists and artists, has quietly become embedded in daily life through pervasive technologies. Companies like Veesion have turned gestures into data points, training artificial intelligence systems on CCTV footage, eventually succeeding to capture over 100,000 shoplifting actions. This AI detects theft in real-time by reading subtle physical cues, hesitant glances, and quick hand movements that can transform ordinary gestures into markers of suspicion. The implications are considerable as we see how our everyday physical interactions and our bodies themselves have become texts to be continuously read, interpreted, and policed.
In 1832, the antiquarian Andrea De Jorio, the first ethnographer of body language, noticed something rather intriguing: the gestures depicted on ancient Greek vases and Pompeian frescoes looked remarkably similar to those of people he saw every day in Neapolitan markets. He marvelled at how, despite countless shifts in religions, empires, languages, and cultures, these simple movements had somehow endured in this region for more than 2000 years. Gestures, he suggested, have an extraordinary ability to survive history’s disruptions — carrying subtle meanings forward through time in ways that other cultural expressions simply can’t.
The installation
These conceptual threads have materialized in our exhibition “Uncoded Gestures” at Palo Alto’s Escoleta building, where we reimagined participants’ bodies as both archaeological tools and sites of discovery.
We experimented with AI to match our gestures and body movements to archival images — a way to trace how we might be echoing the past without realizing it. To do this, we built a prototype for pattern matching, using historical photographs from the city archives of Hospitalet and later from Poblenou, both rich photographic archives from our hometown.
After some time, we had the chance to test the idea with a wider audience, so we began developing an installation to bring it into public space — specifically, the Palo Alto Escoleta, a small art gallery in Poblenou, Barcelona. To do this, we created a full exhibition around the power of gesture, posing questions like: What defines you more — your gestures or your words? At the same time, we explored how gestures, when repeated over time, can shape not just how we move, but how we are seen.
For the main installation, a camera mapped the silhouettes of every visitor, translating joints and angles into a constellation of data points. This digital skeleton is then compared to thousands of gestures extracted from historical photographs of the photographic archive of Poblenou — images of dockworkers, schoolchildren, lovers, and protesters, each frozen in a moment of unguarded, genuine self-expression. The algorithm, trained on this archive, performs a kind of séance: it identifies the participant’s closest kin across time, not by name or face, but by the curve of a spine or the tilt of a head.
AI as a haunting medium?
Machines remember differently. They see patterns where we see ghosts. Artificial intelligence, often framed as a tool of the future, is here repurposed as a medium for communion with the past. Yet this communion is fraught with ambiguity. The algorithm does not “understand” grief, fatigue, or defiance; it measures distances between vectors. In its gaze, a raised fist might be grouped with both a revolutionary’s protest and a baker’s kneading motion. This is not a flaw but a provocation: it confronts us with the limits of machine logic and the irreducible complexity of human expression.
The installation embraces these slippages. When a match is found, the historical image stands side by side with the live participant’s gesture. The projection remains distinct and separate, yet each shows such a striking resemblance that it seems as if they share a silent dialogue across time, an odd motion through times, part archive, part software, part flesh, an image that asks: Who is haunting whom? Are we summoning the past, or is the past summoning us?
The poetics of repetition
We do not invent our gestures, we borrow them from the dead. Beneath the technical apparatus lies a poetic thesis: gestures are rituals of recurrence. They bind us to strangers across time, creating a fragile kinship of motion. When a participant’s stance aligns with that of a 1930s fishmonger, the installation amplifies this echo. The projection peels back layers of time, revealing not just the matched image but ghostly traces of others who shared that gesture, a student in the 1960s, a child in the 1990s. This is history as a living collage. The participant confronts the duality of gesture: it is both deeply personal (my exhaustion, my joy) and profoundly collective as it discovers its repetition through the past. The installation asks us to see these repetitions not as coincidence but as continuity, a testament to the body’s stubborn refusal to let go of the past.
The backstage movement
Gestures express how we relate to our environment in physical, ethical, and cultural ways. Unlike written, spoken, or visual language, gestures seem more immediate, more directly tied to our lived experiences. Early in the 20th century, anthropologists such as Marcel Jousse and Marcel Mauss argued that gestures aren’t simply physical actions; they’re embedded in our physiology, culture, and social behaviors. According to them, our movements concretely connect mind, body, and environment, shaping our understanding of the world around us. Contemporary researchers like Maxine Sheets-Johnstone and Jurgen Sterck continue to emphasize this idea, suggesting that bodily interactions form the foundation upon which we build knowledge.
Art theory has also deeply explored gesture’s importance. André Chastel examined the subtle ethical meanings behind gestures in Caravaggio’s paintings of Saint Matthew. Roland Barthes highlighted the significance of gesture and rhythm in Cy Twombly’s artwork, while Georges Didi-Huberman uncovered dramatic tension in body language within Pascal Convert’s pieces. In each case, gestures aren’t just movements — they carry critical, even political implications, revealing deep cultural meanings. Moreover, Philosopher Vilém Flusser took this further by connecting gesture to design, highlighting how movements help us make sense of the world.
In recent decades, digital technologies have amplified the role of gesture, especially in interaction design, a field shaped by thinkers like Adam Kendon and Bill Buxton. Haptic feedback, eye-tracking, wearable devices, and immersive virtual environments often exploit gestures as frictionless ways to drive consumer behavior. However, critically examining these technologies can reveal gestures’ potential beyond consumerism, offering designers unexpected opportunities to create meaningful, transformative experiences.
Gesture as Resistance
In a hyper-coded world, perhaps our simplest acts of defiance are gestures we hardly notice: a shrug, an idle stretch, the quiet nod exchanged between strangers waiting for a bus. They resist neat classification and algorithmic precision, slipping through the digital nets cast around us.
And maybe that’s the point. Our bodies, imperfect and unscripted, carry something machines cannot quite reach: an intimate rebellion, a quiet poetry in motion. Gesture reminds us that beneath the data streams, we still possess ways of speaking to each other and to ourselves that no algorithm can fully interpret or contain. In the silent language of everyday movements lies a subtle yet stubborn freedom: a sly refusal to be pinned down, a leftover quirk from generations of humans who pass on more than they ever realize, a quiet refusal to be fully translated by any algorithm.
Exhibition Information
Uncoded Gestures: Body Politics in a Hyper-Codified World
April 10 — May 28, 2025
Escoleta Building, Palo Alto
Initiative by Fundació Palo Alto
Thematic proposal by Albert Fuster Martí
Curated and developed by Domestic Data Streamers
