Misfortune Cookies

Interactive Testing

Misfortune Cookies is an interactive experience exposing the realities of hate crimes through a familiar cultural symbol. Users shatter a giant virtual fortune cookie with their hand movements, unveiling shocking headlines of violence and discrimination hidden inside. This project utilizes a Kinect camera, Unreal Engine 5, and TouchDesigner to merge interactivity and powerful storytelling, confronting viewers with the weight of unspoken truths while challenging the societal tendency to ignore these injustices.

Screen Capture

Interaction Breakdown

At the beginning of the experience, viewers will be prompted to wave their hand. The Kinect camera will capture this movement, sending OSC signals via TouchDesigner to Unreal Engine, allowing hand movement to activate a trigger box in Unreal, a radial force will be sent to the cookie, shattering it, revealing the message inside. And the message will be randomly picked for every interaction so the viewers will get an authentic fortune cookie experience and a reality check on the current injustice the AAPI community face.

Misfortune Cookie - Projection Mapping

Medium: Projection Mapping on 3D Printed Objects

Misfortune Cookies was born in response to the extreme wave of Asian hate that we’ve seen in recent years, especially since the pandemic. Originally, I had a project using fortune cookies to share the frustrations and experiences of Asians. But this time, I wanted to take it further by transforming the core of that idea.

Instead of sharing typical fortunes, this installation uses projection mapping to display real headlines of hate crimes against Asians. You can view these horrendous events by year through interaction.

The fortune cookie, often seen as a cultural stereotype, becomes a tool for giving voice to the harsh realities that many Asians are still facing.

Please visit https://stopaapihate.org/ to learn more and do more about #StopAAPIHate.

The Next Step

This is an on-going project, more methods and themes will be developed to extend and expand it. And yes, these are real fortune cookies with real news headlines inside them. Stay tuned.