Wearable Magic

Creativity and Cognitive Augmentation

Programmable Magic

Connecting virtual, vivo, and veee don’t know yet

The Ultimate Magic

To infinity, diversity, and beyond!

Introduction

I have always been the person that loved inventing and programming things. I developed my first computer game when I was 10 years old. However, while I was still in High School, I first recognized the true power of technology. I watched one of the most inspirational TED talks I have ever seen. In this talk, Dr. Pattie Maes described the SixthSense Project, which augments the physical world with digital information and lets us use natural hand gestures to interact with the information. This is the first time that I could see that technology can be magical. At age sixteen, I created my first interactive augmented reality system inspired by the SixthSense Project using my school projector and my laptop to run image recognition code and show interfaces on physical surfaces that detected gestures and responds to them.  As a student from Thailand, Dr. Maes’s talk and the spirit of Media Lab awoke my inner magician.

My magical practice came into fruition through my research on the future of human-computer interaction (HCI) designing and prototyping playful interface systems. For example, I have designed and coded: 1) a 3D printed dress that aesthetically responds to the change in hormones of the person wearing it, empowering the person wearing it to know his/her biological state (accepted to Ubicomp/ISWC’17); 2) a mind-controlled food printer that magically creates food based on the emotion of the person wearing an open BCI device (submitting to CHI’ 18), and 3) a voice controlled gene transformer device that allows a person to cast a spell on the organism by changing genes with their voice.

For me, magic is the shared moment of curiosity where the magician designs an amusing experience that transcends the audience’s expectation and helps them see beyond mundane reality. I believe that the culture of magical sharing is the culture of give and take, where there is an exchange of inspiration and permission to push the boundaries. Once people are inspired to think beyond what they know and have the personal permission to act upon it, they are equipped to be the change agent for society; pursuing things that are even more magical, challenging the status quo, and tackling difficulties in front of them. Inspiration grows aspiration. Magic incites magic. Therefore, I see my role in designing magical technology as the way to build a better society, a more resilient world.

In my TEDx ASU talk titled “Prototyping The Impossible,” I shared my skills of prototyping across computable and uncomputable mediums to create a smooth transition between mind and matter, which allows me to execute projects quickly. Another reason I can handle and lead many exciting projects is due to the fact that I have a great support team and many past collaborators. I have cofounded the Futuristic Research in Enigmatic and Aesthetics Knowledge ( FREAK Lab ) with a network of affiliated researchers from around the world that push the frontier of research in wearable technology, biodigital, machine learning, and space exploration.

As for the future of magic, I believe in the next generation of technology that taps into the deeper relationship between cognition and creativity, people and programmability that will enrich our experience in this world and beyond. Here I present  three areas of research interest: wearable magic, programmable magic, and the ultimate magic–space, which I would like to pursue at MIT Media Lab. These three areas represent three unique labs  I admire and appreciate their visions.

Wearable Magic: Creativity and Cognitive Augmentation

Humans are becoming cyborgs, whether they recognize it or not, as mobile technology is becoming our sixth sense. However, today’s trend in developing such technology mostly focuses on making the stand-alone technology smarter and be more intelligent without looking at the way that human-technology can collaborate to improve the quality of life. Therefore, it is important to shift the research paradigm from AI (Artificial Intelligence) to IA (Intelligence Agent = human + technology) and thinking beyond the process of increasing human intelligence to improving human’s heuristic cognition.

I believed that the everyday objects combined with technology can create the magical moment that allow people to be mindful of their body.  In my development of a 3D mind controlled food printer, I coined the term “cognitive food,” which represents the kind of food that is created in relation to the cognitive state of the person. This project used the electroencephalography signals from the wearable EEG device to classify emotion of the person based on the level of valence and arousal. These variables are then sent over to the food printer to create playful edible responses that reflect and augment the state of mind of the person. For example, printing an iced cookie in the shape of a blossoming flower when the person in a joyful mood  to having food in the shape of  words that speak a message in response to the person’s emotion. This project is beneficial for people with eating disorders and loss of appetite, as the technology can personalize the design, nutrients, and amount of food based on the mental state. It can also create an interesting interplay between humans and machines during the responsive cooking experience by integrating the notion of cognitive food fabrication as one method in the spectrum of the cooking process.

 

The storyboard shows how the augmented reality device can become an inspirational machine.

To augment creativity in people, I am inspired by George Bernard Shaw’s quote, “You see things; and you say ‘Why?’ But I dream things that never were; and I say ‘Why not?’” One of the ideas I would like to explore is on how we might augment people’s creativity by encouraging them to think why not?, and I think augmented reality (AR) is a great tool for that. Instead of augmenting an object with information, why not augmenting the object with semantically unrelated questions to spark interesting ideas.

 For example, imagine walking in the park and seeing an apple while wearing an AR device. The device would recognize the apple, search its property on the ConceptNet, and potentially pick up one feature in its subset such as “environmentally friendly.” The device will then look for the concept that is totally on the opposite side of the apple, and potentially pick up the object “car.” The device will then ask if you would like to be inspired by an apple, and might say something such as, “Good day Pat, what if we can make a car out of an apple? You know the apple is environmental friendly!” Having an AR device asking such silly question could inspire the person to rethink the car manufacturing process using biological and environmental friendly material. This scenario demonstrates how the AR device can become an inspirational machine.

As Robin Williams said that creativity requires a little spark of madness, this process will allow many people to have a creative agent whispering to them creative ideas that they might take on. It is about opening yourself up to thinking differently. By practicing this, people can learn to start thinking innovatively and would converge their creativity with augmented creativity.

Programmable Magic: Connecting virtual, vivo, and veee don’t know yet

Researchers have developed different kinds of programmable objects from virtual to tangible, including hologram to hallucinations, organisms to organ (musical instrument), tangible interfaces to VR and AR. Two of my previous projects have demonstrated the conversion between unrelated programmable objects, which I will discuss below.

The first is “VR storyteller,” a project created during the 2016 MIT Media Lab VR Hackathon. Here I collaborated with a team to develop an algorithm that can read a story that the user types in or says orally to the device, which then extracts key elements in the story using semantic analysis, and predicts the mood of the story using pre-trained machine learning classifiers. The VR generative algorithm then matches the key elements extracted from the story to the 3D objects in the library, places them in the scene by looking at the context, and stylizing the scene through ambient light and sound based on the mood of the story. This project was designed for variety of applications, for example: 1) for educational purposes to encourage children to write and visualize stories; 2) for screenwriters to rapidly render a scene from a script; 3) for received two awards from the hackathon: the most refined VR experience and the best up and coming hackers, and also gained interests from investors and Lucas Films.

 

The programmable magic will connect virtual object, biological object, and tangible object; opening up the new possibilities, and enriching human experiences.

The second design is the “voice controlled gene transformer” project where my colleagues and I designed a device that inserts gene into bacteria based on the voice recognition input, creating the notion that people can cast a spell on a living creature to change its physical properties. In the project, we have a library of three genes: GPF (produces green pigment), RFP (produces red fluorescence pigment), and PGLO (produces blue pigment). This project demonstrates how biological material such as bacteria can be reimagined as digital pixels.

I am interested in continuing this trajectory or research at MIT media lab developing an ecosystem of programmable objects that go beyond one type of medium. I imagine creating a kind of universal language that would allow people to write one code that can be converted into codes that run on different types of programmable objects. I call this new language “programmable magic” because it could allow people to create a richer and more sophisticated environments.  We experience our world through seeing, hearing, smelling, touching, and sensing. Now, programmable magic will allow us to create interfaces that touch on all of those senses, enriching our experiences in the way that makes us feel more connected and engaged – that helps us transcend beyond the ordinary.

Building “programmable magic” requires strong technical experience and great sense of experiential aesthetics. My experience working with artists, scientists, humanists, and designers will allow me to be able to create this kind of magical technology that will open up many possibilities for people to design the future of interfaces, storytelling, play, and entertainment that engage with everyday objects in the magical ways.

The Ultimate Magic: Space – to infinity, diversity, and beyond!

Leonardo Da Vinci wrote, “For once you have tasted flight you will walk the earth with your eyes turned skywards, for there you have been and there you will long to return.” This statement describes humanity’s longstanding curiosity with flight and exploration of the unknown. Our global society may be curious, but we have also created major social issues that highlight the injustice, inequality, violence and environmental degradation that define us too. I wonder, however, In an age of modern space exploration, what part of humanity will we take to other planets as we we search for life? I think it is essential that instead we co-design a new era of space exploration that is decentralized, diverse, and environmentally friendly, and thus showing humanity at its best.  

 

The diagram that shows my vision on the future of space exploration that is decentralized, and built on top of local knowledges through a participatory approach.

Citizen science communities have been making progress toward decentralizing the process of knowledge discovery beyond academia. The mature citizen science communities such as DIY biotech have developed different kinds of sharing platforms (iGEM parts registry, metafluidic, etc.) and mechanisms for growing community through outreach programs, open competitions, and beyond.  It would be interesting to study the complex network of interactions between different agents across the community focusing on cubesat to biotech. With ethnographic data, social media data, and data from different open platforms, I believe that we can create a model of citizen driven discovery that reveals and encourages the kind of structure that promotes “idea flows” among citizens and allows them to thrive.

As a person growing up in Thailand and a member of the first citizen space research group in my home country that won the Thai National Space Exploration proposal call for onboarding  the international space station, I believe in the different paths of socio-technical development. Before the first airplane, the Chinese community was one of the first groups to learn the essence of aerodynamics and develop a flying human made object. What if there is other cultural and traditional knowledge that can contribute in the future of space exploration?  Can the combination of topological optimization algorithm and the traditional knowledge of kite design improve the way we built low-cost and environmental friendly spacecrafts?

One example that I am fascinated with is the centuries-old festival in Thailand and Laos, where the community celebrate the beginning of the rainy season by building a rocket called “Bang Fai” together as message to thank the spirit in the sky.

 

 

Social practices like this demonstrate that space technology can come from act of spirituality, one that is not motivated by military uses or academic advancement. Building on top of this practice, we can synergize with the community to make the ceremonial act of thanking the sky contribute in the protection of their environment. It is possible to develop educational outreach around air pollution protection using the spiritual rocket as a platform for integrating environmental sensors that will help bring greater awareness of their environment and space exploration. It is not a top down approach where knowledge is imported, but a bottom up practice in which knowledge is shared in the community.  But it can also be used to unite people as the future of space exploration is intercultural, inter-continental, and inter-generational.

Conclusion

I presented many ideas here, both ones I have worked to realize and ones that inspire my future research interests, but the thread that connect all these potential projects together is the sense of magic that emerges from connecting technology, design, and people in unexpected ways. I believe that a great project is the one that constantly inspires new questions. As a natural prototyper, I think big, but I know how to carefully address the nano details so that magical dreams can become reality.   

Frank Moss wrote in his book that Media Lab’s professors are the “digital sorcerers” of our time. Moss reveals the highly unorthodox approach to creativity and invention that makes all this possible, explaining how the Media Lab cultivates an open and boundary-less environment where researchers from a broad array of disciplines – from musicians to neuroscientists to visual artists to computer engineers – have the freedom to follow their passions and take bold risks unthinkable elsewhere. I see no other place that a magician like me would thrive, while having fun learning the magic for transforming world for a greater good. We live in the world where we need magic, we need to counter the negative powers that seems unbeatable and drag down human potential. With the support of MIT Media Lab, I propose works meant to save magic in the world, by inspiring others, teaching, and mentoring. Together we can make the world a better place with magic.