September–October 2019
Artist, Designer & Researcher
HoloLens, Maya, Qlone, Adobe Premiere
A speculative design project intertwining nature and technology by means of digital media art
This immersive experience was part of my MFA Thesis work at San José State University. It included multiple design fiction workshops and two interactive art exhibitions, providing a platform for collectively imagining a sustainable future. The speculative series of works were designed to help form a symbiotic relationship between humans and the environment. This was accomplished by writing compelling narratives and prototyping technobotanic life that could exist in 2121. Augmented Reality (AR) was used as a lens to restore our bond with nature, and to turn the participants’ storytelling into story-living.
In order to build new tools and advanced technologies, humans use resources from nature. We are unintentionally causing damage to the ecosystem and as a result we are faced with serious environmental challenges.
By discussing storylines that embrace technological development yet empower the natural world, we could metaphorically hybridize living plants and digital technology, in order to create a new ontology.
Concept video art – the merging of the World Wide Web and Wood Wide Web rhizomes
At the core, my exploratory research was attempting to answer this question: What kind of technological evolutions and biological mutations would help build mutualistic interactions between nature and technology? I gathered some background information from the following books–here is a summary of my main takeaways:
1. Speculative Everything: Design, Fiction and Social Dreaming, by Anthony Dunne and Fiona Raby
Fiction is an important category in art and design since these disciplines have the power to make possible futures and alternative scenarios more believable. They go beyond the imaginary and have one foot in the field of the real.
2. Botanical Speculations: Plants in Contemporary Art, by Giovanni Aloi
In order to intervene into current attitudes towards climate change and sustainability, we must shift our perspective from viewing plants as passive objects and give them the agency to nurture and restore nature.
Secondary Research
These books along with other literature, scholarly articles, and media [all listed on my thesis bibliography] had a meaningful impact on this project and helped me discover a hypothesis. Some of the key insights based on my secondary research findings include:
Primary Research
I sent out a survey targeting my general audience–consisting of local artists, new media enthusiasts and creative-minded citizens–to learn more about their expectations, experience with art exhibitions and digital media. I got 36 responses. Here are the results that helped guide my design process:
Based on the survey results, I learned that tackling a big problem like ecological crisis requires collective effort. I began engaging and empathizing with people that would participate in the exhibition more deeply to understand their motivations. Then, I narrowed down the target audience and created the primary user persona.
User Persona
User Flow
Part of my MFA thesis requirements was to have an on-campus exhibition. However, I assumed it would be more effective to discuss the future of plants while being surrounded by them. Therefore, the exhibition was simultaneously taking place at Gallery 3, SJSU and Backyard–a pop-up community garden and event space near campus. There were flyers at both venues that invited people to participate in the workshops and visit the other location as well.
Storyboarding
Discussing technology aided plants of 2121 in a social group
Creating a prototype of technobotanic life and writing a fictional description about it
My part: collecting the digitized prototypes and combining them into a single .fbx file for HoloLens
Creating a story living experience by displaying a virtual garden of the future plants.
Journey Map
I wanted the viewers in either space (Gallery 3 and Backyard SJ) to begin their journey with a sense of wonder since the moment they arrive. The vivid pink light coming from the gallery room and the plants peeking through Backyard’s perimeter give away that initial sense of wonder. As the viewers start exploring what’s around them they should feel curious. The interactive, immersive art experience needs to captivate their interest. They will leave the space feeling inspired–more connected with the community and nature, having attempted to make a positive change.
By connecting people from the same community together in close contact with nature to share ideas and tell stories about a sustainable future, their overall awareness about current environmental issues will be improved.
Interconnecting with the Wood Wide Web – AR Mock-up
3D scanned prototype created by participant Mersal Zahir during a workshop session [view more here]
AR Design Consideration
Spatial user experience design requires a 3D mindset; this makes it very different from designing for 2D apps. Each wearable device has its own unique limitations–particularly with Microsoft HoloLens 1, it was the extremely narrow field of view (FOV). The hologram view is a quarter of normal binocular vision which is 120° wide, horizontally. Since the holograms viewed through the headset blend with whatever is behind them, color and contrast suitability were also key considerations.
Using Autodesk Maya to combine 3D scans of the virtual plant prototypes into a single .fbx file for displaying with HoloLens
Gallery Exhbition
My thesis show at Gallery 3 was ongoing from October 15-24, 2019. I displayed a collection of the virtual plants my workshop participants had modeled with Play-Doh along with their labels on shelves illuminated by a grow light. The growing of these future plant prototypes was metaphorical, but using the light transformed the ambiance of the gallery to a vibrant pink–creating an ethereal aesthetic.
Videos on Display
To give the gallery viewers a vision of what was happening at the Backyard or Botanical Greenhouse venues–if they could not make it there–I played a looped recording of the pop-up greenhouse on a vertical screen.
I dedicated one of the gallery walls to project the concept video artwork. This video is a mashup of found royalty free stock footage, 3D animated GIFs, and still images poetically visualizing a type of interconnection that uses the network of cell towers to connect with virtual plants.
AR w/ HoloLens
I used the 480nm HoloViewer app to display holograms on HoloLens. After loading the virtual garden and enabling spatial surfaces, I was able to switch in and out between the UI menu and presentation modes using voice commands. The spatial tracking mesh merges the real world with the virtual world by creating a triangular mesh mapped onto all visible surfaces. This feature allowed me to visualize the immense network of relationships that links human bodies to nature and their surrounding environment.
Holographic Greenhouse
The Hololens was placed on top of a pedestal inside the pop-up greenhouse (4'x4'x8'). I wanted the physicality of the greenhouse to have a digital feel to it, almost as if it is a 3D rendering, since it was where the immersive digital experience began. I built the structure out of PVC pipes and transparent iridescent plastic sheets that gradually change colors depending on the angle of view or the angle of illumination.
Once the dates of my exhibition at the first venue (Backyard SJ) were over, I moved the greenhouse to the San Jose State University Shade House for another week of public display.
Designing this project was a very rewarding and fun experience. I learned a lot about narrative design and the affordances provided by the Augmented Reality platform. With more time, I would have done more user testing and also develop my own AR app using Unity.
What I found most valuable about this work was the large-scale collaboration. As Spatial Computing technology becomes more mainstream and accessible, I hope that I can continue to create more opportunities for communication and solving problems collectively.
Leily Khatibi