This week marked the start of a short project on experiential storytelling, memory, and light. We were asked to think about three memories in which lighting was an important factor, and write a short description of this moment. Emphasis was put on the word moment - this is not meant to be a life story. The idea is to bring the viewer into this moment, understand what it is that’s happening, and then exit in 15 seconds.
I started thinking about memories that had specific focus on lighting, and found it was more difficult than expected. There were plenty of memories where I could remember what the lighting was and appreciated it, but I had to find three where it really stood out to me. I found that when writing about them, I was walking a fine line between what I’m saying to the viewer and what they’ll actually be seeing in the experience. The descriptions were going to be recorded and part of the audio. For each chosen memory I already had a vague impression of what I wanted to accomplish, the most difficult part was deciding how much visual detail I would be including along with narrative, and how specific that narrative would be.
Chosen Path
“I was looking for Orion. He’s there, as always, but tonight he brought friends to fill the usually empty sky. Standing barefoot on the deck with only the glow of the candle, we stared at each other over the hills, making introductions.”
This memory is from last May, standing on the deck of our house in North Carolina. My partner and I drove down from Columbus for my birthday. The area isn’t very populated, mostly woods - on a clear night it’s easy to see all of the stars. We stood outside the first night we got there with all the lights out except a candle just looking at the stars. As a child I would always look for Orion every time I walked outside, it was the only constellation you could usually see from where we lived in Miami. The light from the stars, the candle, and the houses on the other hill really stand out to me in that memory, and I chose it because I feel that I could bring the essence of this moment to a viewer with varying levels of abstraction.
Panorama off the deck at North Carolina at sunset. Original scene where the memory takes place.
Process
In the past, my research has required virtually the same pipeline visually every step of the way. Block modeling in Maya, some texturing in Substance Painter (occasionally), and then putting it in Unity and adding lights. The focus was on making the program itself function rather than impart an experience visually. I want to take a step back and create something that imparts meaning without necessarily requiring the viewer to actively be a part of it.
Oculus Quill presents some really interesting opportunities for animating in virtual reality. I spent some time looking around and finding examples of these animations that might be similar to my own.
Artist Goro Fujito spends his time creating animations in Oculus, showing a variety of scenes and perspectives. Viking Rockstar is a great example of the type of color and lighting I want to use in my own scene, includes multiple shots and sound design. I wouldn’t categorize this as a virtual experience, but as an animation it’s beautiful and on the right track stylistically.
This short looped animation puts the viewer in the perspective of driving through the rain. With the sound design and lighting, it’s incredibly effective and shows how the user can be brought into an experience.
Fortunately, Fujito also posts videos where he shows his process and explains his animation workflow. I watched this to get a better understanding of how Quill functions and if it would be a good option for me moving forward.
The official website provides some resources on how to export animations and FBXs to Unity, though I needed to look externally for information on how to do this in Unreal Engine. I was considering using UE4 specifically for its lighting capabilities. I worked on the lighting for Project Sphincter while at CCAD, and Unity just hasn’t been able to compare. As of now, I am leaning towards this option.
Putting the final software choice aside for a moment, I decided to get into Quill and see if this was really something I wanted to commit to. Granted, I’ve spent maybe a grand total of 3 hours in it and probably need to watch some more tutorials, but the learning curve is pretty rough. I had a difficult time getting a hang of the controls, which are not well explained when first entering the program beyond a little diagram that pops up by default. These were the initial sketch results:
Next Steps
After spending some time in the Oculus, I don’t think it’s practical to do the entire scene in this way. At least, not from scratch. I need to investigate bringing in models and animating over top of them, possibly as reference. It’s very difficult to gauge depth in there once the scene is moved around. I also need to look into animating only certain objects in the scene with quill rather the entire environment, or blending them together. This will help me determine a production schedule for the next two weeks.
Beyond the pipeline research, I will spend this next week gathering my final visual reference, sketching out a storyboard, and recording my story for timing. I have been gathering some information on lighting and technical terms in order to actually discuss the decisions I’m making on the lighting in the scene, and will be discussing that more next week as well.