May 17, 2019, by Matt Davies

Navigating the world of VR & 3D: Aja’s Virtual Reality Experience – Project Overview by Alice Stapleton

Since early 2018, DTH’s Leonardo Fellow, recording and performance artist Aja Ireland has been working to create a virtual reality experience with a team of Nottingham based artists, DTH staff and student volunteers. Aja isn’t afraid to push the limits, creating music and live performances that she describes as ‘Experimental Rhythmic Noise’.  This project has been no different, creating an immersive and engaging VR environment that pushes the boundaries of live performance and audio experience.

The team has achieved a lot since the project’s launch in early 2018, so let’s recap the progression that the team has made over the last year.

February 2018– The project launches and the excitement of building a VR world begins. Artists Jake Moore and Joey Holder, and DTH Research Associate Paul Grossman join Aja to start work on the VR experience using Unity to build the environment and Blender to create some of the flora and fauna!

October 2018– by this point Paul had built up a lot of knowledge and experience with Unity working on the foundations of Aja’s experimental VR world but sadly for us had graduated in June taking all of that knowledge and experience with him! However, as we said goodbye to Paul, we welcomed a new cohort of volunteers willing to learn about virtual reality but also techniques in 3D capture.

With the foundations of the VR environment already in place, the goal for the next stage of the project was to capture 3D scans of the incredible outfits that Aja wears for her performances. The costumes are conceived, designed and made by Lu La Loop  in collaboration with Aja and are an integral part of her performance, so scanning them and then morphing them into features of the alien landscapes was essential.

February 2019– Aja and the DTH team learnt some new skills at a photogrammetry session with Dimitrios Darzentas from the Mixed Reality Lab. Dimitrios covered the basic techniques of photogrammetry which uses a DSLR camera to capture an object from a number of different angles. The high resolution images are then ran through a number of processes in a programme called Agisoft to create a 3D version.

We also welcomed artist Ryan Heath to the team, who brought with him much needed Unity skills. Ryan comes from a fine art background, but decided to move into the digital sphere. His love for digital arts stemmed from a frustration with the static images that he was creating, and a realisation that more job prospects lie within the digital world. He gained his knowledge and expertise of using Unity on a virtual reality course at QUAD, digital arts centre in Derby. Ryan currently works at BACKLIT gallery in Nottingham and runs an art space there called Chaos Magic with co-team member Joey Holder.

March 2019- The team tried different techniques to capture Aja’s costumes in 3D. DTH manager Matt Davies and Research Associate William Pidzamecky made some brave- Heath Robinson inspired -attempts (using a green screen, re-purposed sound baffle board and projector stand!) to capture the costume using the photogrammetry process -as described above. Unfortunately the intracacies of the elaborate costume, and nature of the fabric – along with a fairly complicated and time consuming process -proved unsuitable in this case.

Eventually the team found that the quickest and most effective way to capture the costumes in 3D was to use one of the DTH’s recently purchased Structure Sensors. The Structure Sensor, developed by California based Occipital, is a 3D scanner which attaches to an iPad. It uses a system of structured light projection and detection to create the shape and distance of objects, which is then combined with images taken by the ipad’s own camera provide a pretty accurate 3D model. The results are less accurate and of a lower resolution than the photogrammetry process so less open to post capture processing, but are a lot faster and work well with Unity. Finally Ryan and Aja were able to export the 3D versions of the costume into the VR environment and the costumes now stand as weird and wonderful mountain landscapes in Aja’s VR world!

April 2019; Aspects of the VR world were used as parts of the projections that make up Aja’s joint installation with Annie Tadne at Brewhouse Yard as part of Nottingham’s annual Cave City Festival.

May 2019; Ryan Heath completes a trailer for Aja’s VR world which features the outfit as well as virtual creatures created by Joey Holder (using blender) and sound recordings by Aja. View the trailer here. 

Watch out for DTH student volunteer Alice Stapleton’s upcoming blog piece which will feature an interview with Aja Ireland.

Find out more about the Structure Sensor here.

• Share on Facebook
• Share on Twitter
• Share on Google+
• Share on LinkedIn