November 24, 2023, by Laura Nicholson

Medicine and Health Sciences Faculty Takeover: Microsoft HoloLens 2 and Escape Rooms

Expanding Horizons: Opportunities for Teaching and Learning with HoloLens 2

Throughout the 2023-4 academic year, we are running a new feature on the LT blog, a Faculty Takeover month! Each month (excluding December), we will feature posts from a different faculty at the University. Every Friday, posts will highlight interesting work and ideas related to technology in teaching and learning and showcase unique projects from within the various disciplines across UoN. In October, we had our Faculty of Science Takeover, this month we are promoting Medicine and Health Sciences (MHS).

As a Learning Technology Officer for MHS, I have decided to share some of my recent experiences with the HoloLens 2 and some ideas on utilising it for teaching and learning.

 

What is the HoloLens?

The HoloLens is a mixed reality headset that combines augmented reality (AR) with virtual reality (VR), allowing users to interact with holographic images and digital content in their real-world environment.

 

HoloLens for teaching and learning

Previous research examining the effectiveness of using a HoloLens as a teaching tool have yielded positive results relating to learning and retention (Stackhouse et al.,2023; Bala et al.,2021). Popular uses of the HoloLens in health education are to practice medical techniques in simulation environments (Speare-Cole, 2022) or to use the headset for direct teaching. For the latter, the lecturer wears the HoloLens, which then livestreams to the students remotely, allowing them to interact with the tutor and patient without even being in the remote vicinity. The HoloLens lends itself particularly well to training in anatomy and diagnostics (Cartucho et al.,2020; Maniam et al., 2020) and in pathology education (Hanna et al., 2018).

 

Initial investigations

My initial focus for the HoloLens2 was to explore alternate ways a HoloLens could be used in the absence of apps and direct lecturer input. To investigate this further I trialled different platforms and activities to see how well they worked when integrated with the HoloLens. Some didn’t quite go according to plan, mainly because they were too fiddly to use with the HoloLens control gestures. The other consideration was the keyboard, which you input into using one finger at a time, making it quite time consuming. Therefore, any plan I had in terms of how to use HoloLens needed to avoid activities requiring too much typing.

So, after a bit of trial and error, I reached the conclusion that the HoloLens 2 would be excellent for integrating gamification, particularly in the format of escape rooms. A little research into my idea revealed that using a HoloLens for an escape room has been trialled by others, with some inspiring results. So I decided to delve deeper to investigate how everything could be applied in practice.

 

What are escape rooms?

An escape room involves the participants solving a series of puzzles using clues and hints to achieve an end goal. The game requires the participants to solve clues before they can unlock the next task and move on to the next area.

The rooms are designed to immerse the participants in a unique and challenging environment, and the puzzles can vary widely in terms of difficulty and complexity. The design will help develop skills relating to collaboration, communication and critical thinking skills, in an engaging and enjoyable way.

 

Key considerations

If the creation of escape rooms was to be an effective way to use the HoloLens, it became evident that whatever was developed would need to be versatile and support others in creating their own escape rooms. Given the time-intensive nature of creating escape rooms, something in short supply already for lecturers, I quickly recognised that templates were the most practical approach. These templates could encompass various gamification elements, allowing lecturers to input their scenario into them, with the activity side of it already prepared for immediate use.

The other key consideration was that escape rooms are very much collaborative problem-solving experiences; how could other students be included? I discovered the answer to this would be to live stream the HoloLens to a screen so those not wearing a headset could still input and engage with the content.

 

What gamification elements would work?

Having had some experience with the HoloLens, I am mindful that mastering the gestures to control the HoloLens may initially pose a challenge. The headset is intended for student use, so it should ideally require minimal manipulation of objects and windows to ensure ease of use right from the start. This led me to the idea of incorporating pre-set windows with information to streamline the experience and reduce the need for training in the equipment before use.

In terms of overall design of an escape room, numerous approaches can be taken but here’s one suggestion. You could set up a room, maybe a lab or ward, and position various virtual ‘windows’ around the room which contain information or have problem-based activities to solve. The problem-solving activities could come in many forms, such as crosswords, mathematical equations for code-breaking scenes to lead to a new window, multiple choice questions or image hotspots. Needless to say there are so many innovative ways to embed collaborative problem-solving activities into a case study scenario and the result could be a highly engaging activity that proves reusable over time.

 

Potential escape room features

Initial set up

Further options

Set up quick links to web resources     

The escape room can have different windows pre-loaded for the HoloLens user. In this example, a page has been created which has a web link embedded to enable quick access to other online resources that the HoloLens wearer can quickly and easily access. Images can also be set up to have hotspots, which are compatible with the HoloLens interactions. 

 

 

 

 

      

Interact with 360 images

In this example, a 360 degree image can be manipulated to view the different angles using the HoloLens gestures. Audio can also be recorded and added to the image for more explanation and context.

 

 

 

 

 

Play videos

Another Window could show a video explaining a concept or process. The HoloLens 2 user will receive the audio input that can also be projected onto a linked screen in the room to enable everyone to access the content.

 

 

 

 

 

Solve the clues in a crossword

Why not add some gamification through crosswords or multiple choice quizzes? The escape room could potentially be set up to prevent the user from progressing to the next window until they have correctly answered the clues.

For crosswords, the HoloLens wearer points to the area they wish to input an answer and a virtual keyboard appears. Alternatively,for multiple choice questions, there could be the requirement to calculate equations or predict doses according to a given scenario.

 

Conclusion

I have only had time to showcase a small fraction of the numerous ways in which the HoloLens can use escape rooms for teaching and learning. Many more possibilities await and I’m eager to discover the perfect combination of options to create my very own escape room!

Are you interested in incorporating the HoloLens into your teaching?

The faculty of Medicine and Health Sciences is always on the lookout for staff interested in using the HoloLens in their teaching, either to develop escape room scenarios or for anything else. If this is something you would like to find out more about, please do get in touch by emailing laura.nicholson@nottingham.ac.uk

 

Reference list

Bala, L., Kinross, J., Martin, G., Koizia, L.J., Kooner, A.S., Shimshon, G.J., Hurkxkens, T.J., Pratt, P.J. and Sam, A. H. (2021) ‘A remote access mixed reality teaching ward round’. Clin Teach. 2021;18:386–390. doi: 10.1111/tct.13338

Cartucho J., Shapira D., Ashrafian H., Giannarou S. (2020) ‘Multimodal mixed reality visualisation for intraoperative surgical guidance’. Int. J. Comput. Assist. Radiol. Surg. 2020;15:819–826. doi: 10.1007/s11548-020-02165-4

Hanna, M.G., Ahmed, I., Nine, J., Prajapati, S., Pantanowitz, L. (2018) ‘Augmented reality technology using microsoft hololens in anatomic pathology’. Arch. Pathol. Lab. Med. 2018;142:638–644. doi: 10.5858/arpa.2017-0189-OA

Maniam,P., Schnell, P., Dan,L., Portelli, R., Erolin,C. and Mountain,R. (2020) ‘Exploration of temporal bone anatomy using mixed reality (HoloLens)’, Journal of Visual Communication in Medicine, 2020 doi: 10.1080/17453054.2019.1671813

Speare-Cole, R. (2022) ‘UK medical students training on hologram patients in world-first’. Independent, Monday 27th June 2022. Available at: https://www.independent.co.uk/news/health/cambridge-university-of-cambridge-los-angeles-b2110392.html (last accessed 17/11/23)

Stackhouse, A.A., Rafi,D., Walls, R., Dodd, R.V., Badger, K., Davies, D.D., Brown, AC.A., Cowell, A., Meeran.K., Halse, O., Kinross, J., Lupton, M., Hughes, E.A and Sam, A.A. (2023) ‘Knowledge Attainment and Engagement Among Medical Students: A Comparison of Three Forms of Online Learning’, Advances in Medical Education and Practice. 2023;14:373-380 DOI: 10.2147/AMEP.S391816

 

• Share on Facebook
• Share on Twitter
• Share on Google+
• Share on LinkedIn