Augmented and Virtual Reality are relatively new and fast-growing categories of immersive technology. Aside from gaming, their primary use is in education. This project explores how VR could impact medicine, a field reliant on extensive hands-on training and known for having extensive and complex education requirements.

Research included scholarly articles, interviews with medical students and VR designers, and internship experience at Ghost Productions Medical Animation & VR Surgery.

Currently, leading VR surgery companies like Ghost Productions use a combination of hand-tracking and household VR controllers like the Oculus Quest. While hand-tracking is becoming increasingly accurate and efficient, it cannot replicate the feeling of a tool in one’s hand, and bulky handheld devices with trigger-like controls are not conducive to an accurate simulation.

An example of how this issue has been addressed in the art and design category is the development of stylus pen tools, for both a 2D screen (Wacom, Huion, Apple Pencil) and 3D VR space (Logitech VR Ink Pilot).

Medical students feel underprepared with the amount of practice they receive. Physical demonstrations of procedures have major limitations:

Cost:
‍Cadavers, lab space and equipment, opportunity cost of practicing on real machines, and instructors cost money.

Accessibility:
‍Demos occur at scheduled times and locations. Students cannot repeat demos to their own liking.

Accuracy:
‍Practice scenarios feel unrealistic or lack nuance.

Diversity:
‍Using a "default" patient makes it hard to prepare for different pathologies.

To address this gap, I envisioned a concept for a set of VR controllers that would correspond to the main structural categories of surgical tools.

Comparisons

01 Stylus

• Scalpel
• Electrocautery pen
• Curette
• Syringe

02 Hinged

• Cutters
• Forceps
• Axial laparospic handle

03 Ringed

• Scissors
• Clamps
• Shank laparoscopic handle

The controllers are designed considering the ergonomics of different ways each tool can be held. For example, the button on the stylus tool can be pressed with either the thumb in a scapel pencil grip or the index finger in a scalpel palm grip. The top button can be pressed like a syringe. The hinge tool can be held like tweezers, or reversed like an axial laparoscopic handle. The controllers have slanted top surfaces to house the IR lights for a VR headset to read easily at multiple angles.

I created physical prototypes iteratively and made adjustments to the design after each one, focusing first on ergonomics and then on VR capabilities.

Version 1: Foam

Version 2: Wire and Clay

The hinged controller only accounted for the mechanic of uni-directional force (closing), whereas some medical instruments allowed for bi-direction force (opening and closing). To accomodate this, I introduced a loop controller.

I 3D modeled my final set of controllers in Autodesk Maya for production.

The controllers were printed in white PLA. I encountered difficulties printing a living hinge with enough tension for the hinge controller, so I printed it as separate pieces which I then connected with a coiled 18-gauge wire coated in white foam clay.