What once only existed inside Star Trek’s fantasy world is now becoming a reality thanks to NYU researchers; the development of the Holodeck — a staging environment in which participants can engage with various virtual reality environments — has received a multi-million dollar grant from the National Science Foundation and NYU.
Agnieszka Roginska, who studies Music Technology at the Steinhardt School of Culture, Education, and Human Development, will be able to experience the Holodeck’s various environments first hand. While she currently has a lab that can recreate a range of sounds through 20 loudspeakers and two subwoofers, once the main Holodeck is built, she’ll have access to four times that amount of equipment.
“You want to be in the jungle, we’ll make it into a jungle,” Roginska said. “You want to be in a concert hall, where [there are] musicians with you and musicians somewhere else, we’re going to make it a concert hall. We can make it sound like any other environment, we can make it look like any other environment.”
This is an intensely interdisciplinary pursuit; researchers from the Courant Institute of Mathematical Sciences, the Rory Meyers College of Nursing, Steinhardt and the Tandon School of Engineering are all participating. Jeremy Rowe, a senior research scientist at the NYU-X Lab with STEM Nursing, explained the different technologies that will be used to gather data.
“We’re in the process of putting together the various components that can be used to study the way that people behave and interact in a virtual environment,” Rowe said. “There’s a visual component, augmented reality, virtual reality, mixed reality projection and spatial audio. There will be thermal and haptic feedback, individual ability to look at physiological measures, things like galvanic skin response, eye-tracking, and a variety of things of that sort.”
The measures Rowe mentioned will let scientists know how a subject’s body is responding and processing their simulated environments. Components like thermal and haptic feedback are what will allow subjects to receive responses to the heat of their bodies and force of their touch. For example, Rowe has done research concerning VR surgery games that use the kind of haptic responses one gets from a game controller to mimic performing surgery. The result would be faster and more efficient training for surgeons.
Professor of Computer Science at NYU Ken Perlin says the NSF was mostly interested in this question of potential uses and reactions to VR by humans. To explore this, the Holodeck will also be equipped with a large range of technologies that will allow research teams to collaborate and create a wide set of data to probe the question of how humans will react to and use virtual reality technology.
“What will the future of people being together in person, in the same room, be like, as technology allows us to enhance our sensory experience?” Perlin asked. “We’re looking at the time when cyberspace will be right here between us.”
A version of this article appeared in the Monday, Nov. 19 print edition. Email Shanti Escalante at [email protected].