New technological developments may give students a good excuse for using their phones during class. A team from the Tandon School of Engineering’s Mechatronics program is developing a new integrated platform called Mixed Reality Learning Environment that will allow students to control laboratory testbeds virtually via mobile devices.

Doctoral candidate Jared Frank — who specializes in augmented reality, mixed reality and educational technology — started research on MRLE in 2010 with the help of mechanical and aerospace engineering professor Vikram Kapila. Frank said the idea was inspired by his experience teaching a laboratory course at the Tandon School of Engineering.

“We realize that the students are really engaged by their phones,” Frank said. “They bring their phones to classes and get distracted in the middle of the lab activity [while] checking their emails and talking to their friends. Instead of taking their phones during the lab, what we can do is to take advantage of the engagement.”

Frank began putting the computational and interactive power of mobile devices to educational use. After one year of software development and stability tests, he conducted a two month-long experiment with two classes of undergraduate engineering students. The result of the evaluation on the efficacy of the new environment proves that it can enhance students’ learning experience.

“We recognize that the sensing, data storage, computation and communication capabilities of mobile devices have become sufficient for the measurement, control and interaction of laboratory platforms,” Frank said. “It led us to develop apps that exploit real-time computer vision, augmented reality graphics and touchscreen interaction to enhance laboratory learning.”

Kapila said that due to the high cost of testbeds and the number of students, during lab experiments, some students get less interaction with software and hardware than others. He said that the use of mixed reality devices was revolutionary for laboratory education.

“For several years we have been developing natural and intuitive mechanisms for allowing humans to interact with machines using devices such as tablets and smartphones,” Kapila said. “Now students can not only see video of physical [systems] but through augmented elements interact with those systems and affect their behavior in real-world.”

Tandon spokesperson Karl Greenberg said that the mixed reality environment not only helps students understand dynamic systems and control concepts better, but it also cuts the cost of lab equipment enormously.

“Thanks to their innovative way of thinking about mixed reality technology for human-robot interactions, it happens not to require costly and hard-to-calibrate laboratory-grade equipment, as it relies on the supercomputers that we carry about with us every day in our pockets and backpacks,” Greenberg said.

Kapila said that he will use the mixed reality system in his Tandon laboratory courses.

“We have pilot-tested [these] devices with students,” Kapila said. “Our goal will be to have the technology acquired by an established vendor of educational laboratory equipment. In the interim, we will use it in our own lab courses in my lab.”

Email Coco Wang at [email protected].