Language Learning with VR and AI 

The project was co-developed by the iTLAB team—David Lasala (Director) and Andrew Stapp (Experiential Designer)—in close partnership with Professor Venetsanos to ensure effective course supplementation and pedagogical alignment. Students attended guided sessions at the iTLAB, where they learned to use the Meta Quest 3 XR headset to explore immersive language learning. Experiential Learning Activities: 

• Mondly: Offered structured language-learning scenarios, such as riding in a taxi or engaging in travel-based conversations.

• AI-Powered Avatar Experience: Enabled students to engage in freeform conversations with an AI avatar that dynamically responded in both Modern Greek and English.

The primary goal was to create an interactive, immersive learning experience that improved language acquisition and retention. Specifically, we aimed to:

• Develop a technology-driven, student-centered approach to language learning.

• Enable students to practice speaking skills, review mistakes, and learn in a low-risk learning environment.

• Build student confidence by allowing them to practice speaking without fear of errors.

We expanded our engagement goals by providing students with immersive simulations that allowed them to practice language skills in realistic scenarios, including:

• Train Simulation: Interacting with a fellow passenger and resolving a ticket issue with a conductor.

• Restaurant Simulation: Ordering a meal and initiating a conversation.

• Hotel Simulation: Checking in and interacting with a maître d' and a bellhop.

A core objective of the iTLAB is to equip students with foundational skills in XR technology, including:

• Introduction to XR: Familiarizing students with a mixed reality headset.

• Basic VR Skills: Teaching students how to optimize their VR experience for comfort and immersion.

• Best Practices: Demonstrating effective VR usage to help students confidently navigate immersive environments.

Anticipated Challenges and Mitigation Strategies:

• Technological Barriers: Most students lacked VR experience. To overcome this, the iTLAB provided structured onboarding and technical support during sessions.

• Pedagogical Alignment: Ensuring that VR activities met course objectives required close collaboration with Professor Venetsanos. This was achieved through continuous feedback and agile adjustments.

• Resource Constraints: Limited VR equipment was managed through careful scheduling, maximizing access for all students.

Results from Anna Venetsanos post-survey and iTLAB feedback form. 

• “It was really fun and innovative.” 

• “I really enjoyed it and liked the change in classroom environments. Switching things up every once and a while made learning more fun.”

• “It was great. I had a lot of fun using it and was engaged more by actually speaking with people. It was a great way to make me more comfortable speaking.”

• “It was very helpful for learning and an engaging way to practice conversational Greek.”

• “I had a great experience practicing language conversation skills in an interactive, realistic environment. I definitely learned from it and felt more comfortable speaking out loud.” 

• “I can't wait to see where it goes in the future and hopefully we can continue it in Intermediate I and II.“

Outcomes: What could be improved?

• “I wish we had 30 minutes of uninterrupted time in the different apps.” 

• “I think that the audio/mic-calibration could be improved as well as the answers of the AI, but it is very good as it is right now.”

• “I think that the graphics and audio could be improved, specifically so that the VR character picks up on what we are saying more accurately.”

• “I think some technical issues could be resolved in terms of volume/hearing what I was saying. It also was limited in terms of the vocabulary it knew, but it improved more towards the end of our sessions, which was great.”

• “The uppercase and lowercase when analyzing what we're saying could be more accurate. Everything else was amazing, and there is not much you need to change.”

Key Takeaways And Conclusions

• Immersive Innovation: This project reflects NYU’s commitment to pedagogical innovation by integrating VR into Modern Greek instruction, creating a more engaging learning environment.

• Scalable Model: The successful implementation demonstrates how immersive technology can be scaled across other language courses.

• Institutional Alignment: By integrating immersive technology into language learning, the project supports NYU’s commitment to academic excellence.

• Path Forward: Building on this foundation, future iTLAB collaborations will continue to expand XR applications in language learning.

The Future For XR Language Learning at the iTLAB and NYU

Building on the success of this Modern Greek VR initiative, the iTLAB is poised to expand its reach and impact. Key next steps include:

• Advancing AI-XR Language Learning: Develop XR applications that support conversational practice, pronunciation feedback, and contextual language learning across additional languages.

• Scaling Immersive Language Programs: Leverage more of space to support larger classes and expanded language course offerings.

• Building Capacity: Invest in XR software and hardware, grow the team, and strengthen long-term staff retention to ensure sustained innovation and service delivery.

Acknowledgments

We extend our sincere thanks to the students who participated in this project. Their curiosity, enthusiasm, and thoughtful feedback were instrumental in shaping and refining this immersive language-learning experience.  Their engagement not only brought the technology to life, but helped shape its future at NYU.