The SynXR (Synesthesia XR) Lab explores Human-Computer Interaction (HCI) at the intersection of synesthetic experiences and performance art in extended reality (XR) environments. Through an interdisciplinary approach combining Design, Computer Science, Psychology, and Performing Arts, the lab investigates how multi-sensory integration and cross-reality interactions can enhance human perception, creativity, and artistic expression in digital spaces.
Hi, I am Shuo Yan, an Associate Professor at School of New Media Art & Design / State Key Laboratory of Virtual Reality Technology and Systems in Beihang University. I received my Ph.D. in Intelligent Digital Performance from Beijing Institute of Technology, during which I was a visiting Ph.D. student in the Fluid Interfaces at MIT Media Lab. I received the Outstanding Doctoral Dissertation Award from China Simulation Federation, the Outstanding Graduation Project Supervisor Award from Beijing Higher Education Institutions, and the Xiaomi Young Scholar Award.
My research focuses on creating novel immersive XR experiences. I have published more than 30 papers and 20 artworks at the top international HCI/XR conferences.
⭐ Looking for highly motivated students and research interns. 招收硕士生及暑期/日常学生科研，有意请联系本人邮箱
👩🏻🏫 Teaching
- VR Interaction Design 虚拟现实交互设计（本科）
- Information Visualization Design 信息可视化设计（本科）
- Human-Computer Interaction and User Experience Design 人机交互与用户体验设计（研究生）
- Fundamentals of Digital Media Design 数字媒体设计基础（本科）
🧪 Research Foundations
- Humanities and Social Science Fund of Ministry of Education of China, Project: XR-Based Intelligent Performance Perception Generation and Interaction Collaboration(2022-2025) 教育部人文社会科学基金规划项目
- National Natural Science Foundation of China, Project: Research on Affective Evaluation and Content-Based Adaptive Feedback for Intelligent Interactive Performance from Audience Perception(2019-2021) 国家自然科学基金青年项目
- China National Arts Fund, Project: Construction of Digital Promotion Platform for Intangible Culture Heritage (ICH) ‘Hua er’ (2018-2020) 国家艺术基金传播推广项目
- National Key Research and Development Program of China, Project: Yunnan Minority Dance Digital Performance Creativity and Display System Based on VR (2019-2024) 云南省科技厅重点研发项目（子课题）
- Independent Project of the State Key Laboratory of Virtual Reality Technology and Systems, Project: Research on Innovative Design of Immersive Performance under Human-AI Collaboration(2024-2026) 虚拟现实技术与系统全国重点实验室自主课题
🔔 News
- Dec, 2025 · SIGGRAPH ASIA Conference, 📍HongKong, China
- Nov, 2025 · ICXR Conference XR Gallery Chair, 📍Qingdao, China
- May, 2025 · CHI Conference, 📍Yokohama, Japan
📖 Publications
Conference From Body to Interface: Co-design Gesture-based Interactions with Experts for Immersive Experiences
Zixiao Liu, Shuo Yan, Haiyan Jiang, Fengyi Yan and Henry Been-Lirn Duh
- We developed a co-design framework and apply it to a foot-based interaction project exploring habitual foot actions as natural inputs for virtual-physical switching experience.
Conference Exploring and Co-Designing Typical Collaborative Scenarios in Mixed Reality: A Role-Based Approach
Fengyi Yan, Shuo Yan, Wensi Dai, Likun Zhao and Zixiao Liu
- We proposed a comprehensive Role Interpretation Framework based on three core interpersonal dimensions—Dominance, Sociability, and Task Orientation—to systematically capture and explain user roles in MR collaboration.
Yu Wang, Yulu Lu, Shuo Yan, Xukun Shen
- We developed the MyAIPal prototype system, which enables users to interact with AI agents of personalized everyday objects through voice dialogue and natural behaviors.
Conference Beyond Realities: Designing Cross-Reality Interactions in the Immersive Narrative Experience
Zixiao Liu, Shuo Yan, Wenjie He, Xiaomeng Wan, Xukun Shen
- We designed an immersive narrative “Beyond Realities”, and implemented four CR interactions in it: Portal Passthrough, Virtual Guidance, Object Passthrough, and Bystander Avatar.
Conference MetaMood: An AI-based Shared Emotion Visualisation in Immersive Healing Spaces
Fengyi Yan, Siyu Luo, Shuo Yan, Xukun Shen
- Metamood is an AI visualization system that automatically generates visual expressions of emotions in group art therapy.
Conference “Achi Mukua Experience Space” —— A Multi-Player Immersive Experience of Yunnan ICH Dance
Siyu Luo, Shuo Yan
- We develops design methods for ICH dance that combine social presence, collaborative multiplayer interaction, and emotional feedback to enable the co-creation immersive experiences.
Conference NoPhobiar: Designing A VR Game to Prevent Childhood Dark Phobia with Children and Stakeholders
Xinyi Su, Shuo Yan
- We developed a VR game for treating dark phobia in children aged 4-6 through participatory research involving children and stakeholders to identify design requirements.
Yuetong Zhao, Shuo Yan, Xuanmiao Zhang, Xukun Shen
- We designed a wearable gesture-based interface for external users in two main asymmetric VR scenarios: (1) Object Transition, (2) Collaborative Game.
Zixiao Liu, Shuo Yan, Yu Lu, Yuetong Zhao
- We proposed “Hua’er and the Youth,” an interactive VR system integrating virtual avatar, participatory performance, and game-based knowledge acquisition for experiencing Hua’er performance.
Conference Effects of Augmenting Real-Time Biofeedback in An Immersive VR Performance
Saixi Ge, Siyu Luo, Shuo Yan, Xukun Shen
- We proposed a support system to investigate how performer’s heart rate (HR) and audience’s electroencephalogram (EEG) visualizations, and the corresponding auditory feedback, influence audience’s sense of perception, immersion, and the narrative understanding.
Conference Performing with Me: Enhancing Audience-Performer Interaction in An Immersive Virtual Play
Shuo Yan, Saixi Ge, Jinhui Wang, Xukun Shen
- We presented a novel immersive theater-based approach for enhancing audience-performer interaction in virtual plays through individual-based, scenario-based, and narrative-based interactions.
😈 XR Demos and Artworks
Zhu Zhu, Shuo Yan
- This project integrates Chinese Sign Language into immersive interaction, enhancing expressiveness and inclusivity while enabling barrier-free communication that bridges Deaf and hearing communities beyond natural gestures and instruction.
Conference Bamboo Rhyme Craft: Immersive VR Bamboo Weaving through Coordinated Bimanual Gesture Interaction
Chenyue Zheng, Yaxuan Zhao, Runxin Guo, Shuo Yan
- Immersive VR with coordinated bimanual gesture interaction revives traditional bamboo weaving, blending cultural heritage with engaging, hands-on experiences to promote, preserve, and creatively reinterpret intangible traditions for wider audiences.
🏆 Honors and Awards
- 2024 Mentor Award
- 2023 Winner, Second Prize of Outstanding Innovation Cases of Aesthetic Education in Beijing Universities，Beijing Education Commission
- 2023 Mentor Award, BUAA Excellent Master’s Thesis BUAA
- 2022 Grant, $14000 Research Funding from Xiaomi Youth Scholarship Xiaomi
- 2022 Grant, $7000 Research Funding from Social Science Youth Scholarship BUAA
- 2021-2022 Mentor Award, Outstanding Undergraduate Graduation Design (Thesis)，Beijing Education Commission
- 2019 Winner, Third Prize of “Teaching Design” in the 7th National Collegiate and Universities Digital Art and Design Competition Ministry of Industry and Information Technology (MIIT)
- 2017 Winner, Outstanding Doctoral Dissertation Award of China Simulation Federation(CSF)
- 2015 Fellowship, National PhD Scholarship in China Ministry of Education of China
📚 Educations
- 2013.09 - 2014.09, Visiting PhD Student, MIT Media Lab.
- 2012.09 - 2017.06, PhD in Software Engineering, Intelligent Digital Performance, BEIJING INSTITUTE OF TECHNOLOGY
- 2006.09 - 2010.06, Bachelor of DESIGN, Industrial Design / Business Administration (Double Degree), BEIJING INSTITUTE OF TECHNOLOGY
💬 Invited Talks
- 2025.11, Research on Innovative Design of Human-Computer Collaborative Intelligent Performance, , Dance Research Institute, China National Academy of Arts.
- 2019.09, TEDxChengdu