Privacy-Preserving Mapping and Localization for Immersive Health Care Applications

Project Investigators

Principal Investigator (PI): Bo Han, associate professor, George Mason University Department of Computer Science

Co-PI: Songqing Chen, professor, George Mason University Department of Computer Science

Co-PI: Hong Xue, associate professor, George Mason University Department of Health Administration Policy

Rationale and Background

Mobile spatial computing takes physical actions such as head and body movements, gestures, and speech and presents it as virtual content on devices’ visual cortex through head-mounted displays.

 The rise of augmented and mixed reality (AR/MR) technologies advances spatial computing into the three-dimensional (3D) space, which relies on innovations in vision-based spatial mapping and image-based localization. 

When immersive applications are used, the maps and visual features could contain sensitive information about the surrounding environment.

If attackers or malicious users get access to this visual data, they can reconstruct the original images/scenes with a high degree of accuracy. Robust mechanisms are needed to protect user privacy. 

Methodology

• Privacy-preserving image-based six degrees of freedom (6DoF) pose estimation/localization. Researchers will design and train a lightweight machine-learning model to rank extracted features on mobile devices, then select features based on their importance.
  
• Preserving privacy for spatial maps constructed via structure from motion. Researchers will introduce perturbation noises to a spatial map to impede the reconstruction of original scenes. 
• Implementation and evaluation. Researchers will create a holistic system for dietary monitoring, implement a prototype, and evaluate its performance through in-lab experiments and user studies.

Projected Outcomes

• White papers and guidance on preserving privacy for vision-based and immersive health-care applications.
• Publication of invented algorithms and experimental results at conferences and refereed international journals.
• Demonstrations of a proof-of-concept at conferences and outreach events.
• Open-source software, patent applications, and proposals for external grant opportunities.