Optical see-through augmented reality (AR) glasses are designed to overlay digital information onto the real world, but improving the realism of virtual objects often comes at a cost. Existing occlusion technologies physically block part of the incoming light, reducing the brightness of the real environment and making everyday tasks more challenging. Overcoming this balance between virtual realism and real-world visibility is a key hurdle for next-generation AR systems.

Addressing this challenge, a research team including Assistant Professor Xiaodan Hu, the key contributor to the project, from Shibaura Institute of Technology, Japan, in collaboration with Dr. Yan Zhang and Professor Xubo Yang from Shanghai Jiao Tong University, China, and Professor Kiyoshi Kiyokawa from the Nara Institute of Science and Technology, Japan, developed a perception-driven display strategy that jointly considers human visual perception and real-world lighting conditions. Instead of relying solely on hardware improvements, the researchers introduced a mask balancing method that dynamically adjusts the visibility of real and virtual scenes according to what users can actually perceive.

The proposed method combines a polarized component that supports pixel-level occlusion with another polarized component that bypasses the optical system and preserves the natural brightness of the real world. By adjusting the cross-angle between a polarizing beam splitter and a linear polarizer, the system dynamically blends these two views. Real-time eye-tracking and scene analysis estimate the visibility of both the environment and virtual objects, allowing the display to continuously optimize the balance between them.

To establish perceptual thresholds, the researchers conducted a series of user studies. Experiments involving 12 participants quantified how much contrast was required for users to recognize textures in virtual objects, while another study with 24 participants evaluated the perception of lighting effects. The team then integrated these findings into a dynamic balancing strategy and validated it with a benchtop prototype. A final user study with 12 participants demonstrated that the system improved real-world visibility while maintaining a convincing appearance for virtual content across different illumination conditions.

“Our research demonstrates that human visual perception can be used to dynamically balance the visibility of the real world and virtual content,” stated Prof. Hu. “By adapting the display according to what users can actually perceive, our method improves real-world visibility while preserving the appearance of virtual objects under different lighting conditions.”

The study also highlights a broader shift in AR display design. Rather than optimizing optical hardware alone, the researchers show that understanding how people perceive visual information can lead to more effective display control strategies. This perception-driven approach could help future AR glasses present virtual objects that blend more naturally into the real world while maintaining user safety and comfort.

Potential applications extend well beyond consumer electronics. Future optical see-through AR systems could support industrial maintenance, medical assistance, education, navigation, and remote collaboration, where users must simultaneously monitor their surroundings and interact with digital information. By preserving both real-world awareness and virtual image quality, the technology could make AR devices more practical for demanding real-world environments.

“In our previous research titled ‘Perception-driven soft-edge occlusion for optical see-through head-mounted displays,’ we found that human perception of AR displays can differ significantly from predictions based solely on optical measurements,” stated Prof. Hu. “This inspired us to explore AR displays from a perceptual perspective because future AR glasses should be designed not only according to optical performance but also according to how humans actually perceive visual information.”

Overall, the researchers believe that integrating perception science with display engineering may help overcome one of the most persistent barriers to widespread AR adoption. By leveraging human visual characteristics instead of depending entirely on hardware improvements, future systems may achieve more natural, photorealistic, and user-friendly AR experiences.

Journal: IEEE Transactions on Visualization and Computer Graphics
DOI: 10.1109/TVCG.2026.3679903
Article Title: Mask Balancing: Perception-Driven Dynamic Visibility Enhancement for Occlusion-Capable Optical See-Through Head-Mounted Displays
Publication Date: May 1, 2026

Source: EurekAlert