Lei Chen

Computer Vision Group

Overview

Multi-sensor perception and generalizable scene understanding

The Computer Vision Group studies how intelligent systems can perceive, integrate, and understand complex environments from heterogeneous sensory observations. Our research focuses on multi-sensor and multimodal perception, including RGB, depth, thermal imaging, video, audio, point clouds, and robot-state information, with particular emphasis on building coherent representations across different sensing modalities, spatial scales, and temporal contexts.

Our work spans object- and entity-level perception, 360-degree scene and video understanding, cross-modal fusion and alignment, 3D spatial perception and reconstruction, and audio-visual intelligence. Beyond recognizing isolated visual content, we aim to enable AI systems to continuously organize heterogeneous sensory signals into unified, persistent, and actionable representations of the physical world.

Our long-term goal is to develop general-purpose perceptual intelligence that can seamlessly integrate heterogeneous sensors, understand dynamic 3D environments, and provide reliable world representations for embodied agents operating in complex open-world environments.

Research areas

  • Multimodal Perception and Robotic Data. Understanding objects and surrounding scenes through entity segmentation, 360-degree video segmentation and tracking, and the integration of language, vision, depth, point clouds, and robot-state data. We are particularly interested in building object-centric and scene-centric representations that can support downstream robotic perception and embodied reasoning. [E-SAM, Leader360V]
  • Cross-Modal Fusion and Alignment. Learning robust correspondences across heterogeneous sensing modalities, including RGB, thermal, depth, language, audio, and geometric observations. We study feature-, semantic-, and representation-level fusion and alignment to exploit complementary information while remaining robust to modality gaps, sensor degradation, and changing environments.
  • 3D Spatial Understanding and Scene Reconstruction. Recovering geometry and spatial structure from images, panoramic observations, videos, depth sensors, and point clouds. Our research explores panoramic depth estimation, 3D scene reconstruction, spatial correspondence, geometry-aware perception, and dynamic scene modeling toward persistent and metrically grounded representations of real-world environments.
  • Audio-Visual and Multisensory Intelligence. Understanding environments and human activities by jointly reasoning over visual, acoustic, and temporal signals. We investigate audio-visual correspondence, sound-source understanding, speech and affect perception, event recognition, and multisensory scene interpretation, with the goal of extending visual perception toward richer environment-aware intelligence.