Abstract: Data-driven deep learning has achieved state-of-the-art performance in computer vision by relying on large, carefully curated datasets containing clean, high-quality images. In real-world deployment, models frequently encounter images captured under challenging conditions such as uneven illumination, viewpoint changes, and motion blur which results in performance degradations. In this thesis, we adopt a data-informed approach and present methods to leverage data-based insights and modeling to improve robustness in deployable computer vision systems.

We first propose a space-variant motion blur augmentation method that synthesizes realistic blurred images using binary foreground maps derived from semantic segmentation annotations. We demonstrate improved semantic segmentation performance on real motion-blurred datasets compared to training on clean images alone. To remove the dependency on manual annotations, we next leverage a state-of-the-art promptable segmentation foundation model to generate foreground masks. We show that these automatically generated masks effectively replace manual annotations and improve robustness, extending the approach to general scene understanding tasks. Going further, to improve the realism of camera motion blur, we subsequently propose a depth-consistent augmentation strategy that accounts for scene geometry rather than approximating camera motion as space-invariant. The resulting synthetic data better matches the distribution of real blurred images and improves robustness across multiple scene understanding tasks.

We next investigate a practical application in sports, where fast-moving balls exhibit translational and rotational blur with significant scale variation. We develop a classical computer-vision-based pipeline for automatic ball annotation from videos captured using low-cost consumer-grade cameras. Our pipeline enables automatic generation of annotations to augment existing supervised datasets with multiple scales and views of the ball.

Finally, building on our depth-consistent camera motion blur modelling, we propose a weakly supervised camera motion deblurring framework which leverages only camera motion trajectories as supervision during training. Our trained model achieves significant zero-shot deblurring performance on real motion-blurred images while using only a fraction of the training data required by supervised approaches highlighting the underexplored benefits of data-informed modeling.

Event Details
Title: Data-Informed Motion Blur Modeling for Resilience in Dynamic Scene Understanding Tasks (PhD Viva Voce)
Date: July 10, 2026 at 04:00 PM
Venue: Google Meet (https://meet.google.com/jjt-ycrm-ksh)
Speaker: Ms. Aakanksha (EE18D405)
Guide: Dr. Rajagopalan A N
Type: PHD seminar

Updated: