Sanjoy Chowdhury

I am a second year CS PhD student at University of Maryland, College Park advised by Prof. Dinesh Manocha. Prior to this, I was working as a Machine Learning Scientist with the Camera and Video AI team at ShareChat, India. I was also a visiting researcher at the Computer Vision and Pattern Recognition Unit at Indian Statistical Institute Kolkata under Prof. Ujjwal Bhattacharya.

Previously, I was a Senior Research Engineer with the Vision Intelligence Group at Samsung R&D Institute Bangalore. I primarily worked on developing novel AI-powered solutions for different smart devices of Samsung.

I received my MTech in Computer Science & Engineering from IIIT Hyderabad where I was fortunate to be advised by Prof. C V Jawahar. During my undergrad, I worked as a research intern under Prof. Pabitra Mitra at IIT Kharagpur and at the CVPR Unit at ISI Kolkata under Prof. Ujjwal Bhattacharya.

Email  /  GitHub  /  Google Scholar  /  LinkedIn  /  Twitter

Selected publications

My research is at the intersection of Computer vision, deep learning with a focus on multi-modal learning (Vision + X), generative modeling, visual understanding, and their various applications. I'm broadly interested in studying the interplay between different modalities with minimal supervision.

APoLLo : Unified Adapter and Prompt Learning for Vision Language Models

Our method is designed to substantially improve the generalization capabilities of VLP models when they are fine-tuned in a few-shot setting. We introduce trainable cross-attention-based adapter layers in conjunction with vision and language encoders to strengthen the alignment between the two modalities.

AdVerb: Visually Guided Audio Dereverberation

We present a novel audio-visual dereverberation framework that uses visual cues in addition to the reverberant sound to estimate clean audio.

Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation

In order to comprehensively evaluate albedo, we collect a new dataset, Measured Albedo in the Wild (MAW), and propose three new metrics that complement WHDR

AudViSum: Self-Supervised Deep Reinforcement Learning for Diverse Audio-Visual Summary Generation

Introduced a novel deep reinforcement learning-based self-supervised audio-visual summarization model that leverages both audio and visual information to generate diverse yet semantically meaningful summaries.

V-DESIRR: Very Fast Deep Embedded Single Image Reflection Removal

We have proposed a multi-scale end-to-end architecture for detecting and removing weak, medium, and strong reflections from naturally occurring images.

Listen to the Pixels

In this study, we exploited the concurrency between audio and visual modalities in an attempt to solve the joint audio-visual segmentation problem in a self-supervised manner.

Blog(s)

Have tried my hand at writing technical blogs.

The devil is in the details: Video Quality Enhancement Approaches

The blog contextualizes the problem of video enhancement in present-day scenarios and talks about a couple of interesting approaches to handle this challenging task.

Academic services

I have served as a reviewer for the following conferences:

CVPR: 2023

ICCV: 2023

WACV: 2022, 2023, 2024

ACMMM: 2023