Structuring MoE Expert Selection for Agentic Reinforcement Learning
Aligning mixture-of-experts routing with agent operations through hierarchical control during reinforcement-learning post-training.
Papers & projects
Publications
Selected work, organized by year. Every paper includes its representative figure and original resource links.
Selected publications
Full list on Google Scholar ↗I am broadly interested in problems at the intersection of Computer Vision, Computer Audition, and Machine Learning, with the goal of building AI systems that can perceive, reason, and interact with complex real-world environments. My research focuses on multimodal learning (Vision + X), particularly for generative modeling and cross-modal understanding with minimal supervision.
In the past, I have also worked on computational photography, including image reflection removal, intrinsic image decomposition, inverse rendering, and video quality assessment. This selection also includes my recent work on agents and efficient reasoning. Preprints are labeled separately from peer-reviewed publications.
2026
6 papers
RLTL;DR: Self-improvement by Internalizing Self-generated Feedback
Learning from brief, self-generated insights after failed attempts, then internalizing those insights during training.
Simulate to Generalize: Scaling Stateful Supervision for API-calling Agents using LLM World Models
Generating stateful API-calling trajectories with LLM world models to train agents that generalize to unseen environments.
AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding
A benchmark for agentic multi-speaker understanding, paired with the RAFT alignment framework.
Presentations: Research presentation · Apple booth, CVPR 2026
Uncovering the Representation Geometry of Minimal Cores in Overcomplete Reasoning Traces
Identifying the compact support within long reasoning traces and studying its representation geometry.
A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications
A survey of the techniques, applications, and open challenges in multimodal agentic frameworks.
2025
5 papers
MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks
Multi-agent retrieval and reasoning over audio-visual evidence distributed across multiple videos.
AURELIA: Test-time Reasoning Distillation in Audio-Visual LLMs
An actor–critic framework for structured audio-visual reasoning at test time, alongside AVReasonBench.
Presentations: Invited presentation · CLVL, ICCV 2025 Invited presentation · AVGenL, ICCV 2025
AVTrustBench: Assessing and Enhancing Reliability and Robustness in Audio-Visual LLMs
Evaluating audio-visual reliability under adversarial attacks, compositional reasoning, and modality-specific dependencies.
EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perception
Adaptive multisensory distillation and policy learning for efficient egocentric perception.
Presentations: Oral presentation · BinEgo-360°, ICCV 2025 Invited talk · NYC Computer Vision Day, NYU 2025
AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning
Evaluating audio-visual reasoning traces by separating factual consistency from the validity of core inferences.
2024
5 papers
Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time
Fine-grained audio-visual grounding in space and time through modality alignment and instruction tuning.
Presentations: Invited talk · AVGenL, ECCV 2024
Towards Determining Perceived Human Intent for Multimodal Social Media Posts using The Theory of Reasoned Action
Predicting perceived intent in multimodal social media posts using visual, textual, and psychological cues.
ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlations
Language-guided image augmentation to reduce dependence on spurious visual features.
Can LLM’s Generate Human-Like Wayfinding Instructions? Towards Platform-Agnostic Embodied Instruction Synthesis
Generating platform-independent wayfinding instructions with in-context learning.
MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models
Synthesizing music from image and text cues with a diffusion model and visual synapses.
Presentations: Invited talk · Sight and Sound, CVPR 2024
2023
3 papers
APoLLo: Unified Adapter and Prompt Learning for Vision Language Models
Combining adapters and prompts to improve few-shot generalization in vision-language models.
AdVerb: Visually Guided Audio Dereverberation
Using visual context to help estimate clean audio from reverberant recordings.
Presentations: Invited talk · AV4D, ICCV 2023
Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation
A measured-albedo dataset and evaluation metrics for intrinsic image decomposition.
2021
3 papers
AudViSum: Self-Supervised Deep Reinforcement Learning for Diverse Audio-Visual Summary Generation
Self-supervised reinforcement learning for diverse, informative audio-visual summaries.
V-DESIRR: Very Fast Deep Embedded Single Image Reflection Removal
A multi-scale architecture for fast single-image reflection removal.
Listen to the Pixels
Self-supervised joint audio-visual segmentation by exploiting correspondence between the modalities.
Download BibTeX for seven recent papers (arXiv records)
* Equal contribution. Author order and resource links follow the original publications.