Salman Khan

Salman Khan

Faculty at MBZUAI. Working on the crossroads of Vision, Language, and Learning.

I am an Associate Professor at MBZUAI and a member of the Intelligent Visual Analytics Lab (IVAL). Previously, I was a Senior Scientist at Inception Institute of Artificial Intelligence and a Research Scientist in the Computer Vision Research Group (CVRG) at Data61, CSIRO (Commonwealth Scientific and Industrial Research Organization). I am also an adjunct faculty member at College of Electrical & Computer Science (CECS), Australian National University (ANU) since 2016. I was a research visitor at NICTA (National ICT Australia) in 2015 and also served as a research affiliate with Australian Center for Robotic Vision (ACRV).

I work on model generalization, multi-modal learning, learning from limited data (zero and few-shot learning), and continual life-long learning systems for computer vision problems. The above-mentioned tasks can help us realize intelligent autonomous systems that can better understand the real world for improved recognition, detection, segmentation, and detailed scene comprehension. I am also interested in the applications of vision and learning algorithms to earth observation and climate science.

Positions Available! I am looking for exceptional candidates for two tracks: a) PostDocs, b) Research engineers at MBZUAI. PostDoc candidates are preferred to have a background in AI4Climate, Remote Sensing or Multimodal models. Research engineers must demonstrate strong development skills via past projects (background in GIS and remote sensing, AI4Climate is a plus). Candidates are expected to have good grades in their BS/MS programs and relevant background in AI/computer vision/machine learning (demonstrated through relevant coursework and past projects). Prior publication record in CVPR/ECCV/ICCV/NeurIPS/ICLR/ICML is a plus. If you are interested, please email me your CV and a link to your GitHub profile.

🔥 News

  • 🎉 Congratulations to MBZUAI students Hashmat and Huzaifa on winning the Best Student Paper Runner-Up Award at ACCV’24 for our ObjectCompose work!
  • Our BiMedix project, led by Dr. Hisham, has won the inaugural Llama Impact Innovation Award!
  • Four papers accepted at NeurIPS’24 and ICLR’25!
  • Serving as an AC for CVPR’24, ICML’24 and ECCV’24.
  • Seven papers accepted at CVPR’24! Congratulations to all students and collaborators.
  • Seven papers accepted at ICCV’23! Congratulations to all team members.
  • Acting as an Area Chair for CVPR’23, ICML’23 and NeurIPS’23.
  • Nine papers accepted to CVPR’23! Congratulations to all students and collaborators.
  • Co-organizing workshops on “Vision Transformer: Theory and Applications” at ACCV’22 (5th Dec ’22) and NeurIPS’22 (9th Dec ’22).
  • Two papers accepted to NeurIPS’22! Congratulations to all students and collaborators.
  • Acting as an Area Chair for NeurIPS’22 and ACML’22.
  • Five papers accepted to ECCV 2022! [Jul ’22]
  • Six papers accepted to CVPR 2022 (including 3 Orals)! Congratulations to all students and collaborators. [Mar ’22]
  • Our paper on improving adversarial transferability of ViTs accepted in ICLR’22 as a spotlight (rate 5%). Congratulations Muzammal and Kanchana!
  • 🎉 Glad to share that my Ph.D. student, Dr. Shafin Rahman, has been awarded the J.G. Crawford Prize for best thesis at ANU (Science Category). Congratulations Shafin!
  • Organizing TPAMI Special Issue on “Transformers in Vision.”

đź“ť Publications

(Selected publications, full list can be found on Google Scholar)

ACL 2024
ACL 2024

Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models

M. Maaz, H. Rasheed, S. Khan, F. S. Khan (ACL 2024)

PDF Code Presentation

CVPR 2024
CVPR 2024

GLaMM: Grounding Large Multimodal Model

H. A. Rasheed, M. Maaz, S. S. Mullappilly, A. M. Shaker, S. Khan, H. Cholakkal, R. M. Anwer, E. P. Xing, M.-H. Yang, F. S. Khan (CVPR 2024)

PDF Code Page

CVPR 2024
CVPR 2024

GeoChat: Grounded Large Vision-Language Model for Remote Sensing

K. Kuckreja, M. S. Danish, M. Naseer, A. Das, S. Khan, F. S. Khan (CVPR 2024)

PDF Code

ICLR 2024
ICLR 2024

LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts

H. Gani, S. F. Bhat, M. Naseer, S. Khan, P. Wonka (ICLR 2024)

PDF Code

NeurIPS 2023
NeurIPS 2023

PromptIR: Prompting for All-in-One Image Restoration

V. Potlapalli, S. W. Zamir, S. Khan, F. Khan (NeurIPS 2023)

PDF Code

CVPR 2023
CVPR 2023

MaPLe: Multi-modal Prompt Learning

M. U. Khattak, H. A. Rasheed, M. Maaz, S. Khan, F. S. Khan (CVPR 2023)

ArXiv Project Page Code Video Slides

🎖 Honors and Awards

  • 2024 Winner of Meta’s Llama Impact Innovation Awards for “BiMediX2: A Bilingual Medical LMM” (PI: Dr. Hisham).
  • 2024 PI, Silal and ADQ funded project: “AgroCastGPT: AI-Based Yield Forecasting and Recommendation Platform”.
  • 2024 Co-PI, UAE Rain Enhancement Program by National Center for Meteorology: “Identification of clouds microphysical seed-ability” (PI: Prof. D. Rosenfeld).
  • 2023 Google Research Award – Co-PI on “A Climate Change and Sustainability Tailored Arabic LLM” (with Dr. H. Cholakkal & Dr. R. Anwer).
  • 2023 Listed among top 0.4% scientists in AI (rank 1212/356,955) by Stanford University’s top 2\% list.
  • 2024 Best Student Paper Runner-Up – ACCV 2024 for “Object-Compose: Evaluating Resilience of Vision-Based Models”.
  • 2022 Best Paper Finalist – CVPR 2022 (Burst Image Restoration and Enhancement; top 33/8161).
  • 2021 NTIRE Image Enhancement Challenge – 2nd rank in CVPR-NTIRE 2021 (Dual-Pixel Defocus Deblurring).
  • 2021 Outstanding Reviewer – CVPR 2021.
  • 2020 Best Student Paper – ICPRAM 2020.
  • 2019 NTIRE 2019 Challenge – 2nd rank (Image Enhancement) & 3rd rank (Real Image SR).
  • 2019 Outstanding Reviewer – CVPR 2019; Outstanding & Emergency Reviewer – ICCV 2019.

🙌 People

I have been fortunate to work with amazing students, colleagues and collaborators. Below is a non-exhaustive list.

Students

Current

  • Muhammad Maaz (MS@MBZUAI, 2020-2022; PhD@MBZUAI, 2022-)
  • Hanoona Abdul Rasheed (MS@MBZUAI, 2020-2022; PhD@MBZUAI, 2022-)
  • Muhammad Ali (PhD@MBZUAI, 2020-)
  • Hashmat Shadab Malik (MS@MBZUAI, 2020-2022; PhD@MBZUAI, 2022-)
  • Muhammad Sohail (PhD@MBZUAI, 2023-)
  • Akashah Shabbir (PhD@MBZUAI, 2023-)

Former

  • Sameera Ramasinghe (PhD@ANU, 2018-2022) (joined Amazon)
  • Muzammal Naseer (PhD@ANU, 2017-2022) (joined MBZUAI)
  • Moshiur Farazi (PhD@ANU, 2016-2020) (joined Data61, CSIRO)
  • Shafin Rehman (PhD@ANU, 2016-2020) (Awarded J.G. Crawford Prize for the best thesis at ANU) (joined North South University)
  • Lin Li (PhD@ANU, 2017-2022) (joined Sensetime)
  • Hanif Rasyidi (PhD@ANU, 2017-2022) (joined Canberra University)
  • Abass Bamidele Abdulsalam (MS@MBZUAI, 2020-2022)
  • Rushali Grandhe (MS@MBZUAI, 2020-2022) (joined OurCrowd)
  • Abdelrahman Mohamed (MS@MBZUAI, 2020-2022) (joined MBZUAI)
  • Muhammad Uzair Khattak (MS@MBZUAI, 2021-2023) (joined Max Planck Institute)
  • Zhang Sheng (MS@MBZUAI, 2021-2023)

Research Interns/Engineers/Visitors