Nishit Anand

I am a final-year MS CS student at the University of Maryland, College Park. My research focuses on Multimodal LLMs, AI Agents, and Audio understanding models under the guidance of Prof. Dinesh Manocha in the GAMMA Lab and Prof. Ramani Duraiswami in the PIRL Lab.

Before my Master's, I worked as a Research Scientist at Radien, a Seattle-based AI startup funded by the Allen Institute for AI (AI2). There, I built agents to predict visual similarity in UI elements and intelligently simplify codebases. I was also the Founding Research Engineer at Ananas Labs, a Bengaluru-based startup founded by a former Staff Research Scientist at Google Research India. I developed our low-latency Indic-SpeechLLM, optimizing it for our app using model quantization, and researched a novel multilingual phoneme-based tokenizer to reduce token inefficiency in Indic languages.

Prior to my industry roles, I was a Research Fellow at the Vision and Graphics Lab, IIT Delhi, under Prof. Chetan Arora. Working on a government-funded project, I created state-of-the-art Vision Transformer-based OCR models for 13 Indian languages, deploying them with inference optimization and caching techniques for higher throughput. Earlier, as a Research Engineer at the Autonomous Networked Systems Lab at IIIT-Delhi under Prof. Saket Anand, I led the Driver Status Monitoring (DSM) module of a $300K government-funded autonomous driving project. I trained and optimized facial landmark detection and object detection models for deployment on our custom Android app and NVIDIA Jetson devices.

I completed my B.Tech (Honours) in Computer Science from Jaypee Institute of Information Technology Noida (JIIT Noida) in 2022, graduating in the top 5% of the department and scoring the highest grades in all final-year courses. I have also had the privilege of collaborating with Prof. Salman Khan, Prof. Mohamed Elhoseiny, and Prof. Ruohan Gao, among other exceptional mentors.

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn

News

Research

A novel line-level OCR approach that processes text in lines rather than words, improving accuracy and context understanding.

A comprehensive benchmark for evaluating audio-language models across diverse audio understanding tasks.

A novel benchmark for evaluating voice assistants on multimodal interactions and complex reasoning tasks.

The first benchmark for evaluating hallucinations in egocentric video understanding models.

Test-time reasoning techniques to improve state-of-the-art Audio-Visual LLMs through knowledge distillation.

Proposes RobustCLAP, a novel technique to improve audio-language representations that are robust to linguistic variations and paraphrases in text queries, improving text-to-audio retrieval performance by up to 13%.

A novel task-specific prompt ensemble method that improves zero-shot audio classification performance by 16%.

An inference-time approach to reduce verbatim memorization in LLMs using activation steering techniques.

A deep learning approach for automatic video summarization that identifies and extracts the most relevant segments from videos.

Proposes a Time Series Characteristic-based Hurst Diffusion Model for influence maximization in online social networks, achieving up to 590% higher influence spread compared to baseline models.

A real-time system that detects guns and knives from CCTV footage using YOLOv4, achieving 90.35% accuracy and instantly notifying authorities via SMS and WhatsApp to prevent crimes.

A deep learning-based approach for detecting deepfake images and videos using advanced neural network architectures.

Website template from Jon Barron