Liwei (Levi) Che

I am currently a PhD student in Computer Science at Rutgers University, advised by Prof. Vladimir Pavlovic. I also work closely with Prof. Ruixiang Tang from Rutgers and Prof. Ranjay Krishna from UW. My recent work focuses on improving the general-purpose capabilities and safety of LLMs / MLLMs, with particular interests in hallucination mitigation, visual reasoning, and post-training methods.

I previously worked as an Applied Scientist Intern at both Amazon Retail and Amazon Ads, where I was fortunate to be mentored by Wenyi Wu, Ming Wang, and Samarth Gupta. My internship projects focused on MLLM-based embedding retrieval and large-scale training of generative retrieval models for personalized ads recommendation with contribution to Amazon production systems.

Research Interest

Visual reasoning and mechanistic interpretability for vision-language models
LLM / MLLM safety alignment and post-training
Hallucination detection and mitigation in LVLMs
Earlier work on federated, multimodal, and semi-supervised learning

News

Jan 2026	I received the Rutgers DCS PhD Fellowship.
Sep 2025	I was awarded an ICCV travel grant.
Aug 2025	I was selected for the ICCV Doctoral Consortium.
Jun 2025	One paper was accepted by ICCV 2025 on LVLM object hallucination detection and mitigation.
May 2025	Started my internship at Amazon Ads as an Applied Scientist II, working on generative retrieval models.
May 2024	One paper was accepted by ECML PKDD on distributed learning with multimodal foundation models.
May 2024	Started my internship at Amazon as an Applied Scientist II, working on MLLM for embedding-based retrieval.

Selected Work

2026

arXiv

Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models

Liwei Che, Zhiyu Xue, Yihao Quan, and 7 more authors

2026

under review

Paper

2025

ICCV

Hallucinatory Image Tokens: A Training-free EAZY Approach to Detecting and Mitigating Object Hallucinations in LVLMs

Liwei Che, Tony Qingze Liu, Jing Jia, and 3 more authors

2025

ICCV 2025

Paper Code Website

2024

ECML-PKDD

Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modality

Liwei Che, Jiaqi Wang, Xinyue Liu, and 1 more author

In ECML PKDD, 2024

Paper Code

2023

NeurIPS

Towards Personalized Federated Learning via Heterogeneous Model Reassembly

Jiaqi Wang, Xingyi Yang, Suhan Cui, and 4 more authors

2023

Paper
Sensors

Multimodal Federated Learning: A Survey

Liwei Che, Jiaqi Wang, Yao Zhou, and 1 more author

Sensors, 2023

Paper Abs

Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL studies focus on taking unimodal data, such as image and text, as the model input and resolving the heterogeneity challenge, i.e., the challenge of non-identical distribution (non-IID) caused by a data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been conducted to improve system performance utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic of multimodal federated learning (MFL) and present a literature review on the state-of-art MFL methods. Furthermore, we categorize multimodal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research.
IEEE TSMC

FedStream: Prototype-Based Federated Learning on Distributed Concept-Drifting Data Streams

Cobbinah B. Mawuli, Liwei Che, Jay Kumar, and 4 more authors

IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023

Paper

2021

IEEE BigData

FedTriNet: A Pseudo Labeling Method with Three Players for Federated Semi-supervised Learning

Liwei Che, Zewei Long, Jiaqi Wang, and 3 more authors

In 2021 IEEE International Conference on Big Data (Big Data), 2021

Paper Code