Huichi Zhou

Huichi Zhou is currently a graduate student at Imperial College London with YangLab advised by Professor Guang Yang. Currently, I collaborate with Professor Jun Wang and Dr. Linyi Yang.

My research interest is broadly in Adversarial Machine Learning and Large Language Models.

Email: h.zhou24 [at] imperial.ac.uk

Github     Google Scholar

We introduce RadVUQA, a novel Radiological Visual Understanding and Question Answering benchmark, to comprehensively evaluate existing LVLMs.

ACIR (Anomaly-aware Content-based Image Recommendation) is a medical image retrieval framework that enhances accuracy and robustness through multi-level feature fusion (DaRF), structure-aware contrastive hashing (SCH), self-supervised OOD detection, and content-guided ranking. It achieves up to 38.9% improvement in mAP on anatomical datasets over existing methods.

We introduce TrustRAG, a robust Retrieval-Augmented Generation (RAG) framework. It defends against corpus poisoning attacks by a two-stage mechanism: identifying potential attack patterns with K-means clustering and detecting malicious docs via self-assessment.

We introduce VFF, which is a dataset and training framework that improves the format-following ability of 7B-level LLMs using Python-based verification and progressive training. It enhances LLMs’ ability to follow specific formats (e.g., JSON) through self-generated data and direct preference optimization (DPO).

LLM4RGNN is a framework that enhances the adversarial robustness of Graph Neural Networks (GNNs) using Large Language Models (LLMs). It distills the inference capability of GPT-4 into a local LLM to identify malicious edges and trains an LM-based edge predictor to find missing important edges. By removing malicious edges and adding important ones, LLM4RGNN significantly improves GNN performance under adversarial attacks, outperforming existing robust GNN frameworks in various datasets and attack scenarios.

AVLLM is a framework for evaluating and improving the validity of word-level adversarial attacks on LLMs. It fine-tunes a lightweight LLM to provide a validity score and explanation, helping to generate semantically consistent adversarial examples. AVLLM improves attack quality and consistency through enhanced semantic understanding.

GUI-WORLD is a comprehensive video-based benchmark and dataset designed for GUI-oriented multimodal understanding. It includes 12,379 GUI videos covering six scenarios (e.g., software, websites, mobile, XR) and eight question types. The dataset captures dynamic and sequential GUI content, addressing challenges faced by existing models. The paper introduces GUI-VID, a fine-tuned Video LLM that improves dynamic GUI understanding but highlights that current models still struggle with complex GUI tasks.

MLLM-as-a-Judge is a benchmark designed to evaluate the judging capabilities of Multimodal Large Language Models (MLLMs) in vision-language tasks. It assesses MLLMs on Scoring Evaluation, Pair Comparison, and Batch Ranking across 14 datasets. Results show that while MLLMs align well with human judgments in pair comparison, they struggle with scoring and batch ranking due to biases and hallucinations. GPT-4V performs best, but the study highlights the need for improvements in consistency and reasoning.