I am a PhD student at Beihang University, supervised by Prof. Xianglong Liu. I am currently focused on the practical application of large AIGC models such as LLM and Diffusion, including inference acceleration and model quantization. I hope that advanced technological methods can truly bring convenience to peopleโ€™s lives.

๐Ÿ“ Publications

arxiv
sym

How Good Are Low-bit Quantized LLaMA3 Models? An Empirical Study

Wei Huang*, Xudong Ma*, Haotong Qin, Xingyu Zheng, Chengtao Lv, Hong Chen, Jie Luo, Xiaojuan Qi, Xianglong Liu, Michele Magno

Github Hugging Face

  • This paper explores LLaMA3โ€™s capabilities when quantized to low bit-width, demonstrating its value in advancing future models.
arxiv
sym

Towards Accurate Binarization of Diffusion Model

Xingyu Zheng*, Haotong Qin*, Xudong Ma, Mingyuan Zhang, Haojie Hao, Jiakai Wang, Zixiang Zhao, Jinyang Guo, Xianglong Liu

Github

  • This paper proposes BinaryDM, a novel accurate quantization-aware training approach to push the weights of diffusion models towards the limit of 1-bit.
ICML 2024
sym

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention

Haotong Qin*, Xudong Ma*, Xingyu Zheng, Xiaoyang Li, Yang Zhang, Shouda Liu, Jie Luo, Xianglong Liu, Michele Magno

Github

  • This paper proposes a novel IR-QLoRA for pushing quantized LLMs with LoRA to be highly accurate through information retention.
ACM MM 2023
sym

DIsolation and Induction: Training Robust Deep Neural Networks against Model Stealing Attacks

Jun Guo, Xingyu Zheng, Aishan Liu, Siyuan Liang, Yisong Xiao, Yichao Wu, Xianglong Liu

Github

  • This paper proposes Isolation and Induction (InI), a novel and effective training framework for model stealing defenses.

๐ŸŽ– Honors

  • 2024, Excellent Undergraduate Graduation Thesis.
  • 2024, Excellent Graduate of Beihang University.
  • 2022, Lanqiao Cup, National First prize.

๐Ÿ“– Educations

  • 2024 - now, PhD student, Beihang University.
  • 2020 - 2024, Undergrad student, Beihang University.