My Research
My long-term research goal is to develop the science of language models (LMs) by asking and answering the following questions: What role does each hidden representation play in LLMs? When will they malfunction or cause counterintuitive phenomena? How can we adapt the mechanisms to transcend their inherent limits and better serve the downstream tasks?
To this end, I have been working on a theoretical and empirical analysis of LLM hidden representations LMs, with the aim to provide insights and develop useful tools and cures for LMs.
Awards and Honors
News, Service and Experiences
- [Oct 2024] Gave an invited talk at UIUC CS 591 MLR seminar on "Towards A Physiology of Language Model Representations".
- [Sep 2024] Gave a lightning talk at AICE Symposium on "Towards A Physiology of Language Model Representations".
- [Sep 2024] Serving as Area Chair for EMNLP 2024 Demos.
- [Sep 2024] Co-organizing "Towards Knowledgeable Foundation Models" workshop at AAAI-2025.
- [Sep 2024] Our work LM-Infinite and LM-Steer are covered by UIUC News.
- [Aug 2024] Obtained my solo paragliding certificate!
- [Fall 2024] Teaching assistant for CS 546: Advanced Topics in Natural Language Processing, with a guest talk on the science of LMs.
- [Jun 2024] I gave one talk on "LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models" at AI Time.
- [2024] Co-organizer of UIUC Spring 2024 NLP Seminar series.
- [2023-2024] BBQ Co-Chair for the UIUC AI and DAIS group.
- [Dec 2023] I gave one talk at Data Intelligence.