Hanqi Yan

I'm a PostDoc at King's college London, supervised by Prof. Yulan He, where I am focusing on robust and reliable language models.

I passed my PhD viva with no corrections after a great time in University of Warwick (2020.10-2024.04), advised by Prof. Prof. Yulan He and Dr. Lin Gui. I finished my M.S. at Peking University (2017-2020) in Academy for Advanced Interdisciplinary Studies and my B.E. at Beihang University (2013-2017) in School of Instrumentation and Optoelectronic Engineering.

During Ph.D., I started my Causality Journey in visiting professor Kun Zhang affiliated with Causal Learning and Reasoning Group@CMU. Before Ph.D., I started my NLP journey in visiting professor Wenjie Li affiliated Natural Language Processing Group @PolyU Hong Kong.

Email  /  CV  /  Scholar  /  Twitter  /  Github

Strive not to be a success, but rather to be of value. -- Albert Einstein

Research

I study interpretable and robust NLP models:

• In the LLM era, I focus on constrained planning (search) in the model decoding/fine-tuning phases, to achieve safe, reliable and fair generation.
• Align the models' behaviors with human knowledge in self-explanatory framework through probablistic perspective.
• Propose empirical and principled methods to alleviate spurious correlation and learn robust representations under data distribution shifts.
• Address the intrinsic limitations of Transformer structure for better robusteness and efficiency.

Publication

Large Language Model

We observed the pitfalls of LLMs in detecting and intervening the implicit sentiment, so we provide Identification guarantees for successful disentanglement of the content and style variables. This principled representations can shed light on the llm alignments, i.e., safe and moral generation.

Introduce a Navigator model to interact with the Reasoner by providing question-specific and diverse guidance in knowledge-rich self-reflection process without any supervision.

We attribute the order sensitivity of CausalLMs to the auto-regressive attention masks, which restrict each token from accessing information from subsequent tokens. Thereby leading to our proposed consistencey-based representation learning method in addressing this vulnerability of LLMs.

From Macro perspective, Why In-Context Learning can learn Different Algorithms without gradient descent, e.g, Regression, Bayesian.

Self-Explainable Models

To ease the humman annotation for rationales in Recommender, a prior from user-item interactions is incorporated into the textual latent factors for explaination generation.

Unsupervised self-explanatory framework for document classification. It can extract word-, sentence-, and topic-level rationales explaining the document-level decision.

Robustenss

Commonsense Knowledge, i.e., ConceptNet is applied as invariant feature to tackle the distribution shift and Position Bias.

Provide Identification guarantees for successful disentanglement of the content and style variables, further supports the intervention of latent attributes of the text. This principled representations can shed light on the constrained, i.e., safe and moral generation for large language models with noisy pertaining data.

Token uniformity implies more vanished dimensions in the embedding space. _SoftDecay_ is proposed to a range of transformer-based language models and improved performance is observed in STS evaluation and a range of GLUE tasks.

Token uniformity issue is still observed in in-context learning, we proposed an adaptor for more discriminative representation learning and improved performance is observed in fine-grained text classification tasks.

Professional Activities

Event Organiser: Co-Chair of AACL-IJCNLP (Student Research Workshop) 2022

Reviewers: EMNLP22',23', NAACL24', EACL23', AACL24', ACL Rolling Review,UAI23', AISTATS24', Neurocomputing, TOIS

Basically
Blog Posts

Feel free to steal this website's source code. Do not scrape the HTML from this page itself, as it includes analytics tags that you do not want on your own website — use the github code instead. Also, consider using Leonid Keselman's Jekyll fork of this page.