Hanqi Yan

I'm a PostDoc at King's college London, supervised by Prof. Yulan He, where I am focusing on robust and reliable language models.

I passed my PhD viva with no corrections after a great time in University of Warwick (2020.10-2024.04), advised by Prof. Prof. Yulan He and Dr. Lin Gui. I finished my M.S. at Peking University (2017-2020) and my B.E. at Beihang University (2013-2017).

During Ph.D., I started my Causality Journey in visiting professor Kun Zhang affiliated with Causal Learning and Reasoning Group@CMU. Before Ph.D., I started my NLP journey in visiting professor Wenjie Li affiliated Natural Language Processing Group @PolyU Hong Kong.

Email  /  CV  /  Google Scholar  /  Semantic Scholar  /  Twitter  /  Github

Strive not to be a success, but rather to be of value. -- Albert Einstein

Research Summary

My research interests lie in the intersection of Machine Learning and Natural Language Processing, i.e., incorporating fundamental representation learning to enhance the interpretability and robustness of different NLP models.

1. Founded by Representation Learning : I address the intrinsic limitations in representations learnt in Transformers structure: order sensitivity (InfoAC), dimension collapse (TokenUni); principled disentanglement (Matte) ; study the neuron-level interpretability in human-preference alignment (DecPO).
2. Extend the Impact to Practical NLP Applications
• Text classification: robust classifier enhanced by Lexicon (LexicalAT), hireachical interpretable classifier (HINT), A Lightweight Adpator for ICL (DiscAda)
• Recommendation System: GCN-based Q-learning network(GCQN), explainable recommender (GIANT)
• (Causal) Relation Extraction: weak-to-strong extractor(ReWire), knowledge-augmented graph network (KAG)
• Controllable generation: identificability-guarantee (MATTE)
• Reasoning: multiple-perspective self-reflection(Mirror)

Publication

Large Language Model

Study Model-level Monosemanticity (mechanistic Interpretability) in preference alignment process.

A weak-to-strong information extraction model that uses partial annotated data for reward model while still achieving high performance in PPO.

Introduce a Navigator model to interact with the Reasoner by providing question-specific and diverse guidance in knowledge-rich self-reflection process without any supervision.

We attribute the order sensitivity of CausalLMs to the auto-regressive attention masks, which restrict each token from accessing information from subsequent tokens. Thereby leading to our proposed consistencey-based representation learning method in addressing this vulnerability of LLMs.

From Macro perspective, Why In-Context Learning can learn Different Algorithms without gradient descent, e.g, Regression, Bayesian.

We observed the pitfalls of LLMs in detecting and intervening the implicit sentiment, so we provide Identification guarantees for successful disentanglement of the content and style variables. This principled representations can shed light on the llm alignments, i.e., safe and moral generation.

Self-Explainable Models

To ease the humman annotation for rationales in Recommender, a prior from user-item interactions is incorporated into the textual latent factors for explaination generation.

Unsupervised self-explanatory framework for document classification. It can extract word-, sentence-, and topic-level rationales explaining the document-level decision.

Robustenss

Commonsense Knowledge, i.e., ConceptNet is applied as invariant feature to tackle the distribution shift and Position Bias.

Provide Identification guarantees for successful disentanglement of the content and style variables, further supports the intervention of latent attributes of the text. This principled representations can shed light on the constrained, i.e., safe and moral generation for large language models with noisy pertaining data.

Token uniformity implies more vanished dimensions in the embedding space. _SoftDecay_ is proposed to a range of transformer-based language models and improved performance is observed in STS evaluation and a range of GLUE tasks.

Token uniformity issue is still observed in in-context learning, we proposed an adaptor for more discriminative representation learning and improved performance is observed in fine-grained text classification tasks.

Professional Activities

Event Organiser: Co-Chair of AACL-IJCNLP (Student Research Workshop) 2022

Reviewers for NLP: AACL24,EACL23', EMNLP22',23'24, ACL23'24', NAACL24',

Reviewers for ML and AI: Neurocomputing, Knowledge and Information System, TOIS, UAI23', AISTATS24'25', NEURIPS24', ICLR25'

Invited Talks

UC San Diego, NLP Group, 02/2024. Robust and Interpretable NLP via representation learning and Path Ahead

Yale University, NLP Group 01/2024. Robust and Interpretable NLP via representation learning and Path Ahead

Turing AI Fellowship Event, London, 03/2023, Distinguishability Calibration to In-Context Learning

UKRI Fellows Workshop, University of Edinburgh, 04/2022. Interpreting Long Documents and Recommendation Systems via Latent Variable Models

Basically
Blog Posts

Feel free to steal this website's source code. Do not scrape the HTML from this page itself, as it includes analytics tags that you do not want on your own website — use the github code instead. Also, consider using Leonid Keselman's Jekyll fork of this page.