Projects
Human Bias, Fairness, and Explainability in Human-AI Interactions
As artificial intelligence (AI) assisted information systems have become ubiquitous in workplaces and everyday lives, understanding and accounting for fairness has gained increasing attention in the design and evaluation of such systems. In this project, we not only characterizes data and algorithmic biases, but also highlights the cognitive and perceptual biases that may exacerbate system biases and lead to unfair decisions. Within the framework, we also analyze the interactions between human and system biases in human-AI interactions, especially in the context of AI-enabled chat and conversational search, intelligent recommender systems, and user-chatbot interactions.
CRII:III: A bias-aware approach to modeling users in interactive information retrieval
People often act intuitively and are subject to systematic biases when making decisions under uncertainty. The project seeks to study users’ systematic biases and leverage the learned knowledge in improving the explanatory and predicative power of IR models. The outcomes of this project can help people better leverage the power of information through 1) incorporating the knowledge about their biases into search algorithms, and 2) proactively capturing bias-related search problems and promoting informed, unbiased decision-making. The work is supported by an NSF award.
Session Level Evaluation Metric
A search session is generally comprised of several queries trying to complete a particular search task. Each query is representative of the search state of the user involved in the search task. In this project, our focus is to go beyond the traditional query level evaluation metrics and make progress towards a more holistic evaluation metric that can represent both the user satisfaction and the relevance of the documents retrieved in a search session.
Modeling Interaction between Human Biases and System Biases for Fair IR
This project aims to leverage the insights about human and algorithmic biases in developing and evaluating novel L2R methods for useful, fair and efficient large-scale information. The ultimate goal will be designing and evaluating a bias-aware information retrieval system that promote useful and fairly distributed results under varying motivating tasks. The system, once developed, will enhance the effectiveness of search interactions with limited in-situ user feedback, help address user biases in query formulation and information examination, and improve the fairness in exposure of different topics, opinions, content generators from varying backgrounds in cyberspace, which is essential for equal information access, critical thinking and quality education.
Assessing and Improving the Reusability of Research Resources in Interactive Information Retrieval
Reusing resources from existing studies is essential for algorithmic experiments, and often serves as the initial step for replication studies, which is critical for advancing scientific research. To evaluate search systems of different forms, researchers often reuse standard test collections as it provides a common empirical basis for comparison and improves the cost-effectiveness of data collection. This project seeks to develop a novel reusability assessment framework for session test collections, which are collected under complex search tasks and have been scarcely studied in previous studies. We will 1) assess test collection reusability in evaluating system performances in retrieving topically relevant and practically useful documents; 2) comprehensively examine the extent to which IIR research resources can be reused in evaluating systems developed under new tasks.
Understanding Users’ Biased Usefulness and Credibility Judgments of COVID-19 Search Results
Despite the increasing research attention towards algorithmic fairness and transparency, the interactions between boundedly rational users and biased information systems still remain largely unexplored. Also, it is critical to untangle the positive effects and negative impacts of users’ biases and heuristics in different modalities of information interactions (graphical user interface based system, text-based and voice-based conversational systems). This project will focus on the problem of COVID-19 information evaluation in Web search, and explore how users’ biases affect the way in which users evaluate the usefulness and credibility of retrieved search results on COVID-19.
Interest Development and Knowledge Construction in Search as Learning
Among many cognitive factors associated with search as learning (SAL), we consider interest development as an important dimension because it significantly affects users’ learning performances but still remains understudied in interactive IR (IIR). We develop an interest-search-learning (ISL) model to reconceptualize learning activities in information seeking and retrieval. This project seeks to achieve three goals: 1) characterizing the interactions between interest development, learning, and search behaviors; 2) synthesizing and proposing useful measures for capturing in-situ progresses and state variations in interest development and knowledge learning; 3) developing system supports to facilitate interest development and effective learning in search interactions.