CRII:III: A bias-aware approach to modeling users in interactive information retrieval
About the Project
People often act intuitively and are subject to systematic biases when making decisions under uncertainty due to their inability to calculate all the possible consequences of their choices - a fundamental cognitive phenomenon called "bounded rationality". Without proactive information supports, these decisions could be driven by misleading information, cognitive biases and heuristics and may result in significant deviations from desired outcomes: Health information seekers may easily trust medical misinformation that confirms their existing expectations. Students often heavily rely on top ranked results and stop at short satisficing answers, rather than exploring more credible and informative Web pages. Online shoppers tend to quickly accept immediate mediocre recommendations after encountering several bad quality products (with low reference levels), without examining all available options. By investigating users’ systematic biases, this project aims to break new grounds for information retrieval (IR) research and address fundamental bottlenecks in the development of bias-aware search systems. The outcomes of this project can help people better leverage the power of information through 1) incorporating the knowledge about their biases into search algorithms, and 2) proactively capturing bias-related search problems and promoting informed, unbiased decision-making.
The project seeks to study users’ systematic biases and leverage the learned knowledge in improving the explanatory and predicative power of IR models. The technical aims of the project include: (1) understanding the relationships between search interactions and users’ systematic biases; (2) building bias-aware prediction models of search interactions; (3) developing a scalable and potentially transformative approach to modeling users and their decision-making processes under biases in interactive IR. To achieve these goals, the investigator will conduct a series of user studies and experiments. First, the research team will carry out controlled lab studies to examine the associations between users’ search interactions and several major systematic biases that have been widely confirmed by behavioral experiments, including reference dependence, framing effect, and loss aversion. Then, the team will extract new features and create bias-aware models for predicting users’ search behavior, experience, and problems. Finally, this project will apply deep neural networks in developing more fine-grained bias-aware models based on large scale test collections and search logs, and evaluate the performance of modified models in a wider range of search scenarios. The proposed models can provide a more solid behavioral and psychological basis for supporting the simulations of search interactions. Such simulations, properly constructed, could address major challenges in the design of boundedly-rational formal models and bias-aware intelligent systems.
Progresss
Fall 2021
A short paper accepted and presented at CIKM 2021.
Spring 2022
Running a lab study on user expectation in IIR.
Summer 2022
A late breaking paper presented at JCDL 2022. A paper presented at NTCIR 2022.
Summer 2022
Continuation of the experimental study on predicting in-situ user biases and designing bias-aware evaluation metrics.
Spring 2023
Starting a crowdsourcing study to investigate user bias in online searching.
Related Publications
-
Wang, B., & Liu, J. (2023) Characterizing and early predicting user performance for adaptive search path recommendation.
-
Wang, B., & Liu, J. (2023) Investigating the Role of In-situ User Expectations in Web Search. Information Processing & Management.
-
Liu, J. (2023). A behavioral economics approach to interactive information retrieval: Understanding and supporting boundedly rational users. Springer Nature.
-
Liu, J. (2023). A two-sided fairness framework in search and recommendation. In Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval. Austin, TX, USA. (CHIIR 2023)
-
Brown, T., & Liu, J. (2022). A reference dependence approach to enhancing early prediction of session behavior and satisfaction. In Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries (pp. 1-5).
-
Ghosh, S., & Liu, J. (2022). OUHCIR at the NTCIR-16 Data Search 2 Task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, June 14-17, 2022: Tokyo, Japan.
-
Liu, J., & Yu, R. (2021). State-aware meta-evaluation of evaluation metrics in interactive information retrieval. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (pp. 3258-3262).
-
Wang, B., & Liu, J. (2021). Extracting the Implicit Search States from Explicit Behavioral Signals in Complex Search Tasks. Proceedings of the Association for Information Science and Technology, 58(1), 854-856.
Presentations
-
Liu, J. (2022). Understanding boundedly rational users in task-based intelligent information retrieval. Department of Information Sciences and Technology, George Mason University, VA, USA. March 24, 2022.
-
Liu, J. (2021). Toward bias-aware user models in intelligent information retrieval. The Information School, University of Wisconsin-Madison, WI, USA. October 29, 2021.
-
Liu, j. (2021). User-centered information retrieval evaluation and meta-evaluation. School of Information Resources Management, Renmin University of China, October 22, 2021.
-
Liu, J. (2021). Modeling boundedly rational users in interactive information retrieval. College of Information, University of North Texas, July 19, 2021.
-
Liu, J. (2021). Investigating reference dependence effects on user search interaction and satisfaction. Information & Innovation Lab, University of South Australia, Adelaide, Australia, May 4, 2021.
About the Project
People often act intuitively and are subject to systematic biases when making decisions under uncertainty due to their inability to calculate all the possible consequences of their choices - a fundamental cognitive phenomenon called "bounded rationality". Without proactive information supports, these decisions could be driven by misleading information, cognitive biases and heuristics and may result in significant deviations from desired outcomes: Health information seekers may easily trust medical misinformation that confirms their existing expectations. Students often heavily rely on top ranked results and stop at short satisficing answers, rather than exploring more credible and informative Web pages. Online shoppers tend to quickly accept immediate mediocre recommendations after encountering several bad quality products (with low reference levels), without examining all available options. By investigating users’ systematic biases, this project aims to break new grounds for information retrieval (IR) research and address fundamental bottlenecks in the development of bias-aware search systems. The outcomes of this project can help people better leverage the power of information through 1) incorporating the knowledge about their biases into search algorithms, and 2) proactively capturing bias-related search problems and promoting informed, unbiased decision-making.
The project seeks to study users’ systematic biases and leverage the learned knowledge in improving the explanatory and predicative power of IR models. The technical aims of the project include: (1) understanding the relationships between search interactions and users’ systematic biases; (2) building bias-aware prediction models of search interactions; (3) developing a scalable and potentially transformative approach to modeling users and their decision-making processes under biases in interactive IR. To achieve these goals, the investigator will conduct a series of user studies and experiments. First, the research team will carry out controlled lab studies to examine the associations between users’ search interactions and several major systematic biases that have been widely confirmed by behavioral experiments, including reference dependence, framing effect, and loss aversion. Then, the team will extract new features and create bias-aware models for predicting users’ search behavior, experience, and problems. Finally, this project will apply deep neural networks in developing more fine-grained bias-aware models based on large scale test collections and search logs, and evaluate the performance of modified models in a wider range of search scenarios. The proposed models can provide a more solid behavioral and psychological basis for supporting the simulations of search interactions. Such simulations, properly constructed, could address major challenges in the design of boundedly-rational formal models and bias-aware intelligent systems.
Progresss
Fall 2021
Spring 2022
Summer 2022
Summer 2022
Spring 2023
Related Publications
- Wang, B., & Liu, J. (2023) Characterizing and early predicting user performance for adaptive search path recommendation.
- Wang, B., & Liu, J. (2023) Investigating the Role of In-situ User Expectations in Web Search. Information Processing & Management.
- Liu, J. (2023). A behavioral economics approach to interactive information retrieval: Understanding and supporting boundedly rational users. Springer Nature.
- Liu, J. (2023). A two-sided fairness framework in search and recommendation. In Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval. Austin, TX, USA. (CHIIR 2023)
- Brown, T., & Liu, J. (2022). A reference dependence approach to enhancing early prediction of session behavior and satisfaction. In Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries (pp. 1-5).
- Ghosh, S., & Liu, J. (2022). OUHCIR at the NTCIR-16 Data Search 2 Task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, June 14-17, 2022: Tokyo, Japan.
- Liu, J., & Yu, R. (2021). State-aware meta-evaluation of evaluation metrics in interactive information retrieval. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (pp. 3258-3262).
- Wang, B., & Liu, J. (2021). Extracting the Implicit Search States from Explicit Behavioral Signals in Complex Search Tasks. Proceedings of the Association for Information Science and Technology, 58(1), 854-856.
Presentations
- Liu, J. (2022). Understanding boundedly rational users in task-based intelligent information retrieval. Department of Information Sciences and Technology, George Mason University, VA, USA. March 24, 2022.
- Liu, J. (2021). Toward bias-aware user models in intelligent information retrieval. The Information School, University of Wisconsin-Madison, WI, USA. October 29, 2021.
- Liu, j. (2021). User-centered information retrieval evaluation and meta-evaluation. School of Information Resources Management, Renmin University of China, October 22, 2021.
- Liu, J. (2021). Modeling boundedly rational users in interactive information retrieval. College of Information, University of North Texas, July 19, 2021.
- Liu, J. (2021). Investigating reference dependence effects on user search interaction and satisfaction. Information & Innovation Lab, University of South Australia, Adelaide, Australia, May 4, 2021.