Abstract:

The growing use of Large Language Models (LLMs) in user experience design and research activities brings forth various reproducibility challenges in the field of Human-Computer Interaction (HCI). This article reviews potential reproducibility issues, analogous to p-hacking and prompt-hacking in research, highlighting how LLMs could impact HCI practices.

Key Findings:

• Introduction of LLMs in HCI could exacerbate general bias and data analysis issues.
• There is an essential need for enhanced documentation and education to support responsible LLM use in HCI.
• The community might face increased pressure due to rapid advancements in technology.

Significance:

This paper discusses the dual aspects of risks and opportunities presented by LLMs in HCI research, encouraging a deeper understanding and development of best practices. It calls for a balanced approach to utilize these technologies effectively within the field.